California English Journal

 
 

CONTENTS

Summer 2005

NO MERIT IN THIS MERIT PAY PLAN
-Angus Dustan

MERIT PAY:A CATCH 22
-Michael Mahoney

SCHOOL REFORM:DIVIDED WE FALL
-Jerry Treadway

A LETER TO GOVERNOR SCHWARZENEGGER
-Ken Elliott

MY GOVERNATOR LET ME DOWN
-Mark Srorer

A LETTER TO GOVERNOR SCHWARENEGGER
-Danielle Vargas

TEACHER PERFORMANCE AND PAY
-Adrienne Mack-Kirschner

THE GOVERNATOR RESPONSE:SOLUTIONS TO THE SORRY STATE OF TEACHING
-Alfee Enciso

CAN A TEN-MINUTE READING ASSIGNMENT DELIVER ON ITS CLAIMS?
-Derek Boucher

Artist of this issue - photography students at Santa Monica H.S.
under the direction of David Woods

Features & Departments

Breiger's Bookshelf

TOP

No Merit in this Merit Pay Plan
By Angus Dunstan

In his State of the State address on January 5th 2005 and later in his February 12th weekly radio address, Governor Schwarzenegger proposed to tackle California’s education problems by linking teachers’ pay to performance and student achievement. Inevitably, many teachers have been appalled by his proposal, including many of the hard-working and meritorious teachers the Governor claims to want to reward. Michael Mahoney, a Sacramento teacher of English and journalism, wrote a clever and satirical response to the Governor’s proposal. Published in The Sacramento Bee on Sunday February 6th, Mahoney’s response takes the form of an imaginary evaluation interview between a teacher, Mr. Yossarian, and Principal Principal.

The principal has no–one to help him with teacher evaluation so he resorts to using student scores on standardized tests. But in spite of the fact that Yossarian’s school’s API has gone up again (to 710), the Superintendent has determined that no-one will be eligible for a raise until they meet the new school target of 730. To make the reference to Joseph Heller’s Catch 22 complete, this scene ends with Yossarian confessing that he loves teaching, even with the “lesson planning and the after-school tutoring and the grading I do on weekends, plus the time I spend coaching the Mathletes,” and the Principal firing him because “anybody would have to be crazy to love working that hard” for Yossarian’s pay and because “I can’t let crazy people near children.”

Coming from an experienced high school teacher, Mahoney’s satire strikes at the heart of the merit pay argument, exposing the absurdity that will inevitably accompany any attempt to quantify something like good teaching. It is not absurd for the Governor and the people of California to expect good teaching or even to demand it and to reward it. It’s just that merit pay is not the way to go about it. There is no doubt that some teachers are not very good, that a few of us are actually quite dreadful, and if we could all be better our students would be better off. But it is important to see the merit pay argument for what it is – not an honest attempt to improve teaching in California, but a cynical distraction from the real problems facing us as we try to do what we’re asked to do.

In formulating his response to what he calls “the educational disaster” in California, I have to think that the Governor learned a lesson from President Bush’s response to the social security crisis. In both cases everyone agrees that there’s a problem and that something must be done. But just as the President offers a “solution” that does not address the real social security problem (at some time in the future there will not be enough income to pay the promised benefits), so the Governor offers a solution that misses the point about our education system, that the affluent schools really do quite well while the poorer ones usually do not. Just as personal retirement accounts might be a reasonable response to the problem that Americans don’t save enough, so merit pay for teachers might be a response to the problem that many teachers just don’t try hard enough because there are no financial incentives. But that’s not the problem. As the Governor himself concedes, there are a great many “wonderful and dedicated teachers” out there.

There are many obvious and complex and well documented reasons why our education system is not better, including:

  • California’s low per-pupil funding;
  • Proposition 13’s devastating effect on schools in low tax-base neighborhoods;
  • The deep inequality between poor and wealthy school districts;
    The lack of a clear career ladder for classroom teachers;
  • The shortage of qualified teachers;
  • The extraordinary demands placed on the system by unprecedented large numbers of English learners;
  • The high teacher turnover in low-performing schools;
  • The deleterious effects of inappropriate, time-consuming and costly testing programs.

By refusing to address these problems and choosing to start “in the classroom ….. with those who hold our children’s learning in their hands,” the Governor made it clear that the “educational disaster” he likes to refer to is the teachers’ fault. The implication of his approach is that if we expel (his word) ineffective teachers, all will be well. If there are really a lot of these ineffective teachers, we’d need to have a serious plan to replace them, and if there are only a few, why base an entire education initiative on trying to get rid of them? The Governor also warned us that his struggle to change things would be a “battle of the special interests versus the children’s interests.” So individual teachers can be excellent, mediocre or bad, but if we unite to oppose the Governor, we become special interests.

I confess I do have a special interest, namely to point out a couple of serious problems with tying teachers’ pay to merit. Even if you think merit pay is a good idea in principle, that it would encourage teachers to work harder and smarter, it will not work in California for three reasons:

1. There will never be enough money in the merit pot to make it worthwhile. The Governor has already said that California has a spending problem, not a revenue problem. If spending needs to be cut, merit pay increases will never be sufficiently attractive to motivate extraordinary effort.

2. There will never be enough administrative resources to assess merit fairly. Like Yossarian’s Principal Principal, most administrators find that their resources for faculty evaluation are being cut rather than supplemented. Providing the mentoring and evaluation systems that a comprehensive merit pay system requires would be a very expensive addition to the current school budget.

3. The criteria for judging merit will almost certainly include student test scores and will always be open to dispute. It is clearly absurd to assess teachers’ merit on measures over which they have only limited control.

Similar problems were raised a few years ago by Richard Rothstein in an article published by the American School Board Journal. Rothstein found, for example, that business corporations “do not generally evaluate professional employees by quantifiable goals, such as test scores” and merit plans “more frequently use team incentives, not individual ones.” He describes Wal-Mart’s merit pay practices which require “each employee to negotiate an annual career development plan with a supervisor” and he refers to the Edison Project, a for-profit company that runs schools. Their merit-based compensation plans take into account multiple measures of merit and involve the whole school. Edison also makes small performance-based adjustments to teachers’ salaries but these increases average only about 1% of pay. As Rothstein notes: “Such small amounts … are unlikely to provide much incentive for teachers to improve performance.” They constitute a token “thank you.”

We lived through an ill-conceived and ill-fated experiment with merit pay at CSU Sacramento a few years ago. A faculty member who wished to be considered for a Performance Salary Step Increase (known as a “pissie”) would first put together a file listing everything accomplished since the last review that might demonstrate merit. In this way, the committee one served on because it was part of one’s professional responsibility now became a source of points in the merit review process. Likewise the number of hours spent advising students, the number of conference papers delivered, the number of meetings attended, etc. And since there was never enough money to reward everyone adequately, my department also engaged in ranking applicants for merit raises, meaning that in order to rank higher than my colleagues, and thus increase my chance of a bigger slice of the merit pie, it was in my interest to inflate the importance of everything I had done and devalue what they had done. It was not a pretty sight, it did nothing to improve faculty morale and it did absolutely nothing to improve instruction. What it did was to make everyone conscious about documenting the things one would do as a normal part of one’s job so that they could be “counted” later.

If the Governor were serious about improving education in California there are many things he could propose that would merit our enthusiastic support – such as addressing any of the problems I mention in my fifth paragraph. But whatever he does or tries to do he has to have the support of classroom teachers. Sadly, by approaching the problem as he has done, the Governor makes it clear that his goal is not actually to improve instruction for California’s students, but to further his reputation as a politician willing to take on the special interests. Our special interest is the teaching of language arts to an incredibly diverse group of students. When the Governor proposes something that will help us do this, we should support him. Until then, we should not be sidetracked by proposals that lack merit.

About the Author
Angus Dunstan is a professor in the Department of English at California State University, Sacramento

TOP

School Reform: Divided We Fall
by Jerry Treadway

Generally, large organizations are likened to enormous ships that once moving in a certain direction are hard to turn around. Yet in spite of the difficulty, at least the extent of the ship is known and all of the parts move together. When thinking about school systems or the educational profession, though, an entirely different structure comes to mind.

Rather than a single ship, I see many canoes all loosely tethered together with a little twine. Extending the metaphor let’s imagine that his loose confederation has been ordered to institute a new rowing reform. And in this instance, this new rowing technique will be assessed by having everyone connected with the state’s schools row from Los Angeles(San Pedro Harbot) to the Catalina Islands.

Since California has six million plus students, let’s begin by putting six million canoes in the water. Augment that with about 400,000 canoes for teachers, and additional ones for district superintendents, assistant superintendents, coordinators, principals and vice principals, the State School Board and local boards, and the State Department of Education. In sum, the expectation is that over six and a half million canoes will converge on the San Pedro Harbor. A daunting thought.

Previously, the legislature in its wisdom and right had signed legislation instituting the new rowing procedure and designated April 1st as the day for students to demonstrate their competence. All of the participants were directed to be in the water “in and about San Pedro Harbor at 7:00 in the morning and to be ready to go.”

Many thought that this reform might work. It carried the unanimous vote of the legislature, had been reinforced by State School Board policy, notice and directives were sent to all of the schools, standards had been developed for rowing, a testing system put in place, and an accountability system set up to winnow out “evildoers.”

On the designated morning, only about half of the rowers showed up. Later, some said that they didn’t hear about the directive. Others said that they didn’t believe in the new rowing procedure and felt secure in disobeying the policy while others lost their way simply trying to locate San Pedro Harbor.

As expected with any statewide legislation on education, a large group of teachers and others deplored the whole exercise. They were sure that the act of rowing was simply another instance of the oppressors oppressing the oppressed. Instantiating their belief was the fact that, to their right, they noticed that some of the more affluent students had rigged sails to their canoes. The teachers thought that the sails gave the affluent students a considerable advantage and proved that persons forced to row without the help of sails were being purposely kept down by the establishment. They implored certain caucuses in the legislature to demand the inclusion of sails. After a little debate, the legislation was passed and sails were quickly procured and placed on all canoes. In no time at all, the harbor was awash with ubiquitous white sails.

Some of the more experienced sailors stated that the students and others should receive some instruction on how to properly use the sails. They noted that the direction and speed of the wind greatly affected performance and that a lack of knowledge could prove dangerous in high winds and seas. They also noted that at times when the wind stalled, rowing was the only way to proceed howsoever oppressive.

In opposition, the teachers and others were unanimous that instruction would only rob the sailors of “discovering” how to sail. Surely, one could only learn through trial and error. They noted that the motivation instilled by this “rich experience” would be the necessary inducement to learn the needed skills. Specific instruction, they said, was as deplorable as having to row in the first place.

Then some educators realized that providing similar sails to all the rowers achieved equality, but failed to achieve equity. Those that could row at 50 strokes per minute still had a competitive advantage over those that rowed at 5 strokes per minute. Something had to be done. The answer was to give larger, more powerful sails to those who were least proficient at rowing.

Some who heard this sounded a cautionary note. They said that while they believed in equity and that additional help ought to be given to students who needed it, this was a case where the proposed remedy didn’t fit the purpose. Rowing and sailing required different skills and while the largest and most powerful sails had the potential to provide significant help, they didn’t help students learn how to row. Yet, the idea was so psychologically sexy and the opportunity for positive social engineering so great that the lack of consonance between the act and the stated goal got lost and the deed was done. The largest sails were placed on the canoes of those expected to have the least achievement when rowing.

At 7:00 the horn sounded. Yet the desired harmonious, syncopated, sound of oars moving through the water in communal unison was not heard. Alas, a cacophony of banging and clanging and raised voices punctuated the air. In the melee, the tethers were broken and almost immediately everyone was more or less on their own. Some of the more proficient rowers began rowing at 50 strokes per minute, while others were at 20 strokes per minute, and still others only five or two. It seemed for a while that more effort was expended avoiding other canoes than in achieving the goal of rowing out of the harbor into open water.

Not even bothering to row, the phonics emphasis and whole language folks used their oars to whack each other and did so with a noticeable ferocity and glee. The same happened between the English Only and Bilingual advocates. One could hear “Take that!” in many languages.

Not fifteen minutes into the exercise, the State School Board recognizing that all was not going well and that events had become chaotic, called an emergency meeting. The School Board quickly set new policy for the achievement of the rowers. Advanced rowers were those who reached the islands. Proficient rowers were those who were “out of sight” from those on land. Basic rowers were those who reached the end of the harbor. Below basic were those who left land at any distance, even if they were there only by din of an out going tide. Far Below Basic were those who still clung to shore. Yet in spite of these new standards, just as the teachers had predicted, there was no appreciable difference in the quality of rowing.

Meanwhile, the students, largely on their own but supported by a few teachers and an occasional administrator, were struggling to get their canoes out of the harbor and into open water. The students asked each other what they were supposed to do. Some thought that they were supposed to row at a continuous rate of 25 strokes per minute. When asked where they were supposed to go, they said it didn’t matter as long as they kept rowing at 25 SPM. Some asked each other about the sails. They said that they noticed that sails had been placed on the canoes, but weren’t sure what they were to do with them because they had never been told. By new experience, though, some mentioned that when the wind shifted the sails swung menacingly to the other side of the boat and the passengers needed to duck quickly in order not to get bashed by the sail as it careened by. Other students said that the objective was to reach the islands, but they didn’t know how to get there because they were never given a map. Others said they didn’t care and some only wanted to know what was for lunch.

In time, a considerable number of students made their way out of the harbor and into open water where a whole new set of conditions and experiences awaited them. The knowledgeable and confident moved ahead swiftly. The confused and fearful were in deep trouble.

While the students were struggling to get out of the harbor, still clinging to shore were the whole language, phonics emphasis, English Only, and Bilingual advocates who lay exhausted in their canoes, each proud of the blows they had delivered to the other side. Close by, the administrators were holding meetings. The main topics were funding and accountability measures. The idea, of course, was how to hold teachers and others accountable, not themselves. Each group, upon their own analysis, was quite sure that they were part of the solution and that it was the other groups that posed the problem. And while blame was being thrown around, they wanted to make sure that the parents and the communities in which the students lived should also receive their share. In education, where scarcity is common, blame is one thing found in surfeit.

Those watching the exercise from land seemed to be of two minds. The first group noted the chaos, confusion and the number of boats that had crashed into each other and the number that had capsized. While watching the process, they often reminded each other of Jefferson’s notion that “the masses are asses.” The second group, buoyed by their new clout that resulted in having sails placed on the canoes, was far more sanguine. They were overjoyed by the commonality of the experience, the community of scholars that was being developed, the “frankness of the discussion between competing groups,” and the authentic nature of the experience. They cooed that they were witnesses to “democracy in action.”

The following month, the statisticians at the State Department of Education gave a report to the board on the activity. They noted that the five categories—Advanced, Proficient, and so on—were parallel structures that allowed for good reliability and validity scores. Their one vexation was the MAS category. MAS was a new alliteration for Missing At Sea. It had been noted with some alarm that nearly a half million canoes neither reached the islands nor came back to shore. A statistician suggested that since the MASs were “out of sight” they should be placed in the Proficient category although doing so would skew the results. After further discussion and debate, it was decided not to list them at all. The statisticians mentioned that there was precedence for doing this since they routinely ignored school dropouts in their tabulations and that the MAS group had kind of “dropped out.”

Mark Twain once stated that he was neither a humorist nor a comedian both of whom he considered shallow. Rather, he was a preacher upon which humor was his mode of sermonizing. Similarly, the preceding piece was not an attempt at metaphor or humor, but rather a way of conveying my despondency at the present state of education. Under current conditions, I see little hope of any breakthrough achievement in literacy, especially with EL students. This situation will continue so long as the profession remains as fractured as it is and as long as each side continues to make its case by cataloging the sins of the other rather than proving the merit of its own theories and approaches.

As a measure of the academic health of ELLs, I go to the web each year and take a look to see what is happening with ELLs in the Los Angeles Unified School District. Below are scores between 1998-2004 for second and fifth grade ELL students who have been in school twelve months or more.

2nd and 5th Grade Scores for ELL Students (12 Months or more in school)
in the Los Angeles Unified School District

2nd Grade/Scores
California Standardized Test (CST) (Lang. Arts)
Sat/9 & CAT/6 Scores (Reading)
Year
FBB
BB
B
PROF.
ADV.
0-25
%ile
25-50
%ile
50-75
%ile
75-99
%ile
Mean Score
1998
70
18
9
3
17
1999
68
20
10
2
18
2000
57
20
15
4
24
2001
47
21
26
6
30
2002
20
30
33
13
2
35
29
26
10
37
2003
19
26
35
16
4
50
24
19
7
27
2004
19
30
32
15
4
48
20
24
8
28
5th Grade/Scores
California Standardized Test (CST) (Lang. Arts)
Sat/9 & CAT/6 Scores (Reading)
Year
FBB
BB
B
PROF.
ADV.
0-25
%ile
25-50
%ile
50-75
%ile
75-99
%ile
Mean Score
1998
79
16
4
1
12
1999
78
16
5
1
13
2000
76
18
5
1
14
2001
73
20
6
1
15
2002
22
30
28
20
7
68
23
7
2
18
2003
23
31
37
8
1
61
26
10
3
21
2004
23
28
37
10
1
56
29
12
3
23

FBB = Far below basic, BB = Below basic, B = Basic, Prof. = Proficient, Adv. = Advanced

The 1998 and 1999 scores are the first two years of standardized testing and represent scores before the current reform programs were put in place. Certainly when the reform effort began, ELLs were not doing well. Those criticizing current reform efforts cannot use past performance as justification to return entirely to old models and methods. And while scores have improved over the past five years, to date, they have failed to reach desired levels. Second grade scores appear to have plateaued significantly below the 50th percentile and the gap between ELLs and English Only (EOs) has not been closed. So, neither those espousing old or new approaches has earned the right to judge or gloat.

Which brings us back to the beginning and the canoes. In the field of education, compromise and community, democracy’s greatest gems are hard to mine. Too often, beating mercilessly on the other as a way of attempting dominance generates greater emotional satisfaction than the more laborious, tedious task of coming together to seek agreement on particulars within an issue. Given a lack of agreement, sincerity and strong belief are often confused as truth, and political power is confused as knowledge.

School reform will not succeed because the federal government provides a little more money and many more regulations, or the legislature and state boards of education issue new dictates, or teachers are empowered, or new theories and strategies are discovered and employed, or better tests are built, or better basals are conceived or children’s authors write better books. Before achievement can be raised in any significant way, we must recognize that no person or group has all the answers and that no answers are to be found by damming others. To succeed, we are going to have to give up a portion of our selfhood (separate canoes) for community and determine that, somehow, humility will trump hubris. No group has the whole answer, but each group has something to contribute.

About the Author

Jerry Treadway is Professor of Education at California State University, San Diego

TOP

Teacher Performance and Pay
by Adrienne Mack-Kirschner

A hundred years ago teachers were pretty much doing the same thing, and being treated the same way, as they are in the 21st Century. Teaching was and in some ways still is considered blue collar work under the strong hand of the white collar manager. Academia is the only profession that clings to a tenure system, lock-step pay based on years of experience, teaching in isolation, and stresses over accountability. Perhaps because half of my adult working life I spent in a non-academic setting as a small business owner, consultant, and corporate executive, I strongly believe that teaching and learning would be enhanced by creating a performance based pay system - though not based on a single test - that allowed for differentiated pay based on multiple factors, more collaborative workplaces where products (student work) are available to everyone in order to analyze the teaching and learning that is happening, and where differentiated professional development is provided and expected.

Fortunately, some of this change is already occurring. Harvard studies indicate that the new teachers, especially those who have spent a few years in other work settings, are arriving with these expectations: differentiated pay, collaborative work places, and on-going, high quality professional development. An entire body of teachers is entering the profession with a different perspective of the role of the union—and many don’t want a union. They want to rise based on their merit, not based on a fixed contract that while it protects them, also holds them back.

What’s here now? We already have National Board Certification as one measure of teacher accomplishment. Current research provides evidence that the students of NBCTs think more deeply and critically, score 15% higher in math, and show other advancements over their peers who do not have NBCTs. The Governor should reinstate the NBCT stipend. Other states, like North Carolina which has shown the greatest gains in closing the achievement gap, pay the National Board application fees, provide quality support programs, and expect all teachers to eventually work toward certification. North Carolina and other states have raised the bar by raising the teaching standards beyond the level of the California Standards for the Teaching Profession which are closer to entry level standards then they are to accomplished teacher standards. In a system that says equal pay for any amount of work, why would one push to excel? But if we changed our pay structure, had some outcome criteria for pay, then each of us would be more inclined to work more effectively with all students.

Unpopular as it may be, we need to provide ways to bring our highest quality teachers into schools where the needs are greater. True, all students need great teachers, but it is also true that our students from academically disadvantaged settings are most in need and should receive the most assistance. Call it merit pay, combat pay, pay to acknowledge the greater challenges, whatever it takes to bring critical masses of highly accomplished teachers to our students most at risk of dropping out. If we don’t spend the extra money now, we’ll spend it by building more prisons, paying for more unemployment, dealing with underemployed broken families and displaced children. Nothing comes free and an educated population is cheap at any price.

Competitive bidding? Isn’t it about time. One LAUSD large elementary school, after they obtained their charter, went from LAUSD mandatory food services to Marriott food services and provided a greater range of meals for their students and teachers, healthier choices, AND saved $300,000 each year. This is money that went directly into teaching and learning programs. Competition with safe-guards can only improve services and I would hope that the districts themselves become competitive bidders.

We can’t expect a 19th century organization to best prepare our students for a 21st century world.

About the Author
Adrienne Mack-Kirschner, a National Board Certified Teacher, works as an education consultant focusing on clasroom instruction. Her books include National Board Certification: Unpacking the Standards and The National Board Portfolio Workbook (Heinemann); Powerful Classromm Stories From Accomplished Teachers (Cowin Press).

TOP

Can a Ten-Minute Reading Assessment Deliver On Its Claims?
By Derek Boucher

Teachers throughout California are being forced to yield their professional expertise to private commercial programs and software in the name standardization and higher test scores. On a computer list serve based in California for language arts teachers, a recent letter by an administrator forbade the faculty in his school district from deviating in any way from McGraw-Hill’s phonics program Open Court. Never mind that Appendix G of the 2000 National Reading Panel Report strongly suggests that Open Court does not help children above first grade comprehend their reading.

Garan (2002, 2004), and Coles (2003), have effectively critiqued this movement, demonstrating that the push for “scientific” programs is driven largely by biased and faulty science. And instead of empowering and equipping teachers, Allington suggests that scripted reading programs may create an environment that allows for mediocre teachers (2002). Teachers follow a script instead of reading professional literature and thinking deeply through their craft.

Many schools around the nation are purchasing Accelerated Reader and STAR test software from Renaissance Learning Inc. (NOT to be confused with California’s Standardized Testing and Reporting assessment system). In 2003, my ninth grade Reading Intervention classes (classes made up of students in my school who don’t score well on the standardized tests our state so values currently) were asked to take the STAR assessment at the beginning and end of the year to identify growth in reading ability.

The STAR Test

On their web site renlearn.com, Renaissance states:

Now, assessing students’ reading progress in grades 1–12 is fast, accurate, and easy! STAR Reading helps you determine the reading level of each student, measure individual and class growth, and forecast results on standardized tests. Students can complete the computer-adaptive assessment in less than 10 minutes, and you get accurate, reliable, norm-referenced scores immediately!


In a computer test that lasts less than ten minutes, the STAR test asks students to read passages of various lengths (from a sentence to several paragraphs). In each passage, students are asked to determine from four choices the meaning of a hi-lighted word. For example, if the hi-lighted word is “interim”, students are given four options to choose the best matching definition.

STAR Reporting Categories

Once the test is completed, the computer spits out a report explaining the test taker’s reading ability based on their STAR test performance. A simple ten-minute test “reveals” the following information about each child.

Percentile Ranking (PR) Ranging from one to ninety-nine, this number compares the students’ STAR test score with that of other students nationally at the same grade level. For example, a student who scored at the eighth percentile on the STAR test scored better than only eight percent of all others at the same grade level in the nation who took the same test. Conversely, ninety-two percent of all students who took the test did better. A ranking at the seventy-fifth percentile on the STAR test (which is fairly high) means the student scored better than seventy-five percent of all others in the nation who took the same test.

Renaissance states that percentile ranking is the best measure of a student’s reading ability relative to that of his or her peers (Score Definition Report, renlearn.com).

Independent Reading Level (IRL) IRL indicates what level students can read at independently with a high level of proficiency. For example, a 3.2 IRL is saying that the student should read books at a level of third grade, second month.

Grade Equivalent (GE) GE indicates what grade level a student is reading at. The GE compares a student’s scores with other students nationally. If a student receives a score of “3.2”, it means they are reading at a level of third grade, second month.

Comparing STAR and CAT 6

When my students’ STAR test results came back after the post-test, I was disappointed. Most of my students had independently read over one thousand pages for the year. We read expository text extensively. Yet according to the STAR report, many of my ninth grader’s Percentile Ranking (PR), Independent Reading Level (IRL) and Grade Equivalent (GE) were very low. In addition, many were currently reading at a level ranging from the second to fifth grade! Could these results really be valid after all our hard work?

Renaissance states that their STAR test “… is highly correlated to state and national standardized tests, including ITBS, CAT, SAT, and TerraNova” (renlearn.com). Since the state of California judges my school as successful or failing based on its CAT 6 test results, I felt that a comparison of the two assessments would be enlightening. If my students’ STAR data matched the CAT 6, I would be forced to accept their poor showing on the STAR test.

But how could I compare the two assessments? After a number of emails with Renaissance Learning Inc., a representative responded that it was considered valid to compare students’ CAT 6 percentile ranking with their STAR percentile ranking despite the fact that correlation data went up through eighth grade only (email. renlearn.com. 11/8/04). Now I had a basis for comparison. When I compared my students’ STAR test PR with their CAT 6 PR, I was quite surprised at the lopsided results.

Period Students with a CAT 6 Percentile Ranking higher than their STAR Percentile Ranking Average STAR Percentile Ranking for the entire class Average CAT 6 Percentile Ranking for the entire class
1 10 out of 10 8 25
2 8 out of 10 7 27
3 6 out of 7 6 20
4 7 out of 7 9 30
6 14 out of 15 8 27

Data from 2003-2004 school year

Summary
Forty-five out of forty-nine students (ninety-two percent) in my five ninth grade intervention classes scored a significantly higher average percentile ranking on their CAT 6 reading score than on their STAR PR. Five students showed no change and were not included in the forty-nine subjects.

Eight Extreme Disparities STAR Reading Percentile Ranking (PR) CAT 6 Reading Percentile Ranking (PR)
Student #1 7 50
Student #2 7 39
Student #3 3 38
Student #4 2 36
Student #5 10 53
Student #6 2 64 (!)
Student #7 10 57
Student #8 2 28

Let’s be clear about what’s going on here since the discrepant results from the two tests are rather shocking. My students took a ten-minute test defining words in context (on the STAR test), and they received relatively low percentile rankings. They also took the California CAT 6 standardized test. The reading portion of the CAT 6 takes literally hours to complete. Students read longer passages on the CAT 6 than on the STAR test, and are tested on literary elements, author’s purpose for writing, inferential meaning, passage generalizations (“what’s the best title for this selection?”), and definitions of words in context. It would seem that the CAT 6 is much more challenging and sophisticated. Thus, one would expect my students to do worse on the CAT 6 since they did so poorly on the STAR, yet the results yielded just the opposite.

In Renaissance’s favor, they state that STAR reading scores are but a “snapshot” of achievement at a specific point in time, and that other factors contribute to a student’s test scores (Score Definition Report, renlearn.com). But currently in the real world of school, districts and administrators frequently use the data in a high stakes manner to channel students into summer school or intervention classes. At the very least, the data is used to control the types of materials students are reading.

The STAR test and Grade Equivalency (GE)

I am also concerned about the psychological impact on students when they are informed of their STAR Grade Equivalency (GE). My ninth graders’ STAR GE indicated that many of them were seemingly reading at an elementary grade level. In contrast, their CAT 6 ranking indicated a reading level much higher.

One of my ninth grade students who ranked at the fifty-third percentile in reading on the CAT 6 was told by the STAR test he reads at a fifth grade level. Another ninth grader who was ranked at the sixty-fourth percentile on the CAT 6 was told by the STAR test she reads at a level of third grade, fourth month. In other words, STAR data is claiming that this student who reads better than sixty-four percent of all other students her age in the nation actually reads at a third grade level! You can imagine what labels like this do to the self-image of already fragile readers.

These two examples are consistent with many of my other students’ rankings and scores despite Renaissance’s claim of providing “…accurate, reliable, norm-referenced scores..” In addition, parents who trust their child’s school without question will receive this information and become extremely alarmed at their child’s low reading level. Unfortunately, most parents are unaware that the STAR results may fail to reflect their child’s true reading ability. After this experience with the STAR test, I am very suspicious of the latest educational programs for sale that claim to be “science based.”

I’ll conclude with these thoughts:

• Despite the slick sales presentations and miraculous claims, many of the commercial programs schools purchase simply do not deliver on their promises (Garan, 2002, 2004).
• Teachers must take back their professional rights and responsibilities. To label our children using a ten-minute computer test is an act of relinquishing our professional duty to know students well. Written response, miscue analysis, even the CAT 6 would offer a picture of the child’s reading ability without the harmful (and misleading!) labels.
• If your school uses the STAR test, check your students’ percentile rankings against their CAT 6 rankings. This is especially important if the STAR test data is used to make decisions that affect students’ educational program. Sometimes, the only advocate students have in school are their teachers.
• It’s imperative that teachers gain expertise in the area of literacy instruction. Read widely the professional literature that is available. Be critical of the slick advertisements for the latest commercial program.
• Read Garan, Coles, Allington, and Krashen in order to become aware of the political pressure and bias behind the current reading mandates being forced on schools.
• If teachers continue to yield to commercial programs that implicitly claim to be “teacher proof”, we may find ourselves out of a job someday. After all, couldn’t a technician who makes eleven dollars an hour just as easily follow a scripted program (or use software) as a teacher who makes forty dollars an hour? As state and school officials wring their hands over how much of the budget teacher salaries assume, wouldn’t less expensive options look attractive to those whose focus is the bottom line? We must think deeply about what it means to be a thoughtful professional as opposed to a technician.
• Teachers should become critically aware of what turned them into readers. This act of reflection can serve as a well from which teachers draw as they endeavor to reach students. If you are being forced into using a “cure all” commercial program ask yourself, “does this resonate with my experience of becoming a passionate and proficient reader?”
• McQuillan (1996) and Allington (2002) assert that the “literacy gap” is a rich-poor gap. Middle class kids tend to have homes and school libraries rich in high interest reading materials. Poor kids lack such access. The sad irony of it all is that commercial programs are gobbling up scarce funding that could be spent more efficiently on high interest reading material.

Allington (2002) and Darling-Hammond (1999) assert that teacher expertise is a highly significant factor in student achievement. Let’s not abdicate this dynamic relationship between student and teacher to the ineffective commercial programs currently inundating our schools. Instead, schools might initiate learning partnerships with the brilliant literacy educators at our local universities as we attempt to reach struggling readers. I would much rather trust my child to a passionate teacher who is an astute observer of students, possesses faith in the learner, and has expertise in the area of literacy instruction.

Professional References

Allington, Richard L. 2002. Big Brother and the National Reading Curriculum: How
Ideology Trumped Evidence
. Portsmouth, NH: Heinemann.

Coles, Gerald. 2003. Reading the Naked Truth: Literacy, Legislation, and Lies.
Portsmouth NH: Heinemann.

Darling-Hammond, L. 1999. Teacher Quality and Student Achievement: A Review
Of State Policy Evidence.
Seattle: Center for Teaching Policy, University of
Washington.

Garan, Elaine M. 2002. Resisting Reading Mandates: How to Triumph With the
Truth.
Portsmouth, NH: Heinemann

Garan, Elaine M. 2004. In Defense of Our Children: When Politics, Profit, and
Education Collide
. Portsmouth, NH: Heinemann.

McQuillan, Jeff, and Krashen, Stephen. 1996. The Case for Late Intervention: Once a
Good Reader, Always a Good Reader
. Culver City, California: Language
Education Associates.

Renaissance Learning, Inc. www.renlearn.com.

Report of the National Reading Panel: Teaching Children to Read. Report of the
Subgroups
. 1999. Washington, DC: National Institute of Child Health and
Human Development.

About the Author
Derek Boucher has taught for ten years at an inner city urban high school in Fresno, California. I teach history courses, as well as remedial/intervention courses in reading and language arts. I received my Masters in Secondary Reading from Fresno Pacific University in 2001. In 2003, I received the Excellence Award from the California Professors of Reading and Language Arts (which I believe is affiliated with the California Reading Association). He can be reached at debouch@fresno.k12.ca.us

TOP

Editor’s message
Carol Jago

This issue of California English has been prepared in response to Governor Schwarzeneggers’ February 12 radio address (reprinted below). CATE asked teachers to offer their solutions to the challenges we face. As I read the many manuscripts, I was moved by the extent to which teachers embrace the challenge. We are not a go along to get along group. We are professionals dedicated to improving the state of education. Teachers know in their bones that California will only truly be a golden state when no child is left behind. This issue is dedicated to the children we serve.

Transcript of Governor Schwarzenegger’s February 12, 2005 Weekly Radio Address:
Hi. This is Governor Arnold Schwarzenegger and I would like to talk a little bit to you about our education reforms. Someone once said, “Education is not the answer to the question. It is the means to the answer to all questions.”

Education is one of government’s most important duties because nowhere are the stakes higher. Our future depends on the quality of education we give our children today. This year, I propose major reforms for education to develop and reward the best teachers and give our kids every chance to excel in the classroom.

Money for schools is terribly important, there’s no two ways about it. And my budget for the coming year will spend nearly $3 billion more than last year – by far the largest spending increase for any program. But if we just spend money on the schools and say we’ve done our job, then we are shortchanging the kids.

We spend almost half of the state’s budget on education and still many of our kids are failing or dropping out. To support the kids and the teachers we need to do more. So this year, I propose reforms to invest education money more wisely, restore local control and reward excellence. The current system does not allow us to reward good teachers. My plan will do that by linking pay to performance and student achievements.

The current system limits the number of opportunities for kids to succeed in school and in life. My plan will change that by expanding charter schools and vocational education programs. The current system prevents the schools from saving money so more can be spent in the classroom. I want to change that by allowing schools to use competitive bidding for services like maintenance, transportation and food services. The schools can save $300 million and that money could go directly into the classroom.

These are big changes and the special interests are already out there trying to block true reform. This is the battle between the children’s interests and the special interests and I know which side will win. Trust me. Please join me, and together we will stop politics-as-usual and win for California the reforms our state needs. http://www.governor.ca.gov/state/govsite/gov_htmldisplay.jsp?sCatTitle=%20&sFilePath=
/govsite/spotlight/021205radio.html

TOP