{"id":7238,"date":"2011-06-01T14:53:43","date_gmt":"2011-06-01T18:53:43","guid":{"rendered":"http:\/\/www.raleighpublicrecord.org\/?p=7238"},"modified":"2012-03-30T14:43:58","modified_gmt":"2012-03-30T18:43:58","slug":"will-evaas-make-wake-schools-better-part-ii","status":"publish","type":"post","link":"https:\/\/theraleighcommons.org\/raleighpublicrecord\/news\/2011\/06\/01\/will-evaas-make-wake-schools-better-part-ii\/","title":{"rendered":"Will EVAAS Make Wake Schools Better? Part II"},"content":{"rendered":"<p><em>This story is the second in a two-part series. <a href=\"http:\/\/www.raleighpublicrecord.org\/news\/2011\/05\/23\/will-evaas-make-wake-schools-better-part-one\/\" target=\"_blank\">Read Part 1 here.<\/a><\/em><\/p>\n<p>The Wake County Public School System, like other districts in North Carolina, uses the Education Value-Added Assessment System (EVAAS) to rate its teachers.<\/p>\n<p>But EVAAS has drawn fire from education academics, teachers and teacher organizations in the state and around the country.<\/p>\n<p>Despite these criticisms, Wake Superintendent Tony Tata is already attempting to use EVAAS to staff struggling schools and classrooms with teachers deemed effective by the program.<\/p>\n<p>Chief among EVAAS\u2019s critics is Audrey Amrein-Beardsley, assistant professor at Arizona State University, who has published the most comprehensive studies articulating problems with the program.<\/p>\n<p>The <a href=\"http:\/\/www.ncae.org\/\">North Carolina Association of Educators<\/a> (NCAE), the state\u2019s largest teacher organization, has also heard complaints from educators across the state and documented abuses of EVAAS.<\/p>\n<p>Critics say EVAAS:<\/p>\n<ul>\n<li>Does not take into account a student\u2019s background<\/li>\n<li>Has been used as a punishment<\/li>\n<li>Has not been sufficiently assessed in peer-reviewed scholarly journals<\/li>\n<li>Is confusing for many teachers<\/li>\n<li>Encourages teaching to the test<\/li>\n<\/ul>\n<p><strong>Student Background<\/strong><\/p>\n<p>Just before the last school board election in 2009, a report surfaced that criticized WCPSS\u2019s in-house system for rating its teachers &#8211; the Effectiveness Index.<\/p>\n<p>William Sanders, developer of EVAAS and senior director of the EVAAS K-12 division at <a href=\"http:\/\/www.sas.com\/\">SAS<\/a>, helped author the report.<\/p>\n<p>SAS criticized the Effectiveness Index for adjusting student performance data by considering socioeconomic factors.<\/p>\n<p>\u201cFor a rich white kid and a poor black kid we should expect the same level of academic achievement and want the same expectation for both kids next year,\u201d Sanders said in a phone interview with The Record.<\/p>\n<p>The report seemed to lend some credence to conservative school board contenders \u2014 who went on to win that October \u2014 looking to do away with the policy of assigning students to schools based on socioeconomic status.<\/p>\n<p>Of course, the new conservative board majority accomplished this goal in 2010. But many education academics believe socioeconomic status is essential to any study of teacher effectiveness.<\/p>\n<p>Beardsley points to a preponderance of data showing socioeconomic background predicts a student\u2019s performance in school. Poorer kids tend to do worse academically than their more affluent counterparts.<\/p>\n<p>\u201cThe real question here is \u2026 do students from, let\u2019s say, low socioeconomic backgrounds learn as fast as students who have two parents, who travel, who have tutors outside of the classroom perhaps, who have homework priorities, who read as part of their leisure time, etc.?\u201d Beardsley wrote in an e-mail to The Record.<\/p>\n<p>If so, she said, then EVAAS \u2014 \u201cthe only sophisticated analytical model for measuring student achievement that does not account for student background factors\u201d \u2014 will show inherent biases against teachers of low-income students and in favor of teachers of affluent students.<\/p>\n<p>The current school year marks the third full year that WCPSS has used EVAAS. Wake County Superintendent Tony Tata has repeatedly said that he wants to use EVAAS data to pair high-performing teachers with low-performing students.<\/p>\n<p>William Sanders, developer of EVAAS, claims that he has evidence that teachers who move from one environment \u2014 affluent and high-achieving \u2014 to a radically different environment \u2014 low-income and underachieving \u2014 generally continue to do just as well in EVAAS terms.<\/p>\n<p>[pullquote]\u201cThe real question here is \u2026 do students from, let\u2019s say, low socioeconomic backgrounds learn as fast as students who have two parents, who travel, who have tutors outside of the classroom perhaps, who have homework priorities, who read as part of their leisure time, etc.?\u201d~Audrey Amrein-Beardsley[\/pullquote]<\/p>\n<p>But Sanders offers no proof of this claim.<\/p>\n<p>Whether teachers perform well with varying sets of students at new schools may depend on how the matches are made.<\/p>\n<p>\u201cThere should be some kind of perk,\u201d said Angela Farthing, manager of the Center for Teaching and Learning of the NCAE. \u201cJust because a teacher is successful at Panther Creek High School and Enloe High School doesn\u2019t mean they\u2019ll be successful at Halifax County Schools. Involuntary transfer is not going to help. It takes all the staff at a school to be successful. If you transfer a teacher to a school, take a high-performing teacher from one school and put them in schools that are not high-performing, that may not be the only piece to help turn the school around.\u201d<\/p>\n<p><strong> \u201cHammer Instead of a Carrot\u201d<\/strong><\/p>\n<p>As potential data for teacher evaluations, EVAAS ratings are protected as confidential personnel information under state law. But the law has not always been respected.<\/p>\n<p>Farthing cites Guilford County, where EVAAS teacher ratings were posted in school hallways, as an example of districts using EVAAS to harm instead of help.*<img loading=\"lazy\" title=\"gcs_evaas_correction\" src=\"http:\/\/www.raleighpublicrecord.org\/wp-content\/uploads\/2011\/06\/gcs_evaas_correction-243x300.jpg\" alt=\"\" width=\"243\" height=\"300\" align=\"right\" \/><\/a><\/p>\n<p>Also under state law, EVAAS ratings can be part of a teacher\u2019s evaluation but cannot be used as the sole basis for termination.<\/p>\n<p>Yet throughout the state, said Farthing, administrators imply or directly state that decisions about termination are linked to testing data.<\/p>\n<p>\u201cThe reason that [EVAAS] has gotten a negative connotation is that principals say that if your scores are bad, you\u2019re gone,\u201d she said.<\/p>\n<p>Using EVAAS data alone \u2014 apart from observations, artifacts and other effectiveness measures \u2014 to make decisions about a teacher\u2019s future goes against scientific standards, Beardsley said.<\/p>\n<p>\u201cGiven this standard [of multiple measures] was developed by the American Educational Research Association, American Psychological Association and the National Council on Measurement in Education, those arguing, not evidencing, that these models are so strong that they can indeed operate alone and in isolation are essentially giving the best measurement specialists and researchers in the field an ideological middle finger,\u201d Beardsley wrote.<\/p>\n<p>Sanders said EVAAS was never designed to be used in isolation.<\/p>\n<p>Wake County uses EVAAS data as part of the teacher evaluation process, but Assistant Superintendent for Evaluation and Research David Holdzkom denied the district uses the program to punish teachers.<\/p>\n<p>Farthing knows of no instances of misuse of EVAAS data in Wake County. But Tata has expressed his willingness to publish the effectiveness ratings of individual teachers.<\/p>\n<p>\u201cI know if my kid was in school and you were a highly effective teacher, I would want to know,\u201d said Tata, after a press briefing in April.<\/p>\n<p>\u201cIt\u2019s being used as a hammer instead of a carrot,\u201d Farthing said. \u201cStudents are more than test scores. We\u2019re focusing more on the test scores than on helping students be learners and thinkers.\u201d<\/p>\n<p>Some education academics question the assumptions programs like EVAAS make about teachers.<\/p>\n<p>\u201cI\u2019m not sure the discussion about good teachers is even a useful one,\u201d said Jason Osborne, associate professor of educational psychology at North Carolina State University. \u201cI\u2019m sure the vast majority of teachers are doing their darnedest to do what\u2019s right for students. It may be that it\u2019s not the right environment. It may be that the students aren\u2019t coming to the table and allowing teachers to help them.<\/p>\n<p>\u201cWe need to get away from seeing teachers as heroes and villains,\u201d he added. \u201cTeachers are only half the story, or less than half.\u201d<\/p>\n<p>EVAAS is password-protected and accessed online. Other than board members and staff in the offices of the superintendent and the Evaluation and Research Department, the only school personnel that have access to EVAAS are principals and any designees that a principal chooses, such as department chairs. Teacher ratings are not visible to other teachers, but student ratings are, making it simple to pair a student\u2019s rating with his or her teacher.<\/p>\n<p>&nbsp;<\/p>\n<p><img loading=\"lazy\" class=\"aligncenter size-full wp-image-7242\" title=\"evaas_english\" src=\"http:\/\/www.raleighpublicrecord.org\/wp-content\/uploads\/2011\/06\/evaas_english.jpg\" alt=\"\" width=\"600\" height=\"330\" srcset=\"https:\/\/theraleighcommons.org\/raleighpublicrecord\/wp-content\/uploads\/2011\/06\/evaas_english.jpg 600w, https:\/\/theraleighcommons.org\/raleighpublicrecord\/wp-content\/uploads\/2011\/06\/evaas_english-336x184.jpg 336w\" sizes=\"(max-width: 600px) 100vw, 600px\" \/><\/a><\/p>\n<p><strong>Peer Review<\/strong><\/p>\n<p>Multiple reports on EVAAS have noted the shortage of available data on the program in respected academic journals.<\/p>\n<p>Sanders points to the U.S. Department of Education\u2019s approval of Tennessee\u2019s growth model, which uses a version of EVAAS, as evidence the program has been fully vetted and validated. But Beardsley and other researchers do not have findings that this evidence adequate.<\/p>\n<p>\u201cBecause these [value-added models] certainly make sense, and nearly everyone agrees that these models are improving the current accountability system, our federal and state policymakers \u2026 have essentially gone \u2018all in\u2019 without nearly enough evidence that they work in the ways intended, and for which they are being oversold,\u201d Beardsley wrote.<\/p>\n<p>Moreover, researchers have questioned the reliability of EVAAS, complaining that in some cases teachers are rated effective one year and ineffective the next.<\/p>\n<p>Sanders acknowledges that with less data EVAAS\u2019s ratings become less reliable. He says that with only one or two years of data available for a teacher, the margin of error can be as high as 50 percent.<\/p>\n<p>\u201cThis simply isn\u2019t good enough to make consequential decisions,\u201d Beardsley wrote. Even with three years of data, the district can only be 80 to 90 percent certain that the ratings are accurate.\u201d<\/p>\n<p><strong>Teacher Confusion<\/strong><\/p>\n<p>Farthing believes that to use EVAAS correctly and to its full potential, teachers and principals have to completely understand it. Impossible, she said, as long as SAS maintains its algorithms as proprietary secrets.<\/p>\n<p>\u201cI have seen presentation after presentation [about EVAAS] and I couldn\u2019t tell you how it all works,\u201d Farthing said. \u201cIt\u2019s something that statisticians understand, but the average educator and the average parent does not understand.\u201d<\/p>\n<p>Farthing believes EVAAS works best when principals allow access to all of their teachers.<\/p>\n<p>Sanders counters that the complexity of EVAAS is its strength.<\/p>\n<p>\u201cThe No. 1 huge disadvantage of more simple value-added models is that they throw away data that they don\u2019t have for kids,\u201d he said.<\/p>\n<p>EVAAS compensates for missing data due to students entering the district late or missing tests, problems that disproportionately affect poorer children.<\/p>\n<p>\u201cI shudder to think how many places have adopted that simple approach [of other value-added models] without realizing how grossly inaccurate they are,\u201d Sanders said.<\/p>\n<p><strong>Teaching to the Test<\/strong><\/p>\n<p>Since EVAAS uses end-of-grade and end-of-course test data, some argue the program pushes teachers to focus only on content and skills featured on those exams.<\/p>\n<p>\u201cSome policy makers say that\u2019s what we should be doing,\u201d said Farthing, who used to teach social studies. \u201cBut for a long time the curriculum was a mile wide and an inch deep.\u201d<\/p>\n<p>The North Carolina Standard Course of Study is the state\u2019s curriculum for most subjects. This curriculum has been improving, Farthing said, but the standardized tests are still lackluster.<\/p>\n<p>\u201cThe U.S. History EOC is an awful test,\u201d she said. \u201cRecall a constitutional amendment or a paper from The Federalist Papers. It\u2019s just a recall test.\u201d<\/p>\n<p>A focus on testing makes for uninspired teachings, Farthing believes.<\/p>\n<p>\u201cTo teach to the test means you have to practice the test,\u201d Farthing said. \u201cStudents get tired and bored.\u201d<\/p>\n<p>Some teachers \u2014 those in grades K-2 and those not teaching core subjects \u2014 do not have to deal with the standardized tests. This puts unfair pressure on teachers of higher grades and subjects like reading and math, Farthing said.<\/p>\n<p>She acknowledges that EVAAS has proved useful in student math placement.<\/p>\n<p>\u201cOne thing I\u2019ve heard [from teachers] is that it does help them if they know a student is going to need help in algebra,\u201d she said.<\/p>\n<p><em>bleclaire@raleighpublicrecord.org<\/em><\/p>\n<p>&nbsp;<\/p>\n<p><strong>Read more about EVAAS pros and cons in these related studies and published papers: <\/strong><\/p>\n<p><script type=\"text\/javascript\" src=\"http:\/\/s3.documentcloud.org\/viewer\/loader.js\"><\/script><br \/>\n<script type=\"text\/javascript\">\/\/ <![CDATA[\n  DV.load('http:\/\/www.documentcloud.org\/documents\/97252-beardsleyevaas.js', {\n    width: 600,\n    height: 500,\n    container: \"#DV-viewer-97252-beardsleyevaas\"\n  });\n\/\/ ]]><\/script><\/p>\n<p><script type=\"text\/javascript\" src=\"http:\/\/s3.documentcloud.org\/viewer\/loader.js\"><\/script><br \/>\n<script type=\"text\/javascript\">\/\/ <![CDATA[\n  DV.load('http:\/\/www.documentcloud.org\/documents\/97250-etsvalueaddedcritique.js', {\n    width: 600,\n    height: 500,\n    container: \"#DV-viewer-97250-etsvalueaddedcritique\"\n  });\n\/\/ ]]><\/script><\/p>\n<p><script type=\"text\/javascript\" src=\"http:\/\/s3.documentcloud.org\/viewer\/loader.js\"><\/script><br \/>\n<script type=\"text\/javascript\">\/\/ <![CDATA[\n  DV.load('http:\/\/www.documentcloud.org\/documents\/97251-sanders-wright-response-to-amrein-beardsley-4-14.js', {\n    width: 600,\n    height: 500,\n    container: \"#DV-viewer-97251-sanders-wright-response-to-amrein-beardsley-4-14\"\n  });\n\/\/ ]]><\/script><\/p>\n","protected":false},"excerpt":{"rendered":"<p>The Wake County Public School System, like other districts in North Carolina, uses the Education Value-Added Assessment System (EVAAS) to rate its teachers. But EVAAS has drawn fire from education academics, teachers and teacher organizations in the state and around the country. Despite these criticisms, Wake Superintendent Tony Tata is already attempting to use EVAAS to staff struggling schools and classrooms with teachers deemed effective by the program.<\/p>\n","protected":false},"author":24030,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[15,55],"tags":[251,123],"_links":{"self":[{"href":"https:\/\/theraleighcommons.org\/raleighpublicrecord\/wp-json\/wp\/v2\/posts\/7238"}],"collection":[{"href":"https:\/\/theraleighcommons.org\/raleighpublicrecord\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/theraleighcommons.org\/raleighpublicrecord\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/theraleighcommons.org\/raleighpublicrecord\/wp-json\/wp\/v2\/users\/24030"}],"replies":[{"embeddable":true,"href":"https:\/\/theraleighcommons.org\/raleighpublicrecord\/wp-json\/wp\/v2\/comments?post=7238"}],"version-history":[{"count":0,"href":"https:\/\/theraleighcommons.org\/raleighpublicrecord\/wp-json\/wp\/v2\/posts\/7238\/revisions"}],"wp:attachment":[{"href":"https:\/\/theraleighcommons.org\/raleighpublicrecord\/wp-json\/wp\/v2\/media?parent=7238"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/theraleighcommons.org\/raleighpublicrecord\/wp-json\/wp\/v2\/categories?post=7238"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/theraleighcommons.org\/raleighpublicrecord\/wp-json\/wp\/v2\/tags?post=7238"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}