Assessing the Assessment
Assessing the Assessment
By Dr. Eugene Maier
The Oregon statewide assessments, mandated by the legislature in the 1991 Oregon Education Act, are up and running. And so is the opposition. An article in the local paper tells of a group of parents who are actively opposing the tests. They say they "are full of flaws and are a waste of money," while negatively affecting the quality of educational programs by cutting into classroom instructional time and draining funds from such things as field trips and counseling programs.
On the other hand, state department officials maintain that the assessment efforts improve education by providing a comprehensive record of a students' performance, determining how well a school's curriculum is preparing students, and aiding local and state leaders in setting educational policy.
As for the math assessment, the department has developed an elaborate problem-solving scoring guide and trained scorers in its use. The problems in the assessment are intended to cover those topics listed in the state's math standards. For each problem, in addition to a score based on accuracy, the student receives a score for each of four "dimensions": conceptual understanding, process and strategies, verification, and communication.
Is the assessment actually measuring these things? A look at the sampler of student solutions and the scores may cause one to wonder. A grade 8 problem under the heading "Algebraic Relationships" states that a certain disk jockey charges a fee of $150 plus $2 per person to provide music at a dance, while a second disk jockey charges $250 plus $1 per person. The student is to "show" how many persons would need to attend the dance to make the fees the same.
In the first place, one might question whether this problem has anything to do with algebraic relationships. The first DJ initially charges $100 less then the second, but charges $1 more per person. So if a hundred people show up, his fee will be the same as the second DJ. Does one need any algebra to figure that out? And what about the four dimensions?
A student who simply showed that the two fees were the same for 100 persons got 16 points out of a possible 29. A second student first obtained a graphical solution by graphing costs against number of people for each fee structure and spotting where the two graphs intersected, then wrote an equation that was solved by painstakingly showing every textbook step of the solution, and finally wrote a page of explanation about what they did. That student got 28 out of 29. (They lost a point on communication, which was judged to be "thoroughly developed," which is worth 5 points rather than the 6 garnered for "enhanced.")
Looking at these responses and others in the sampler, one might conclude that the assessment, rather than fostering concept learning and problem solving, breeds redundancy, verbosity, textbook tedium, and that stock in trade of the accomplished test-taker: blathering (to put it politely).
So, who's to tell? Are the assessments accomplishing something of value, or are they a waste of time and money? We could demand that our state legislature carry through on their agenda of accountability and order an assessment of the assessment to determine if it's doing what the taxpayers can rightfully expect it to do. And when, after ten years of development, the assessment of the assessment is finally administered and the argument rages whether or not it's worth its salt, why, the legislature can order an assessment of the assessment of the assessment. And on and on--creating an assessors' nirvana.
Or we can do something else. We can recognize that any assessment ultimately involves judgment and we can return the assessment of student accomplishment back to the persons most qualified to make that judgment: classroom teachers and the students themselves, with the support and encouragement of the administration.
In schools where there is mutual trust and respect--between and among students, teachers, and administrators--assessment becomes a continual, non-threatening, everyday activity. Where there is no stigma attached to not knowing, students, who know better than any one else what they grasp and don't grasp, have no reason to withhold that information and willingly share it with their teachers. Teachers, based on this information and their own observations, plan activities and assign tasks which help students clear up their misconceptions, deepen their understandings, and strengthen their skills. Administrators support the decisions of the teachers and provide ways for them to hone their pedagogy and increase their knowledge of the subjects they teach.
That doesn't mean things are perfect. A teacher may err in their judgment; a student may dissemble when queried about their knowledge; an administrator may find it expedient to appease an irate parent rather than support a teacher. But all-in-all, in such a setting, education is well served. The professionalism of teachers is honored; students value learning over test-taking; administrators focus on the needs of the community rather than the demands of the state department. And resources aren't squandered on elaborate assessment schemes that promise much more than they deliver.