School Creativity Indices: Measurement Folly or Overdue Response to Test-Based Accountability?

Daniel T. Hickey

A February 2 article in Education Week surveyed efforts in California, Oklahoma, and other states to gauge the opportunities for creative and innovative work. One of our main targets here at Remediating Assessment is pointing out the folly of efforts to standardize and measure “21^st Century Skills.” So of course this caught our attention.

What might come of Oklahoma Gov. Mary Smith’s search for a “public measurement of the opportunities for our students to engage in innovative work” or California’s proposed Creativity and Innovative Education index?

Mercifully, they don’t appear to be pushing the inclusion of standardized measures of creativity within high stakes tests. Promisingly, proponents argue for a focus on “inputs” such as arts education, science fair, and film clubs, rather than “outputs” like test scores, and the need for voluntary frameworks instead of punitive indexes. Indeed, many of these efforts are described as a necessary response to the crush of high stakes testing. Given the looming train-wreck of “value-added” merit pay under Race to the Top, we predict that these efforts are not going to get very far. We will watch them closely and hope some good come from them.

What is most discouraging is what the article never mentioned. The words “digital,” “network,” or “writing” don’t appear in the articles, and no consideration of the need to look at the contexts in which creativity is fostered is present. Schools continue to filter any website with user-generated content, and obstruct the pioneering educators who appreciate that digital knowledge networks are an easy and important context for creative and knowledgably engagement.

Most teachers continue to forbid students from using Wikipedia—why not let them learn how encyclopedic knowledge is created by teaching them how to edit entries that interest them? Debate clubs are fine for the few students they typically attract. But they present argument as a rarified performance art that the rest of the students find meaningless—why not help students learn to blog at the intersection of their values and academic topics?

Rebecca Itow has worked with Angie Cannon at Bloomington (IN) High School North to create and refine a fine high school English module on argument; it is posted at Digital Is where innovative teachers like Angie have implemented it. It is a great example of how new media tools and participatory learning can foster innovation and creativity while accomplishing conventional academic goals. But many teachers won’t be able to use it because their school system won’t let them complete the digital poster in Glogster or view the examples of argument at YouTube.

So yes, it is heartening the states are finally realizing that a decade of test driven reforms has driven creativity and innovation out of schools. But they should also realize that overblown concerns over plagiarism and privacy obstruction are leading a generation of students to assume that the most natural and easy outlet for their creative expression has nothing to do with school. And that is a shame.

Finnish Lessons: Start a Conversation

Rebecca C. Itow and Daniel T. Hickey

In the world of Education, we often talk of holding ourselves and adhering to “high standards,” and in order to ensure we are meeting these high standards, students take carefully written standardized exams at the state and national level. These tests are then used to determine the efficacy of our schools, curriculum, and teachers. Now, with more and more states tying these scores to value-added teaching, these tests are having more impact than ever. But being so tied to the standards can be a detriment to classroom learning and national educational success.

Dr. Pasi Sahlberg of Finland spoke at Indiana University on January 20, 2012 to discuss accounts of Finnish educational excellence in publications like The Atlantic and the New York Times, and promote his new book, Finnish Lessons: What Can the World Learn from Educational Change in Finland? One of his main points was that the constant testing and accountability to which the U.S.'s students and teachers are subjected do not raise scores. He argued that frequent testing lowers scores because teachers must focus on a test that captures numerous little things, rather than delving more deeply into a smaller number of topics.

This point addresses an important discussion that needs to occur: while we push for educational reforms, the reforms are largely the same old ineffective practices with new names. Linda Darling-Hammond makes this point eloquently in The Flat World and Education: How America’s Commitment to Equity Will Determine Our Future. States across the nation are adopting the Common Core Standards. These new standards are certainly an improvement over the patchwork quilt of obsessively detailed state standards that they replace. But the manner in which these standards are used to shape practice may be devastating. There are so many standards to be covered within one year of one subject, and within each standard are several skills to be addressed. Perhaps we are asking schools to focus on too much in a short amount of time. Dr. Sahlberg made this point by sharing that Finland teachers focus on a few skills deeply, rather than many skills shallowly, and students are tested once – at the end of their senior year. The result: Finnish students are making top scores on the PISA survey and generally scoring high in most subject areas.

One of the reasons Finland does not put a huge emphasis on standardized testing, Dr. Sahlberg reported, is because while they claim to make education equitable, they really widen the achievement gap. He shared the comic below to illustrate this point:

http://weknowmemes.com/2011/10/the-educational-system-comic/

Now, Dr. Sahlberg was quick to point out that we mustn’t think that we can take Finland’s system, implement it in the U.S., and expect it to suddenly raise achievement. Of course that won’t work. But what we can do is have a conversation about our approach to assessment, how we (appropriately and inappropriately) use the results of the assessments we take, and what we might learn from others who are experiencing success. Having this conversation does not mean that the United States is inadequate – it means we are smart, that we are reflective, and that we realize there is a problem and that we need to fix it.

Another Misuse of Standardized Tests: Color Coded ID Cards?

An October 4, 2011 Orange County Register article that reports a California high school’s policy to color code student ID cards based on their performance on state exams raises several real concerns, including student privacy. Anthony Cody in his blog post “Color Coded High School ID Cards Sort Students By Test Performance” published on October 6, 2011 in Education Week Teacher writes that “[s]tudents [at a La Palma, CA high school] who perform at the highest levels in all subjects receive a black or platinum ID card, while those who score a mix of proficient and advanced receive a gold card. Students who score "basic" or below receive a white ID card.” These cards come with privileges and are meant to increase motivation to perform well on state standardized exams. Followers’ comments and concerns posted to the blog address “fixing identity” and that testing conveys the idea that “learning and achievement isn't reward in itself. … You're not worth anything unless WE tell you are based on this one metric.” These are valid concerns, but the larger issue being highlighted here is the misuse and misapplication of the standardized tests themselves.

True, the La Palma school policy of color coding does move students into specific identities, but this is already being done, though perhaps not as explicitly as color coded ID cards; students are separated into different leveled classes and groups within those classes, and even in a “mixed level” class, the students tend to divide amongst themselves. While diversity of achievement level does exist within social groups, many times students of similar ability and experience flock together, whether or not they are given color coded ID cards. The concern of individual motivation is valid as well, on both ends of the spectrum; however, this, too, has been prevalent for years. There are those who are motivated to do well on these tests independently, those who are not interested at all, and those who need external motivation. Color coding will likely not have such a great effect on these students.

So let us turn our attention to the tests themselves, and their use and application – or misuse and misapplication. Standardized tests are useful in comparing skill sets among bodies of students. In tests on the scale of these state standardized tests, the information about performance skill among subgroups within a school, performance between schools in a particular district, and performance between districts across the state can be quite informative and helpful in assessing and guiding steps for improvement at each level. They are even helpful in designing curriculum, if used appropriately, because they set a standard of skill to be reached.

However, state standardized tests are not useful when they become the curriculum. The tests are meant to test large skill sets; they are meant to examine a student’s ability to apply a skill learned in one context to a new context. When teachers choose or are forced to exchange developing critical thinking skills for test preparation, the students and the community lose. All test prep does is teach students to take tests. It does not help them learn to be critical, thinking members of society who can analyze a book or film, weed out the spin in a political ad or news source, or understand the gravity of an oil spill or nuclear disaster halfway around the world. These critical thinking skills can and should be honed in every domain, but all too often teachers focus on memorizing vocabulary and formulas and what Romeo said in Act II, Scene i, line 2. (Incidentally, it is “Turn back, dull earth, and find thy centre out.” But does knowing that enhance your life?)

State standardized tests are not useful when they determine a teacher’s teaching schedule or bonus money. Every year I wished our school faculty would be open to sharing test scores with one another. I could see where my students fell short, and I wanted to share that with my colleagues so they would be able to work on those skills next year. I wanted to know how that teacher taught this standard…maybe I could learn something. I wanted to know where this year’s students would need extra help, and work with my colleagues to determine the best strategies to move forward. But no one wanted to share with me. Test scores have become part of a teacher’s personal pride, and often they also determine the types of classes one teaches the following year. A merit system may sound good in theory, but this is not what the tests were meant to determine, so it is not likely to work.

State standardized tests are not useful when they are the determining factor of a student’s acceptance or rejection into a higher level course. While they may give a general picture of a student’s performance – assuming they took the test seriously, were focused, and had an overall “good” testing experience – generalities do not tell the entire story. More than test scores are needed to accurately determine the potential success of a student in a particular class; one must take into account specific skills necessary to achieve, as well as the fact the scores used to place students in a 10^th grade English class often come from their 8^th grade scores, since the 9^th grade scores will not be available in time for registration and the making of the master schedule. This is not what the tests are for.

State standardized tests can be harmful when they determine whether a school will keep its funding. It seems backward to take funding away from an underperforming school and make them pay for their students to be bussed somewhere else. How will they improve? How will they fund professional development and bring in more student assistance? A lack of funding means that materials cannot be updated in a timely fashion and that stressed teachers need to focus more on getting their students to prepare for a test than developing the learning strategies and critical thinking skills that would actually help them learn and grow and perform well on exams and in life.

State standardized tests have their function, and they are good at measuring what they were intended to measure. But when the tests are misused – when a school tells students through the use of color coded ID cards that standardized tests are valued over their development of critical and analytical thinking skills, we have a problem.

Introducing Rebecca

It has been just about six months since I closed up my classroom in sunny Southern California, picked up my life, and moved to Bloomington, Indiana to pursue my PhD in Learning Sciences. I can say that a year ago I certainly did not think I would be posting on a blog about Re-Mediating Assessment. I didn't think I would writing up my research or be helping teachers develop and discuss curriculum that fosters more participation and learning in their classroom. But here I am.

In fact, a year ago I was celebrating Banned Books Week with my AP Language and Composition and Honors 9 English classes, preparing my Mock Trial team for another year of success, starting a competitive forensics team, chairing the AP department, and generally trying to convince my colleagues that my lack of “traditional” tests and use of technology in my almost-paperless classroom were not only good ideas, but actually enhanced learning. A year ago I was living life in sunny Southern California as normal ... then I decided to take the GRE. And I am so glad I did. It has been an interesting journey getting to this moment.

I never thought I would become a teacher. I have an AA in Dance, an AA in Liberal Arts, and a BA in Theatre Directing, but I found that working in the top 99 seat theatre in Los Angeles left me wanting more. When I went back for my MAEd and teaching credential, I was the only one who was surprised. Teaching students, I learned, is very much like directing actors - we want them to come to conclusions, but they need to come to them in their own way in order for the outcome to be authentic.

I have worked as a choreographer, director, and actor. I have taught 10 minute playwriting and directed festivals, as well as developed curriculum around this theme. I studied Tourette Syndrome under Dr. David Commings at the Beckman Research Institute at City of Hope, and informally counseled TS students at the high school. I am a classical dancer and recently picked up circus arts as a hobby.

Each of these very different interests contributed to my teaching. We explored literature through discussion, and often took on the roles of the characters to discuss what a piece said about the society in which it was written and its relevance today. Quite often administrators walked in while students were debating the ethics of the latest redaction of Adventures of Huckleberry Finn or discussing Fitzgerald’s symbolism while dressed to the nines at a Gatsby picnic. Still, I came up against resistance when presenting my methods and ideas to my colleagues; they didn't think that I was teaching if I wasn't giving traditional tests. I had too many A's and too few F's. I knew that I could affect greater change, but I wasn't sure how. Then the opportunity to come to Indiana University and work with Dan Hickey arose, and I had to take it.

Now I am in Bloomington, reflecting on a semester of writing, learning, studying, and creating curriculum. I have immersed myself in the school and culture and work here, and have found smaller networks of people with whom I can engage, play, think, debate, and grow. I am excited and encouraged by the adventures that await in this chapter, and am looking forward.

RMA is back!

After an extended hiatus, Re-Mediating Assessment is back.  In the meantime, lots has happened.  Michelle Honeyford completed her PhD and joined the faculty at the University of Manitoba in Winnipeg.  Jenna McWilliam has moved on to Joshua Danish's lab and is focusing more directly critical theory in new media contexts.  She renamed her blog too.

Lots of other things have happened that my student and I will be writing about.  I promise to write shorter posts and focus more on commentary regarding assessment-related events.  I have a bunch of awesome new doctoral students and collaborations who are lined up to start posting regularly about assessment-related issues.


For now I want to let everybody know that today is the official release day of a new volume on formative assessment that Penny Noyce and I edited.  It has some great chapters.  On the Harvard Education Press website announcing the book, my assessment hero Dylan Wiliam said:

"This is an extraordinary book. The chapters cover practical applications of formative assessment in mathematics, science, and language arts, including the roles of technology and teachers’ professional learning. I found my own thinking about formative assessment constantly being stretched and challenged. Anyone who is involved in education will find something of value in this book."

Lorrie Shepard's foreword is a nice update on the state of assessment.  David Foster writes about using the tools from Mathematics Assessment Resources Services in the Silicon Valley Mathematics Initiative Dan Damelin and Kimberle Koile from the Concord Consortium write about using formative assessment with cutting edge technology. (And we appreciate that the Concord Consortium is featuring their book on their website.

For me the best part was the chapter from Paul Horwitz of the Concord Consortium.  Paul wrote a nice review of his work with Thinker Tools and GenScope and the implications of that work for assessment.  Paul's chapter provided a nice context for me to summarize my ten year collaboration with him around GenScope.  That chapter is perhaps the most readable description of participatory assessment that I have managed to write.  A much more detailed account of our collaboration was just accepted for publication by the Journal of the Learning Sciences and will appear in 2011.

I promise you will be hearing from us regularly starting in the new year.  We hope you will comment and share this with others.  And if you have posts or links that you think we should comment on, please let us know.  I will let the rest of the team introduce themselves and add their bios to the blog as they start posting.

short-sighted and socially destructive: thoughts on Ning's decision to cut free services

Lord knows I'm not a huge fan of Ning, the social networking tool that allows users to create and manage online networks. I find the design bulky and fairly counterintuitive, and modifying a network to meet your group's needs is extremely challenging, and Ning has made it extremely difficult or impossible for users to control, modify, or move network content. Despite the popularity of Ning's free, ad-supported social networks among K-16 educators, the ads that go along with the free service have tended toward the racy or age-inappropriate.

But given the Ning trifecta--it's free, getting students signed up is fast and fairly easy, and lots of teachers are using it--I've been working with Ning with researchers and teachers for the last two years. So the recent news that Ning will be switching to paid-only membership is obnoxious for two reasons.

The first reason is the obvious: I don't want to pay--and I don't want the teachers who use Ning to have to pay, either. One of the neat things about Ning is the ability to build multiple social networks--maybe a separate one for each class, or a new one each semester, or even multiple networks for a single group of students. In the future, each network will require a monthly payment, which means that most teachers who do decide to pay will stick to a much smaller number of networks. This means they'll probably erase content and delete members, starting fresh each time. The enormous professional development potential of having persistent networks filled with content, conversations, and student work suddenly disappears.

Which brings me to my second point: That anyone who's currently using Ning's free services will be forced to either pay for an upgrade or move all of their material off of Ning. This is tough for teachers who have layers upon layers of material posted on various Ning sites, and it's incredibly problematic for any researcher who's working with Ning's free resources. If we decide to leave Ning for another free network, we'll have to figure out some systematic way of capturing every single thing that currently lives on Ning, lest it disappear forever.

Ning's decision to phase out free services amounts to a paywall, pure and simple. Instead of putting limits on information, as paywalls for news services do, this paywall puts limits on participation. In many ways, this is potentially far worse, far more disruptive and destructive, far more short-sighted than any information paywall could be.

If Ning was smart, it would think a little more creatively about payment structures. What about offering unlimited access to all members of a school district, for a set fee paid at the district level? What about offering an educator account that provides unlimited network creation for a set (and much lower) fee? What about improving the services Ning provides to make it feel like you'd be getting what you paid for?

More information on Ning's decision to go paid-only will be released tomorrow. For now, I'm working up a list of free social networking tools for use by educators. If you have any suggestions, I'd love to hear them.

Update, 4/15/10, 6:48 p.m.: Never one to sit on the sidelines in the first place, Alec Couros has spearheaded a gigantic, collaborative googledoc called "Alternatives to Ning." As of this update, the doc keeps crashing because of the number of collaborators trying to help build this thing (the last time I got into it, I was one of 303 collaborators), so if it doesn't load right away, keep trying.