smacking down the Boston Globe

Originally posted May 5 at sleeping alone and starting out early

It's too bad the Boston Globe is staying open, because now I have to smack it down big-time for this editorial arguing that we shouldn't standardize and measure achievement on so-called 21st-century skills. The op-ed offers further proof--as if we needed it--that the Globe's editorial board has no idea how the playing field has been utterly transformed by participatory culture.

The impetus behind the op-ed is a move by the state Department of Elementary and Secondary Education to put its money where its mouth is. The department recently awarded a $146 million contract to the designer of the MCAS, the standardized test mandated in the commonwealth of Massachusetts by No Child Left Behind, and part of that money is earmarked for integration of 21st-century skills assessment. This is a problem, as the Globe's editorial board will point out momentarily.

But first, it uses state MCAS scores as proof of public school rigor. As it explains,

Massachusetts stands apart in public education precisely because it created high academic standards, developed an objective measure of student performance and progress through the MCAS test, and required a passing grade in order to graduate. Students, as a result, rank at or near the top of standardized testing not just nationally but on tough international achievement tests in math and science. Any retreat from this strategy would be a profound mistake.

So to summarize: Massachusetts students are among the top in the nation because their achievement on standardized tests prepares them to...score well on standardized tests. It's like the iconic example of circular reasoning: The MCAS is useful because it prepares them for future learning. How do you know? Because Massachusetts students do well on other standardized tests. What prepares them to do well on those tests? Doing well on standardized tests, of course.

Given the Globe's wholehearted genuflection at the altar of bubble tests, one wonders why this editorial might oppose integrating assessment of 21st-century skills in addition to traditional subjects. It turns out their concern is less about whether we should measure 21st century skills than it is about how doing so on the MCAS will affect test scores in general. As the editorial points out,

[s]tate education officials have done a generally poor job of defining 21st-century skills - which can include interdisciplinary thinking and media literacy - or explaining how to test them statewide.

The problem for the Globe, it turns out, is that if we develop mediocre assessment strategies it'll ruin the MCAS for all of us. Because 21st-century skills can only be measured subjectively, the Globe argues, an "objective" test like the MCAS is an inappropriate place to assess achievement. Instead,

MCAS testers should concentrate on accurately measuring math ability and reading comprehension, which surely correlate with a student's success in the workplace.

Let's leave, just for now, the outrageous assumption that a standardized test could conceivably be considered "objective." Let's leave the assumption that a standardized test could "accurately" measure student ability in anything other than the ability to engage in the weird and peculiar game of test-taking. Which leaves just one last question:

In what world can anybody make the argument that achievement in math and reading without the accompanying facility with 21st-century proficiencies prepares any learning for any workplace worth the energy of applying for employment in the first place?

It's such a weird argument to make, that literacy practices like reading, writing, and doing math can be somehow isolated from the 21st-century contexts that make them meaningful. It's like asking someone if she knows how to tie her shoe, then making her
prove it by writing a detailed step-by-step description of how to do it. It's like asking someone to prove he can build a fire: But is the fire for warmth, for signaling for help, or for burning the whole house down?

Same with math: Knowing how to "do" fractions doesn't mean a learner is equipped to, say, resize a .jpg for a blogpost.

Arguing that we should keep 21st-century skills out of standardized tests in order to keep the tests objective is as lame as the argument that standardized tests are objective in the first place. Neither one makes any logical sense. Neither one gets you anywhere.

why you should invite me to your next party

(hint: because I will entertain your guests with talk of the social revolution)
(cross-posted from sleeping alone and starting out early).

I was at a party last week when someone asked me what I do for a living. I used the opportunity to engage in what, in retrospect, may have been an ill-timed impromptu pronouncement about the status of the social revolution.

It turns out I'll need to rethink how I use that phrase "social revolution," at least in mixed company, because a tubby drunk man wearing a confusing hat walked up to me and tried to steer the conversation toward war atrocities.

"You can't tell me," he bellowed, "that the atrocities that are happening during the Iraq War are any different from the ones that happened during World War II. It's just that we have more media coverage now."

As I wrote in an earlier post, this is what I've decided to call the Space Odyssey mistake. This particular kind of error is explained by Clay Shirky, who describes a scene from 2001 in which

space stewardesses in pink miniskirts welcome the arriving passenger. This is the perfect, media-ready version of the future--the technology changes, hemlines remain the same, and life goes on much as today, except faster, higher, and shinier.

Lately I've been finding Christopher Kelty's notion of a "recursive public" useful in thinking about what, other than hemlines, have changed. As Kelty describes it in Two Bits (available for download, online browsing, and modulation for free online),

A recursive public is a public that is vitally concerned with the material and practical maintenance and modification of the technical, legal, practical, and conceptual means of its own existence as a public; it is a collective independent of other forms of constituted power and is capable of speaking to existing forms of power through the production of actually existing alternatives.

More to the point, a recursive public is a group of people who exist outside of traditional institutions (governments, churches, schools, corporations) and, when necessary, use this outsider status to hold these entities in check. The engagement of these publics goes far beyond simply protesting decisions or stating their opinions. Kelty, writing about geek culture as a recursive public, explains it thus:

Recursive publics seek to create what might be understood, enigmatically, as a constantly “self-leveling” level playing field. And it is in the attempt to make the playing field self-leveling that they confront and resist forms of power and control that seek to level it to the advantage of one or another large constituency: state, government, corporation, profession. It is important to understand that geeks do not simply want to level the playing field to their advantage—they have no affinity or identity as such. Instead, they wish to devise ways to give the playing field a certain kind of agency, effected through the agency of many different humans, but checked by its technical and legal structure and openness. Geeks do not wish to compete qua capitalists or entrepreneurs unless they can assure themselves that (qua public actors) that they can compete fairly. It is an ethic of justice shot through with an aesthetic of technical elegance and legal cleverness.

This is precisely the difference between 1945 and 2009. It's not just that we have more media coverage but that, as Shirky proclaims, everybody is a potential media outlet--everyone has the potential to join a recursive public, whether impromptu or planned.

In fact, the notion that we can all engage in reportage is perhaps a bit too simplistic, at least until we can adjust what we mean by "journalism." When Facebook users joined up in opposition to a change in Facebook's terms of service and successfully pressed administrators to rethink and reword the terms of service agreement, that was the work of a recursive public, loosely banded and easily disbanded once their purpose had been achieved (if necessary, they will quickly gather again in their virtual space and just as quickly disband). We don't recognize this as journalism, often don't even recognize it as civic engagement--but for those who joined this Facebook knotwork, it's certainly some kind of engagement. And what could be more civic-minded than fighting to define the uses of a public space?

The atrocities of war are approximately the same (though, as always, new technologies mean new modes of torture and murder). What's different is the following:

• Citizens within and outside of the territories in question have the capacity to keep mainstream media outlets honest: Biased, incomplete, and corrupt journalism will no longer stand in for the official story.
  
  
• Citizens can engage in consequential debate, which is to say that serious public conversation about the merits of a particular conflict can gather enough voices to sway public opinion and result in real policy changes more quickly than ever before. This is Vietnam to the power of Internet.
  
  
• Citizens without access to a wide range of new media technologies can still have their experiences broadcast to the world by other members of their recursive public. A cellphone picture of a soldier torturing a civilian, for example, can be quickly distributed for maximum effect.

All in all, it was a good party. Near the end, someone produced a Donald Rumsfeld piñata. We were going to hoist it up and smash it, but it seemed kind of...irrelevant.

figuring out "how to go on"

In his paper "Human Action and Social Groups as the Natural Home of Assessment: Thoughts on 21st Century Learning and Assessment," Jim Gee describes what at first glance appears to be two opposing uses of assessment in informal online spaces. As Gee explains,

Assessment for most social groups is both a form of mentoring and policing. These two are, however, not as opposed to each other as it might at first seem (and as they often are in school). Newcomers want to “live up to” their new identity and, since this is an identity they value, they want that identity “policed” so that it remains worth having by the time they gain it more fully. They buy into the “standards.” Surely this is how SWAT team members, scientists, and Yu-Gi-Oh fanatics feel.

At its best, assessment in formal education serves the same dual role; yet something is most assuredly different. What's different is not the degree to which students "buy into" the value of assessment; they see assessment in school as important, just as they would argue that the mentoring and policing of the online spaces they inhabit is an essential element to keeping those spaces alive.

What's different is not the degree of investment; what's different is the degree of relevance. The Yu-Gi-Oh fanatics Gee references want--sometimes desperately--to be accepted into the groups they join, and so they agree to the terms of this belonging, even if it requires being held to at times impossibly high standards of participation. The same is true of novice SWAT team members and scientists; it's less true of 11th graders reading Moby-Dick in an English classroom. To what purpose? they might ask, and they would be right to do so. Until we can align the goals, roles, and assessment practices of the formal classroom--until, that is, we can transform the domain to meet the needs of a participatory culture--investment exists without relevance. Students want A's to get into college; they want A's because that's what their parents tells them equals success; they want A's (or D's, or F's) because their friends tell them they should.

We know that practices are mediated by the tools we use to engage in those practices. We know that writing with a pencil is different from writing with a computer is different from writing with a Blackberry. The notion of "re-mediation" is intended to point to another level of mediation: That the tools that mediate traditional literacy practices get re-mediated by new media, which then re-mediates the practices that we bring to the tools. It's all very meta.

All of this is by way of introduction to re-mediating assessment, a new blog emerging out of a clever little partnership between a plucky crew of assessment-oriented researchers out of Indiana University and MIT. The plucky researchers include:

Daniel T. Hickey, Associate Professor of Learning Sciences at Indiana University, and our intrepid leader. Dan's research focuses on participatory approaches to assessment and motivation, design-based educational research, and program evaluation. He is particularly interested in how new participatory approaches can advance nagging educational debates over things like assessment formats and the use of extrinsic incentives. His work is funded by the National Science Foundation, NASA, and the MacArthur foundation, and has mostly been conducted in the context of digital social networks and videogames. He teaches graduate courses on cognition & instruction, assessment, and motivation, and undergraduate classes on educational psychology.

Michelle Honeyford, a Ph.D. Candidate in the Literacy, Culture, and Language Education Department at Indiana University and the cool head behind this operation. Michelle is a Graduate Research Assistant on the MacArthur-funded 21st Century Assessment Project for Situated and Sociocultural Approaches to Learning, working on a participatory assessment model for new media literacies. Her broader research interests include identity, cultural citizenship, and new literacy studies. Michelle is a former middle and high school English Language Arts teacher, and has taught courses in the teaching of writing at IU.

Jenna McWilliams, the little engine that could. Jenna is a prolific blogger who is working on mastering the art of being both smart and lucky, sometimes simultaneously. She recently got picked up by the Guardian's online site, Comment is Free, and was interviewed about the future of newspapers on the BBC's News Hour program. She currently works as an educational researcher for Project New Media Literacies, a MacArthur-funded research project based at MIT; prior to that, she taught English composition, literature, and creative writing at Suffolk University, Bridgewater State College, and at Newbury College and at Colorado State University, where she earned her MFA in Creative Writing. In Fall 2009, she will begin doctoral study in the Learning Sciences Program at Indiana University, under the tutelage of Dan Hickey, who is her sensei.

Together, this merry band will start working out the simple matter of "how to go on" and how to align classroom practices with the proficiencies called for--indeed, demanded--by a participatory culture.

We're bringing the smart. Wish us luck.