Showing posts with label standardized testing. Show all posts
Showing posts with label standardized testing. Show all posts

Monday, July 20, 2009

making universities relevant: the naked teaching approach

I feel sorry for college deans, I really do*. They face the herculean task of proving that the brick-and-mortar college experience offers something worth going into tens of thousands of dollars of debt for, a task made even more difficult by the realities of a recession that's left nearly a quarter of Americans either unemployed or underemployed.

Then there's the added challenge of proving colleges have anything other than paper credentials to offer in a culture where information is free and expert status is easily attainable. Only in a participatory culture, for example, would it be possible for time-efficiency guru Timothy Ferriss to offer a set of instructions on "How to Become a Top Expert in 4 Weeks." "It's time to obliterate the cult of the expert," Ferriss writes in his mega-bestseller, The Four-Hour Workweek. He argues that the key is to accumulate what he calls "credibility indicators." It is possible, he writes,
to know all there is to know about a subject--medicine, for example--but if you don't have M.D. at the end of your name, few will listen.... Becoming a recognized expert isn't difficult, so I want to remove that barrier now. I am not recommending pretending to be something you're not... In modern PR terms, proof of expertise in most fields is shown with group affiliations, client lists, writing credentials, and media mentions, not IQ points or Ph.D.s.

Ferriss then offers five tips for becoming a "recognized expert" in your chosen field. None of them include earning the credential through formal education.

Just like that, we've gone from the position that expertise takes a decade, at minimum, to develop, to the argument that a person can become an expert in just four weeks.

In the face of this qualitative shift in how we orient to expertise, colleges--the educational institutions that have made their bones on offering a sure path to credentialing--are struggling to remain viable. One strategy--and the one chosen by José A. Bowen, dean of the Meadows School of the Arts--is to offer "naked teaching." Bowen's approach, as described in a recent piece in the Chronicle of Higher Education, is to actually remove networked technologies from the classroom. The article makes it clear that Bowen is not anti-technology; he just thinks technologies are being misused by faculty who overrely on PowerPoint and technology-supported lecturing techniques. He favors using technologies like podcasting for delivering lecture materials outside of the classroom, then using the class itself to foster group discussion and debates.

To support this approach, all faculty were recently given laptops and support for creating podcasts and videos.

According to the Chronicle piece, the group that's most upset about the shift away from the traditional lecture format is...students. According to Kevin Heffernan, an associate professor in the school's division of cinema and television, students

are used to being spoon-fed material that is going to be quote unquote on the test. Students have been socialized to view the educational process as essentially passive. The only way we're going to stop that is by radically refiguring the classroom in precisely the way José wants to do it.


For all the griping we do about No Child Left Behind, test-centered accountability practices, and high-stakes assessment practices, the roaring success of decontextualized accountability structures is their astounding ability to keep formal education relevant. "Success" at the primary and secondary level means high achievement on high-stakes tests; and, achievement depends on the learner's ability to internalize the value systems and learning approaches implicit in the approach of this kind of testing structure. Do well on a series of state-mandated tests and you'll probably also do well on the SAT; do well on the SAT and you're well positioned for the lecture-style, knowledge-transfer and, in general, highly decontextualized experience of most undergraduate-level classes. We gravitate toward the kind of experience that make us feel successful, which means the testing factory churns out its own customer base.

While Bowen's experiment (one that he's been moving toward for years; see this 2006 piece in the National Teaching and Learning Forum) may garner attention for an apparent anti-technology stance, the impetus behind his "naked teaching" approach is an effort to reshape the role of institutions of higher education. In truth, learning can happen anywhere, and Bowen's embrace of this truth through his embrace of technologies for supporting out-of-class information transfer seems like a low-risk and high-yield slant on the role of the university.

If learning can happen anywhere, then the physical community of learners gathered together within four walls, engaged in the act of collaborative knowledge-building: That's the rare commodity. In a world where everyone can be an expert, the promise of credentials become just another strategy for bringing that community together.



*jk I really don't.

Tuesday, July 7, 2009

Weeding, Seeding, and Feeding Social Educational Designs

This post examines the implications of a post at the Harvard Business blog by David Armano titled Debunking Social Media Myths about social business design. He points to three labor-intensive activities that are necessary for a profitable social network: weeding, seeding, and feeding. We examine these three considerations for social education design, and how they are necessary for a worthwhile social network for educators.

First some background and context. One of our primary interests here at Re-Mediating Assessment is how innovative classroom assessment practices can be shared over digital social networks. By assessment practices, we mean both particular assessments for particular activities, as well as expertise associated with those practices. Of course, we know that most efforts to create collaborative networks for educators don’t take hold (Check out the 2004 book Designing Virtual Communities in the Service of Learning by Barab, Kling, and Gray, and a special issue of The Information Society they edited for a good discussion of some pioneering efforts).

We have previously written about the value of insights out of media scholarship for thinking about the sharing of educational practices. In particular Henry Jenkins’ notions of "spreadable" practices have prompted us to launch serial posts at Project New Media Literacies introducing the idea of Spreadable Educational Practices (SEP) and to juxtapose them with the doomed distribution of centrally defined and "scientifically" validated scripts, what we label Disseminated Instructional Routines.

We are currently outlining several new proposals to expand the nascent networks that are forming around various efforts. We also want to design and test strategies for helping other nascent networks succeed by helping to foster spread of effective practices and the necessary social bonds.

Media scholars inevitably consider the for-profit nature of commercial media. Of course, not all media scholars care about markets and eyeballs, and the nature of media markets are undergoing tremendous change. In our prior posts at Project NML, Henry's descriptions of failed corporate efforts to create "viral" messages and "sticky" websites seemed to describe some of the failed efforts to create educator social networks. The point here is that educational social design can be informed by business social design.

Armano’s talk at the Conversational Marketing Summit pointed out something that is easy to underestimate: "Being social means having real people who actively participate in your initiatives." Because educators tend to be so overwhelmed by the daily press of teaching, worthwhile social education design must find ways to get teachers actively involved. And this takes resources. Building a network will require significant support at the outset, likely sponsoring leading participants to welcome newcomers and foster effective practices. As Clay Shirky wrote in Here Comes Everybody, the founder of Flickr said that she learned early on that "you have to greet the first ten thousand users personally.”

Here are three things that Armano says that successful networks must plan for, and what they might look like in an educational network:

Seeding: Someone has to seed a network with resources and practices that your particular users need. In the Participatory Activities and Assessment Network we are trying to build out, we will budget quite a bit of time for research assistants to work with lead educators to identify interesting and engaging on-line activities for their students. However, the participatory assessments that educators can use to implement those activities in their classrooms and refine them over time will likely have to be constructed by the research effort. So we are budgeting for that too. We will work with heavily subsidized and fully-supported lead teachers for the first year to seed the network with useful activities before bringing on less-subsidized and partly-supported teachers. Only then do we think that the network will be sufficiently seeded to expect unsupported users to start participating in large numbers.

Feeding. The network needs a steady stream of content. By content, what we mean is information--information that other participants will find useful. In our case, the most useful information will be the anecdotes and guidelines for implementing participatory activities with actual students, and sharing the "low stakes" evidence obtained from participatory assessments for improving success. For example, we view the posting of accounts, videos, and artifacts from the enactment of successful implementation as crucial content that needs to be fed to the network. It is our job to make sure that there is both a source and an audience. Our lead teachers will need help posting accounts of enactments to the network. For example, most teachers know that they themselves can't post video of their students to YouTube (as researchers we are forbidden from even thinking about doing so because of Human Subjects constraints). But there is nothing to stop us from giving the lead teachers several inexpensive flip cams and letting students post accounts of themselves to YouTube (if the school allows access; they may be better off with SchoolTube which is less likely to be blocked).

Once accounts of practice are up, it is also our job to ensure that there is an audience. In this case, we will have paired teachers up to select activities to complete with their classrooms, and then asking one of them to implement first. This first lead teachers’ posted accounts and informal guidelines will be immediately useful for to second lead teacher. One or two simple successes like this will create a powerful social bond between two otherwise isolated participants. Because this interaction will take place via public and persistent discourse in the network, the accounts will be immediately useful for other participants wishing to use the activities; this discourse will be crucial for helping that newcomer locate and access the informal expertise that is now spread across the network in those two lead teachers.

Weeding. Armano points out that productive social business design must prune content that inhibits growth. This might be the most challenging aspect of a productive social education design. This partly refers to getting rid of problematic content. One of the lessons we learned in our collaboration with Project New Media Literacies working with Becky Rupert at Aurora High School is that need for involving students in helping keep offensive or objectionable material off of school related networks. If the students find the networking activities an enjoyable alternative to traditional activities the quickly become a powerful ally in minimizing transgressions. This is crucial, as teachers simply won’t have time to do it, and will be overwhelmed with the nuanced decisions between creative expressions and those that are patently offensive.

Another important part of weeding is getting rid of stuff that does not work. As educators we have a tendency to hang on to everything and make it available to all. Thus we create a huge obstacle for other educators who have to weed through endless list of resources looking for the right one, and then implement it and hope it succeeds. Our network assumes that most web-based educational resources are not very good at fostering worthwhile classroom participation. This is not because the resources are inherently bad, but because participatory classroom culture is so challenging to attain. Our Participatory Activities and Assessment Network will start with a carefully catalogued and tagged set of activities that have been initially vetted and aligned to one or two Relevant Big Ideas (or RBIs, which in turn can be easily aligned to content standards). As lead teachers select activities for further consideration, they will be tested by research assistants before participatory assessments are created and released along with the activity. If the activities and assessments don’t foster worthwhile participation for the first two lead teachers, it will be tagged as such.

Importantly, the network will contain useful information about the nature of that “productive failure” that will be useful for others. Consider that it may well be that the activity turned out to be too easy or too hard or required too much background knowledge for the particular students. Rather than labeling the activity as “useless” it should be tagged in a way that another teacher who works with students for whom it might be “perfect” can find the activity, along with the information and distributed expertise for using it. It looks to us like building information systems for accomplishing this will be one of our major challenges.

Sunday, June 14, 2009

the harrison bergeron approach to education: how university rankings stunt the social revolution

I've been thinking some lately about the odd and confusing practice of comparing undergraduate and graduate programs at American colleges and universities and producing a set of rankings that show how the programs stack up against each other.

One of the most widely cited set of rankings comes from U.S. News and World Report, which offers rankings in dozens of categories, for both undergraduate and graduate-level programs. Here, the magazine offers its altruistic rationale behind producing these rankings:
A college education is one of the most important—and one of the most costly—investments that prospective students will ever make. For this reason, the editors of U.S. News believe that students and their families should have as much information as possible about the comparative merits of the educational programs at America's colleges and universities. The data we gather on America's colleges—and the rankings of the schools that arise from these data—serve as an objective guide by which students and their parents can compare the academic quality of schools. When consumers purchase a car or a computer, this sort of information is readily available. We think it's even more important that comparative data help people make informed decisions about an education that at some private universities is now approaching a total cost of more than $200,000 including tuition, room, board, required fees, books, transportation, and other personal expenses.

(To access the entire rankings, developed and produced selflessly by U.S. News and World Report, you need to pay. Click here to purchase the Premium Online Edition, which is the only way to get complete rankings, for $14.95.)

The 2009 rankings, released in April, are in the news lately because of questions related to how the magazine gathers data from colleges. As Carl Bialik points out in a recent post at the Wall Street Journal, concerns over how Clemson University set about increasing its rank point to deeper questions about the influence of rankings numbers on university operations. Clemson President James F. Barker reportedly shot for cracking the top 20 (it was ranked 38th nationally in 2001) by targeting all of the ranking indicators used by U.S. News. Bialik writes:
While the truth about Clemson’s approach to the rankings remains elusive, the episode does call into question the utility of a ranking that schools can seek to manipulate. “Colleges have been ‘rank-steering,’ — driving under the influence of the rankings,” Lloyd Thacker, executive director of the Education Conservancy and a critic of rankings, told the Associated Press. “We’ve seen over the years a shifting of resources to influence ranks.”

Setting aside questions of the rankings' influence on university operations and on recruiting (both for prospective students and prospective faculty), and setting aside too the question of how accurate any numbers collected from university officials themselves could possibly be when the stakes are so high, one wonders how these rankings limit schools' ability to embrace what appear to be key tenets emerging out of the social revolution. A key feature of some of the most vibrant, energetic, and active online communities is what Clay Shirky labels the "failure for free" model. As I explained in a previous post on the open source movement, the open source software (OSS) movement embraces this tenet:
It's not, after all, that most open source projects present a legitimate threat to the corporate status quo; that's not what scares companies like Microsoft. What scares Microsoft is the fact that OSS can afford a thousand GNOME Bulgarias on the way to its Linux. Microsoft certainly can't afford that rate of failure, but the OSS movement can, because, as Shirky explains,
open systems lower the cost of failure, they do not create biases in favor of predictable but substandard outcomes, and they make it simpler to integrate the contributions of people who contribute only a single idea.

Anyone who's worked for a company of reasonable size understands the push to keep the risk of failure low. "More people," Shirky writes, "will remember you saying yes to a failure than saying no to a radical but promising idea." The higher up the organizational chart you go, the harder the push will be for safe choices. Innovation, it seems, is both a product of and oppositional to the social contract.

The U.S. News rankings, and the methodology behind them, runs completely anathema to the notion of innovation. Indeed, a full 25 percent of the ranking system is based on what U.S. News calls "peer assessment," which comes from "the top academics we consult--presidents, provosts, and deans of admissions" and, ostensibly, at least, allows these consultants
to account for intangibles such as faculty dedication to teaching. Each individual is asked to rate peer schools' academic programs on a scale from 1 (marginal) to 5 (distinguished). Those who don't know enough about a school to evaluate it fairly are asked to mark "don't know." Synovate, an opinion-research firm based near Chicago, in spring 2008 collected the data; of the 4,272 people who were sent questionnaires, 46 percent responded.

Who becomes "distinguished" in the ivory-tower world of academia? Those who play by the long-established rules of tradition, polity, and networking, of course. The people who most want to effect change at the institutional level are often the most outraged, the most unwilling to play by the rules established by administrators and rankings systems, and therefore the least likely to make it into the top echelons of academia. Indeed, failure is rarely free in the high-stakes world of academics; it's safer to say no to "a radical but promising idea" than to say yes to any number of boring but safe ideas.

So what do you do if you are, say, a prospective doctoral student who wants to tear wide the gates of academic institutions? What do you do if you want to go as far in your chosen field as your little legs will carry you, leaving a swath of destruction in your wake? What do you do if you want to bring the social revolution to the ivory tower, instead of waiting for the ivory tower to come to the social revolution?

You rely on the U.S. News rankings, of course. It's what I did when I made decisions about which schools to apply to (the University of Wisconsin-Madison [ranked 7th overall in graduate education programs, first in Curriculum & Instruction, first in Educational Psychology] the University of Texas-Austin [tied at 7th overall, 10th in Curriculum & Instruction], the University of Washington [12th overall, 9th in Curriculum & Instruction], the University of Michigan [14th overall, 7th in Curriculum & Instruction, and 3rd in Educational Psychology] the University of Indiana [19th overall, out of the top 10 in individual categories], and Arizona State University [24th overall, out of the top 10 in individual categories]). Interestingly, though, the decision to turn down offers from schools ranked higher than Indiana (go hoosiers) wasn't all that difficult. I knew that I belonged at IU (go hoosiers) almost before I visited, and a recruitment weekend sealed the deal.

But I had an inside track to information about IU (go hoosiers) via my work with Dan Hickey and Michelle Honeyford. I also happen to be a highly resourceful learner with a relatively clear sense of what I want to study, and with whom, and why. Other learners--especially undergraduates--aren't necessarily in such a cushy position. They are likely to rely heavily on rankings in making decisions about where to apply and which offer to accept. This not only serves to reify the arbitrary and esoteric rankings system (highest ranked schools get highest ranked students), but also serves to stunt the social revolution in an institution that needs revolution, and desperately.

In this matter, it's turtles all the way down. High-stakes standardized testing practices and teacher evaluations based on achievement on these tests limits innovation--from teachers as well as from students--at the secondary and, increasingly, the elementary level. But the world that surrounds schools is increasingly ruled by those who know how to innovate, how to say yes to a radical but promising idea, how to work within a "failure for free" model. If schools can't learn how to embrace the increasingly valued and valuable mindsets afforded by participatory practices, it's failing to prepare its student body for the world at large. The rankings system is just another set of hobbles added on to a system of clamps, tethers, and chains already set up to fail the very people it purports to serve.