The Role of Artifact Reflections in Participatory Assessment

By Rebecca Itow and Dan Hickey

On June 7, 2012, we hosted Bloomington’s first Hackjam in conjunction with the Monroe County Public Library. In our initial recount of the day’s events, we mentioned that we used artifact reflection and digital badges as ways of gauging, evaluating, and rewarding progress in each activity. In this post, we will explain how and why we chose to use reflection and badges as forms of assessment. To read more about the theory of badges as Transformative Assessment, read our June 10 blog post. 

Assess Reflections Rather than Artifacts

We have been struggling for several years to refine practices for assessing artifacts that students create.  It seems pretty clear that badges are going to highlight a problem that teachers and proponents of portfolio assessment deal with all the time: rubrics.  If you attach consequences to the quality of student artifacts, there is a natural tendency to demand detailed rubrics and individualized feedback as to whether the artifact matches what is demanded by the rubric.  Most learning environments are more concerned with the learning embodied by the artifact than by the artifact itself.  So focusing so much on the artifact and the rubric can be quite problematic. 

Responding to Michael Cole’s Question about Badges

By Dan Hickey

Michael Cole

I was involved in an exchange on XMCA, the listserv established by Michael Cole’s Laboratory of Comparative Human Cognition and the journal Mind, Culture, and Activity.  I mentioned digital badges in the post, and Mike wrote back to ask:

You know there appear to be several people who appear from time to time on xmca involved in the Mac Arthur initiatives where badges are all the rage. For anyone interested in multi-modal representational practices, it is certainly interesting as a subject of CHAT analysis.

Question:  if you are right in your assumption that the BADGE movement will start a trend what do you think that the trend promises or portends more broadly?

I have taken some time to respond, in part because I wanted to get caught up on the latest work by Cole and his students regarding their successes and challenges around the Fifth Dimension after-school computer clubhouses.  The Fifth Dimension is precisely the kind of educational innovation that should be easier to create, sustain, and study when digital badges are widely used.

Summer 2012 Hackjam: The Wiki

Rebecca Itow and Dan Hickey

In the Fall 2011, we decided to put on a Hackjam in conjunction with the Monroe County Public Library. We adapted the curriculum outlined in the Hacktivity Kit to fit our needs, and partnered with ForAllSystems to implement a badging system for the event. You can read an earlier post giving an overall account of the event here.  We were particularly interested in aligning the hackjam with a Common Core English standard on multimodal writing.  We also wanted to make sure that all of the hackers learned how to discuss coding and writing for the web in networked spaces.  This was where they would want to go for help in the future.

Why Use a Wiki?

In adapting and designing the curriculum, it became readily apparent that, if we were going to have the participants hacking pages and reflecting on their learning, they would need a central place to do this. We began thinking that the best space would be a wiki because it is meant to be edited by multiple users, but each page can be customized to individual participants’ personality and needs. Rebecca had used Wikispaces with her 9^th and 11^th grade English students successfully in the past. Her experience in her own classroom combined with her participation in Dan’s online classes where they used “wikifolios” to house work and promote discussion convinced us that wikis were the right space for the type of engagement we wanted to foster.

Rebecca made a simple wiki on wikispaces, using the homepage as the place to access general information such as links to tools and websites that would be used throughout the Hackjam.

Three Firsts: Bloomington’s First Hackjam, ForAllBadges’ App, and Participatory Assessment + Hackasaurus

Dan Hickey and Rebecca Itow

On Thursday, June 7, 2012, the Center for Research on Learning and Technology at Indiana University in conjunction with the Monroe County Public Library (MCPL) in Bloomington, IN put on a Hackjam for resident youth. The six hour event was a huge success. Students were excited and engaged throughout the day as they used Hackasaurus’ web editing tool X-Ray Goggles to “hack” Bloomington’s Herald Times. The hackers learned some HTML & CSS, developed some web literacies, and learned about writing in different new media contexts. We did some cool new stuff that we think others will find useful and interesting. We are going to summarize what we did in this post. We will elaborate on some of these features in subsequent posts, and try to keep this one short and readable.

WHY DID WE DO A HACKJAM?

We agreed to do a Hackjam with the library many months ago. MCPL Director Sara Laughlin had contacted us in 2011 about partnering with them on a MacArthur/IMLS proposal to bring some of Nicole Pinkard’s YouMedia programming to Bloomington. We concluded that a more modest collaboration (like a Hackjam) was needed to lay the groundwork for something as ambitious as YouMedia.

Our ideas for extending Mozilla’s existing Hacktivity Kit were first drafted in a proposal to the MacArthur Foundation’s Badges for Lifelong Learning initiative. Hackasaurus promised to be a good context to continue our efforts to combine badges and participatory assessment methods. While our proposal was not funded, we decided to do it anyways. MCPL initially considered making the Hackjam part of the summer reading program sponsored by the local school system. Even though we were planning to remix the curriculum to make it more “school friendly,” some school officials could not get past the term “hacking.”

Digital Badges as “Transformative Assessment”

                                                            By Dan Hickey

               The MacArthur Foundation's Badges for Lifelong Learning competition generated immense
interest in using digital badges to motivate and acknowledge informal and formal learning. The
366 proposals submitted in the first round presented a diverse array of functions for digital
badges. As elaborated in a prior post, the various proposals used badges to accomplish one or
more of the following assessment functions:

               Traditional summative functions. This is using badges to indicate that the earner
               previously did something or knows something. This is what the educational assessment
               community calls assessment of learning.

               Newer formative functions. This is where badges are used to enhance motivation,
               feedback, and discourse for individual badge earners and broader communities of earners.
               This is what is often labeled assessment for learning.

               Groundbreaking transformative functions. This is where badges transform existing
               learning ecosystems or allow new ones to be created. These assessment functions impact
               both badge earners and badge issuers, and may be intentional or incidental. I believe we
               should label this assessment as learning.

This diversity of assessment functions was maintained in the 22 badge content awardees who were

ultimately funded to develop content and issue badges, as well as the various entities associated with HIVE collectives in New York and Chicago, who were funded outside of the competition to help their members develop and issue badges.  These awardees will work with one of the three badging platform awardees who are responsible for creating open (i.e., freely-available) systems for issuing digital badges.

            Along the way, the Badges competition attracted a lot of attention.  It certainly raised some eyebrows that the modestly funded program (initially just $2M) was announced by a cabinet-level official at a kickoff meeting attended by heads of numerous other federal agencies.  The competition and the idea of digital badges were mentioned in articles in the Wall Street Journal, New York Times, and The Chronicle of Higher Education.  This attention in turn led to additional interest and helped rekindle the simmering debate over extrinsic incentives.  This attention also led many observers to ask the obvious question: “Will it work?” 

This post reviews the reasons why I think the various awardees are going to succeed in their stated goals for using digital badges to assess learning.  In doing so I want to unpack what “success” means and suggest that the initiative will provide a useful new definition of “success” for learning initiatives.  I will conclude by suggesting that the initiative has already succeeded because it has fostered broader appreciation of the transformative functions of assessment.

Encouraging reflection on practice while grading an artifact: A thought on badges

When I started teaching I thought back to all of those teachers who made me write meaningless papers into which I put little effort and received stellar grades, and I vowed not to be that teacher. I promised myself and my future students that we – as equals – would discuss the literature as relevant historical artifacts that are still being read because the authors still have something to comment on in today’s society.

But then I stepped into the classroom and faced opposition from my colleagues who thought my methods would not provide students with the opportunities to master the knowledge of the standards. Worst of all, some teachers actually punished students who came from my class because they “knew” the students had not learned how to write or analyze since I did not give traditional tests or grade in a traditional way.