By Rebecca Itow and Dan Hickey
On June 7, 2012, we hosted Bloomington’s first Hackjam in
conjunction with the Monroe County Public Library. In our
initial recount of the day’s events, we mentioned that we used artifact
reflection and digital badges
as ways of gauging, evaluating, and rewarding progress in each activity. In
this post, we will explain how and why we chose to use reflection and badges as
forms of assessment. To read more about the theory of badges as Transformative
Assessment, read our June
10 blog post.
Assess Reflections
Rather than Artifacts
We have been struggling for several years to refine
practices for assessing artifacts that students create. It seems pretty clear that badges are going
to highlight a problem that teachers and proponents of portfolio assessment
deal with all the time: rubrics. If you
attach consequences to the quality of student artifacts, there is a natural tendency
to demand detailed rubrics and individualized feedback as to whether the
artifact matches what is demanded by the rubric. Most learning environments are more concerned
with the learning embodied by the artifact than by the artifact itself. So focusing so much on the artifact and the rubric
can be quite problematic.
We suggest eliminating rubrics entirely. Instead, provide guidelines for creating
artifacts, then invest in helping students reflect on how the artifacts reveal
that they have participated in something meaningful. Evaluate those reflections instead. As you can see from the examples below, the
main strategy in our reflections is getting students to reflect on how their
own version of the activity impacted what they learned about the concepts we
were teaching them. This notion of
“context x concept” reflection is central to all of our efforts to use
assessment to foster participatory learning.
It turns out that in order to reflect on the intersection of your (more
concrete) context and the (more abstract) targeted concept, you need to have or
get some understanding of that concept.
Since you never directly teach students the definition or even the
abstract meaning of the concept, any evidence of it in the reflection is a good
sign that students have learned about the concept. This is an example of how participatory
approaches balance formative and summative assessment functions.
Assessing Reflections
in the Hackjams
The activities in the Hackjam were posted to individual wiki
pages created by the students using wikispaces.com.
They ranged from informal and “light” hacking of an article in the local paper
to creating a webpage from scratch using Mozilla’s
Thimble App, and within each activity hackers could engage to varying
degrees. For simply completing an activity and demonstrating the skills
outlined in the badge requirements, hackers received a One Star Maker Badge.
For thinking about their learning process and composing a reflection on how
that activity affected the way they understood the web literacies specified,
they could receive a Two Star Reflector Badge.
These first two levels of badges were awarded by the Hackjam
mentors, and a picture of their completed activity or reflection was attached
to the badge as evidence using ForAllSystems’ newly
developed app available in the iTunes store. Three Star Participator Badges
were peer-awarded. Hackers read and reviewed each other’s hacks and
reflections, and could choose to award their peer with a Three Star Badge,
indicating that the author had impacted the reader’s knowledge in some
significant way. Badge awarders were required to include a statement of why
they felt the peer deserved this badge as the evidence attached to the badge
itself. They were also asked to comment on the awardee’s wiki page and tell
them why a Three Star Badge was awarded.
ForAllSystems’ Toby Kavukattu was instrumental in making the
badge system successful. He helped us rework the existing badges in the Hacktivity Kit – which only
have one level of each type of badge – to reflect the principles behind the
development of our curriculum, and to represent different levels of engagement.
He drove down from Chicago to help implement the badge system, and he even
taught the advanced webmaking section of the Jam.
What Did We Learn?
While the overall implementation of badging was successful,
our first attempt at awarding badges has left us with clear goals and
adjustments for future implementations. The first thing we noticed is that we
had too many badges – or learning outcomes – than could possibly be covered in
one six-hour period. For example, we determined that, while the hackers all
enjoyed making their own webpage from scratch, they literally only scratched
the surface of what could have been covered, and really needed a day devoted
just to that activity.