by Andi Rehak and Daniel Hickey
In more formal educational contexts, this practice is usually called "credentialing." We are using a more general term of “recognizing” to emphasize that many of the practices for recognizing learning with badges are more informal. For example, while a Girl Scout badge is technically a credential, few would call it that. The different DML projects recognize various types of learning. The way that learning is recognized and who recognizes it also varies. These differences then have consequences for assessing, motivating, and studying learning.
Our Process for Identifying these Principles
This resulted in the preliminary set of design principles listed below, starting with the ones that were most widely represented. These principles are not set in stone. The principles and (eventually) sub-principles will continue to evolve as we work with them, identify resources that are relevant to each, share them with projects, and begin sharing them more generally. We seek your questions and suggestions. Please post comments directly on the blog.
2. Align badges to standards. Many of the projects used national or international standards to increase the external value of the badge. Alignment to standards is presumed to improve transparency of the credential and help to facilitate better communication of earner knowledge and skills. Some of these standards were more formal such as, the Common Core State Standards, while others were the less formal such as "21st Century Skills." Sometimes the alignment was very formal but other times it was very informal. Highlighting the relationship between recognition and assessment, the formality of this alignment was usually defined by the formality of the assessment practices involved.
3. Have experts issue badges. Having experts issue badges increases the credibility of the badge and likely influences the usefulness of the credential outside of the issuing community. At some level, some expert is associated with issuing badges. But the nature and role of this expert varied quite a bit, as did that way that the expert was him or herself credentialed. Sometimes the expert held an external credential, while other times the expert was credentialed by the community; some projects include both.
6. Use badges as a means of external communication of knowledge and/or skills. As with the previous principle, most projects did this at some level. But some projects really made a concerted effort to increase communication of the learning or accomplishment that the badges represent.
8. Recognize educator learning as well. Some of the projects awarded badges specifically to educators. This is a different principle that relates to the several projects where additional badges were included alongside student badges. These were sometimes that same as the badges for students and other times they were specific to the educators. Generally speaking, these badges were used to recognize the educators’ participation in the broader learning ecosystem.
9. Award formal academic credit for badges. In a few projects, badges were used as a supplement to a formal grade for in-school experiences. Currently, only a couple of projects have successfully created partnerships that allow a badge to directly result in formal academic credit. This of course greatly increases the value of the badge for badge earners.
A Design Implication
We conclude by introducing a dilemma we faced in this process. This dilemma was embodied in the distinction between (a) integrating badges into an existing curriculum and (b) creating a badge system and a curriculum at the same time. For a while we treated this as a design principle for recognizing learning. But we came to see it more as a “design implication.” This difference seems to have great impact for the design of the badging practices (and likely for the learning ecosystem that results). But this distinction was largely a result of the context in which badges were being introduced, rather than a deliberate choice. As such it did not seem like this qualified as an actual design principle. When badges are being added to a pre-existing curriculum, the curriculum will constrain the way learning is recognized. For example, if an existing curriculum is not aligned to standards, it is very difficult to align a badge to standards. Alternatively, when the curriculum is being developed alongside badges, the options for both may seem limitless and overwhelming. A pre-existing curriculum can importantly help to structure design decisions. There are specific advantages for either approach.

The blog uses the phrase "creating a badge system and a curriculum" and I think you might want to broaden from curriculum to "digital engagement or learning experience." Two reasons. first in a curriculum, someone other than the learner decides what needs to be learned and pursued and I wonder if this is OK in all forms of learning. Second a "curriculum" does not seem to track well with open exploratory learning.
ReplyDeleteAm I being too harsh on formal schooling? too narrowly defining "curriculum"?
Sorry for the delay David. I was on holiday when these went up and Andi was away for health reasons.
DeleteI agree with the concern that you raise. We were given a rather narrow charge to focus on the 30 projects funded by the DML 2012 competition. This is partly because Mozilla was funded to work with the much larger open badges community. It turns out that none of the 30 projects we are looking at is focusing directly on open and exploratory learning (at least not from the start). But I think that badges are the ideal recognition practice for moving the open learning community forward. Many of the folks looking at this are part of the Mozilla Open Badges community. I am a bit out of the loop in the last couple of months, but I bet that some of the things being offered this summer in the Chicago Summer of Learning are good examples. Anyone come to mind?
Second in a series of thoughts...
ReplyDeleteI think it is a little odd to say recognize educator learning "as well" (sort of like an after-thought?) If we want these to be general principles for all kinds of badge-based recognition, then anyone can be a recipient - they just have to be a person using the digital system and doing things that earn them badges.
Again, this comes back to the constraints of our project. We wanted to have a way to help connect the specific projects that are focusing specifically on teacher learning (which were funded by Gates). Our goal is that someone who is interested in teacher learning and digital badges can zero in on the projects that enacted this principle. It is a pretty complicated info management thing in the end but I think it will make sense in the end.
DeleteThird in a series of thoughts...
ReplyDeleteOn the item "make them permanent" it might be better to say that badges have some kind of time-sensitive span of relevance that has to be considered as part of the ecosystem niche of the badge issuer. For example, while my knot-tying badge from summer camp might be permanent, I will not use it in my adult job applications. And my "flying license for Cessna's" might be a badge that I would use in an adult resume, but unless I retrain as the plane evolves (new flight characteristics, new instruments) then my badge is no longer relevant for getting a pilot's job.
Completely agree. I think the principle will be renamed "consider the life span of the recognition"
DeleteFourth in a series of thoughts...
ReplyDeleteThe experts issuing badges seems more closely aligned to formal recognition of learning. I think peer-assigned badges (as in community reputation) also have a role - and even an increasingly formal role in the assessment of learning, so I'd recommend discussing both of these concepts.
But we can only do that if we have examples of peer-awarded badges to point to. Interestingly, several of the projects proposed peer-awarded badges but did not follow through. Our challenge is how do we transform their experience into a principle that can guide others? For example, one of the projects could not make it work in their Drupal platform so they dropped it. More broadly speaking, how do we deal with the fact that as that badges become more valuable, the harder it is to do peer awarded badge? I have been dealing with this in my own work by using peer-awarded "stamps" which add up to teacher-awarded badges. But there are lots of other ways people are dealing with peer awarded badges out there that we can learn from. I hope that eventually we can connect our principles to those efforts
DeleteAnother in a series of thoughts...
ReplyDeleteOn number 9, I'm assuming you want these principles to be very broad and applicable even outside of formal education, so I would call this one "Make the rewards clear" and talk about a variety of rewards for earning badges, some of which might open new trajectories, open new doors, can be cashed in for prizes, etc. as well as put on a grade sheet or transcript.
No we are getting at something very specific here. Some of the project are awarding formal academic credit. In particular the Providence After School Alliance has made tremendous progress. This is a much bigger challenge than many people realize. We want to help people find the specific projects that are doing this and help them see the contextual factors discourage or support efforts at doing so. Katie Davis at University of Washington was funded to study PASA and it should be quite interesting.
DeleteI will enter a few comments as separate items here to see if we can get extended discussions on them, so this is the first in a short series...
ReplyDeleteAfter glancing at all the principles, I began to wonder if we need psychometric validity as well as the validity of the expert community, in which case there should be a mention of how reliability is monitored and improved. What if one expert scores a work with an x but the next one scores it with a y - is this OK? An old rule of thumb used to be that if the stakes are low, then don't worry so much about reliability, but I'm not so sure that rule is a great one. (the broken watch is reliable, after all, but fails on validity; but if we do NOT have reliable measures, then the validity suffers).
Thanks for your awesome comments. These last two were also posted at the cross posted at the discussion of the assessment principles at HASTAC so I think I will direct you and others over to the nice discussion that his fostered. Always great to discuss these issues with you.
DeleteI am reminded of the triad of models needed for assessment ( see "Knowing what students know" - Pelligrino etc.) While you might have the principles covering these, you might want to elaborate.
ReplyDelete(1) You need a task that elicits the knowledge and skill you want to measure (and this is tricky - you sort of have that with the levels, standards, hard and soft skills, and relevant context);
(2) you need a model of what a performance of that task looks like when done very well, not so well, etc.(this overlaps with your levels and trajectory concepts and relies on expert judgment - you had mentioned auto-methods in the recognition blog, maybe bring those back here again); and
(3) you need a model of how you are going to use the evidence of the specific performance of someone to make a judgment about how much of the knowledge and skill they showed you (you have the rubrics, but there are many other ways to build an evidence chain).
One point you might want to make is that if the badges are going to be signs that a valid assessment has taken place (as opposed to say just motivating people to keep going), then the system has to have all three of these components.
This post provides a thoughtful breakdown of how digital badges can serve as more than just motivators—they can also be powerful tools for communicating skills and fostering identity within learning communities. The emphasis on aligning badge design with authentic learning goals and assessment criteria is especially important in ensuring credibility and long-term value.
ReplyDeleteAt ,izone-media360.comhttps://izone-media360.com/ we’ve seen how organizations are increasingly integrating digital badges into mobile-first learning platforms to create more personalized and modular learning pathways. One challenge we often encounter is ensuring that badge metadata is rich and transferable enough to be meaningful across different contexts, especially outside of academia.
I'm curious—how do you see the future of badge interoperability evolving across platforms and institutions? Could we see a global standard emerge, or will badge ecosystems remain mostly siloed?
Looking forward to others’ thoughts on how badge design can scale meaningfully without losing depth.