Traditional Approaches to Validity in Assessment Innovation (Part 2: Consequential Validity)

This is the second post in a series on the topic of validity in educational assessment. In my first post, I described the traditional characterization of content-related, criterion-related, and construct-related evidence as they are relevant to educators and credentialing innovators who use and design assessments. This post summarizes traditional characterizations of “consequential validity.” This aspect of validity concerns the broader consequences of administering assessments to learners and using the resulting scores. It is a complex idea that is really crucial to many assessment and credentialing innovators (because broader change is their goal). Many measurement professionals have long argued that it an "unsanctioned" aspect of validity.  Before I write about how that is changing, I want to describe how consequential validity has traditionally been written about and why I have long disagreed.  

Traditional Approaches to Validity in Classroom Assessment and Innovative Credentialing (Part 1)

By Daniel Hickey

In my work with the Participatory Assessment Lab at Indiana University and in my graduate education courses, I spend a lot of time helping people understand validity in the context of educational assessment.  In this post, I describe validity as it has traditionally been presented to educators. I summarize what one leading textbook has long said educators should know about validity when assessing learning in their own classes, and I extend that to credentialing innovators who are developing digital badge systems, micro-credentials, and competency-based educational programs.  In subsequent posts, I will explore traditional views of “face validity” and “consequential validity.” Together, these posts will lay the groundwork for a final post that will explore several new developments in validity theory that I believe are important for these two communities.

2016 AAEEBL Midwest Meeting Keynote: Open Badges + ePortfolios: Searching for and Supporting Synergy

By Dan Hickey

This is a brief report and link to a slideshare from the keynote address at the 2016 Midwest Meeting of the Association for Authentic, Experiential, and Evidence Based Learning.  It summarizes potential synergy between these two important educational technologies, as well as progress towards this synergy by the seven leading eportfolio platforms and between AAEEBL, the Open Badges in Higher Education Project, and the Badge Alliance.

Keynote Address at AAEEBL Midwest Regional Meeting at Notre Dame

Once I get this post up I will get to the several dozen emails awaiting me in my inbox with headers suggesting that we have indeed made some progress towards increased synergy.

A Situative "Roadmap" for Synergy in Motivating Family STEM Engagement

By Dan Hickey

This is the third post about my collaboration with Iridescent Inc., a science education non-profit in LA. This new post describes how a key assumption in newer "situative" theories of motivation can resolve the tensions between prior empiricist and constructivist approaches. When combined with Design Based Research methods, this assumption can result in a coherent "roadmap" towards synergy across the three approaches. I contend that such a roadmap can help Iridescent and other informal STEM innovators find a route that takes them from current levels of engagement to much higher levels of engagement, both in terms of quantity and quality.

This post could use some work and some trimming but I need to get it up for my class and colleagues and get on to other things. Will try to clean it up soon

Motivating STEM Engagement in Children, Families, and Communities

By Daniel Hickey

 This extended post is the second in a series of post about my work with non-profit STEM education start up called Iridescent.  In a previous post I describe their Curiosity Machine and an evaluation I carried out of a program designed to encourage families to complete Engineering Design Challenges on that website from home, after attending two-hour weekly evening workshops. In this post I explore the challenges that organizations like Iridescent and science museums face in attempting to motivate inquiry learning in informal contexts. I am also using this post as my weekly contribution to the graduate education class on motivation that I am currently teaching.

The Data of Learning: A Response to Martin Kurzweil's "Responsible Use of Student Data"

by James Willis

In mid-April, Stanford University hosted the "Learning Summit 2016: Inventing the Future of Higher Education." For those of us who study how the newer processes and protocols of using student data have ethical and legal consequences, one session in particular should be of interest: Marco Molinaro (UC Davis) moderated a panel on the "Responsible Use of Student Data for Individual and Organizational Improvement" which included speakers Martin Kurzweil (ITHAKA S+R), Mitchell Stevens (Stanford), and Kent Wada (UCLA). Kurzweil provided a recent blog posting summarizing the panel discussion, raising some important points.

Engaging Children and Families in Informal Science Learning

By Dan Hickey 

In this post, I share some things I have learned about the design of "semi-formal" science learning environments that I learned working with a remarkable science education non-profit in Los Angeles called Iridescent.

AERA and Open Digital Badges

by James Willis

The 2016 conference of the American Educational Research Association (AERA) will happen next week (April 8 - April 12) in Washington, D.C. There are some events related to open digital badges, so I'll discuss them briefly along with links to additional information. There is also an interesting disparity between the growing embrace of badges and the relatively little coverage at AERA.

Recognizing, Assessing, and Motivating Entrepreneurial Mindsets

By Dan Hickey

In this longer post, I explore some of the issues around recognizing and motivating an entrepreneurial "mindset" using digital badges. I am collaborating with Rebecca DeVasher at Rose-Hulman Institute of Technology and Helen Chen at Stanford University. They and their colleagues are working with the Kern Entrepreneurial Engineering Network (KEEN) on programs to help engineering students develop the dispositions needed to be a successful entrepreneur alongside their more conventional technical skills and problem solving ability.

2016 Developments at the Badge Alliance

by James Willis and Dan Hickey in conjunction with Nate Otto

At the end of 2015 and beginning of 2016, there have been a number of changes at the Badge Alliance. In a recent blog posting, Nate Otto, Director of the Badge Alliance, sets out an ambitious agenda for 2016. Beyond the on-going work of supporting productive and evidence-rich badging projects, the Badge Alliance is also concentrating efforts on communicating across the various communities and providing technical support to the ecosystem. We summarize some of those initiatives here.