Recognizing, Assessing, and Motivating Entrepreneurial Mindsets

By Dan Hickey

In this longer post, I explore some of the issues around recognizing and motivating an entrepreneurial "mindset" using digital badges. I am collaborating with Rebecca DeVasher at Rose-Hulman Institute of Technology and Helen Chen at Stanford University. They and their colleagues are working with the Kern Entrepreneurial Engineering Network (KEEN) on programs to help engineering students develop the dispositions needed to be a successful entrepreneur alongside their more conventional technical skills and problem solving ability.

Rebecca DeVasher

Badging the Entrepreneurial Mindset

One of my favorite things about the Open Badges in Higher Education project is that we get to work as free consultants with some of the most imaginative minds in higher education. I just started working with Rebecca DeVasher at Rose-Hulman Institute, one of the nation's top undergraduate engineering schools, located in Terre Haute. I was introduced to Rebecca by Helen Chen at Stanford's Epicenter, the National Center for Engineering Pathways to Innovation.  Helen is one of the leading experts on ePortfolios and is exploring the intersection between ePortfolios and formal records like transcripts. Helen and colleagues just published a great article on this topic in EDUCAUSE Review. Rebecca and Helen are partnering with the Kern Foundation's Kern Entrepreneurial Engineering Network (KEEN) to operationalize an "entrepreneurial mindset."  Mindset (and "grit") is a popularized version of goal theories that emerged in the 1980s.  I studied and measured these constructs extensively as a grad student in the early '90s, but I stopped studying them around 2000 for reasons I will elaborate below. I have been waiting for just this kind of opportunity to pick them up again within the context of digital badges.

Helen Chen

 Rebecca and Helen are working within a broader effort involving a higher ed non-profit called VentureWell that aims to foster invention, innovation, and entrepreneurship among science and technology students.  Rose-Hulman is famous for its scary-smart undergraduates who come out with graduate-level knowledge of engineering. Their superbly-trained graduates are already highly sought-after by employers. What could be better than a graduate who also presents convincing evidence that he or she has a well-developed mindset for entrepreneurship?  KEEN and the various participants have a nicely defined framework for entrepreneurial mindsets.  So the next phase if figuring out how to assess those mindset and then recognize and credential them.  And of course, once you starting recognizing and credentialing things, you have to start considering how you motivate students to develop those mindset and (particularly) if and how your assessment and recognition practices might impact that motivation.

In a presentation at a recent VentureWell conference, Rebecca and Helen engage in a crucial dialogue that we have witnessed dozens of times. It occurs when people decide to design a badge system to recognize and motivate learning of knowledge that does not fit neatly into traditional competency frameworks (this includes all of the so-called "21st Century bla bla bla"). Because badges contain both claims and links to evidence supporting those claims, the act of defining a badge system forces innovators to ask hard questions about what claims they can make and what evidence they can marshal to support those claims. Conventional exams, grades, and transcripts are normally wrapped in a FERPA-protected cloak of privacy and so nobody really interrogates the claims and evidence. Because badges circulate in social networks, the claims and evidence they contain are suddenly exposed and subject to further endorsement and dismissal.  

Rebecca and Helen wisely recognize that if the evidence (such as links to student videos, projects, reflections, etc.) don't support the claims made in the badges, the badges won't be valued by employers, students, or  peers. Just imagine if a graduate earns an entrepreneurship badge and posts it to Facebook and his or her peers pepper it with a bunch of likes and favorable comments. It might be just the sort of thing that compels a busy HR person to interview that student over a similar peer. These "post-issue endorsements" (yes, we say this) are particularly valuable because there is nothing (other than civility) to stop the opposite from happening. If someone's evidence was unconvincing or pirated, that same badge might get peppered with new frowny face emoticons and derisive comments. This is a great example of what Carla Casilli and I wrote about recently in an article in The Information Society (here is the preprint).

Defining and Measuring the Entrepreneurial Mindset

In their presentation, Rebecca and Helen start with the Kern Foundations three Cs: Curiosity, Connections, and Creating Value.  As pointed out in Kern's Entrepreneurial Mindset 101 video, these are not skills. These are attitudes and dispositions. This is different from technical knowledge and problem solving skills. In contemporary theories of learning, we call these two things disciplinary practices and disciplinary knowledge.  The former is attitudes and dispositions that people who the discipline characterizes as experts "do" in contexts that offer possibilities for action. The latter is the declarative knowledge of facts and skills and demonstrable ability to solve well-defined problems, relatively independent of the context. Because disciplinary practices are so bound to the context where they are learned, used, and demonstrated, they are exceedingly difficult to measure (because measurement aims to control for context) and they are still very difficult to assess in any way that supports valid comparisons across individuals (because the fixed context that allows for valid comparisons inevitably is biased towards some learning and performance context and against others). Given three decades of doomed efforts to measure "21st Century Skills," it is really great to see such a deep embrace of a perspective that most learning scientists have always taken for granted.

Sam Messick (1931-1998)

While I was a postdoc at the ETS Center for Performance Assessment from 1995 to 1997, I became obsessed with this assessment problem and the underlying validity issues. One problem with assessments of disciplinary practices is that they often introduce unknown and largely unknowable amounts of what validity theorist Sam Messick (1994) called "construct irrelevant variance."  This is essentially variation in performance that is irrelevant to the targeted construct. But, Messick also taught me about "construct-irrelevant easiness." This is essentially teaching to the test. Sam was warning us about the validity challenges that were presented by the new performance and portfolio assessments that were widely adopted by states in the early '90s. They then vanished suddenly around 1995 when administrators, legislators, and journalists began to appreciate the profound threats to test validity, along with issues of security, cost, and failure to deliver promised educational improvements.

As Rebecca and Helen point out in their presentation, someone with an entrepreneurial mindset exhibits particular behaviors when the opportunity presents itself:

CURIOSITY

• Demonstrate constant curiosity about our changing world
• Explore a contrarian view of accepted solutions

CONNECTIONS

• Integrate information from many sources to gain insight
• Assess and manage risk

CREATE VALUE

• Identify unexpected opportunities to create extraordinary value
• Persist through and learn from failure

 In their presentation, Rebecca and Helen point to the amazing badge system that was developed by the Supporter to Reporter (S2R) program in the UK. Lucy Neale Lewis and her colleagues at S2R developed one of the most successful of the 29 badge systems funded in MacArthur's 2012 Badges for Lifelong Learning initiative. As we learned in the Design Principles Documentation Project, the S2R medals were awarded for completing sports journalism projects and sharing them over social media. While the introductory badges are quite easy to earn, the higher-level badges require quite a bit of hard work, including feedback from teachers and peers and revisions. While badges list all of the disciplinary knowledge and skills that students presumably gained, they don't actually contain specific evidence. Rather, they state what the earner had to do to earn the badge and most of them link to artifacts (typically videos) and earners' reflections on those artifacts.  

Like two of the other projects we studied whose badge systems are still thriving (Mouse Wins and PBS News Hour Student Reporting Labs), S2R planned a badge system that used completed student work, student reflections on that work, and discussion of that work by peers and experts. Perhaps more importantly, these badges were earned in the process of coming together with peers, classmates, instructors, and/or experts and working together. Because of this and the lack of more specific evidence of competencies, some might dismiss these badges as mere "attendance" or "seat time" badges. But, I beg to differ. I believe that these badges provide compelling evidence of "engaged participation" (I will write more about this later). Meanwhile, none of the seven projects that attempted to build "competency-based" open badges that included detailed measures of specific competencies made it past the pilot stage. If you are interested, here is a paper on those seven projects that I will present this summer in Singapore.  Notably, this included three very well-funded efforts supported by the Gates Foundation's Project Mastery Initiative; this comprehensive summative evaluation of those three projects from the Rand Corporation gave a much more detailed account of the same things our study found.  Both our study and the Rand study confirmed some of the concerns that the Carnegie Foundation raised in their report on the credit hour.  (For the record, I am quite enthusiastic about competency-based education.  In my opinion, the issues raised in these studies and papers need to be addressed to tap its full potential and gain wider acceptance.

Rebecca and Helen have extended the KEEN framework to include other valued aspects of an entrepreneurial mindset, including empathy, resilience and self-awareness. One thing I particularly like is that they have embraced the notions of disciplinary creativity and connections. Many other efforts to motivate and recognize these dispositions lose sight of the fact that if you attempt to recognize, measure, and motivate these dispositions in the abstract, they become quite problematic. In addition to losing their meaning, discipline-free characterizations of these contexts end up being operationalized by their measures. So, it becomes impossible to use those same measures to generate valid evidence that a program that attempts to foster them has actually succeeded. Rather, they are looking at options like student-generated videos, reflections on those videos, and peer and expert endorsements of the badges and the artifacts they contain as evidence of an entrepreneurial mindset.

Motivating Mindsets

With all of the new attention to Mindsets these days, it is worth considering the history of this construct in motivation theory. Its roots are in Goal-Orientation Theory, which itself is rooted in Murray's (1938) Taxonomy of Needs and Maslow's (1954) Hierarchy of Needs. With the cognitive revolution of the 1970s, these frameworks were largely supplanted by Goal-Orientation Theories. Extensive research was carried out regarding the content of goals. The field eventually came to distinguish between intrapersonal goals (i.e., goals that have within-person consequences, including affect, cognition, and subjective organizational factors) and interpersonal goals (i.e., having person-environment consequences, including social relationships and tasks). In contrast to the prevailing behaviorist models of motivation that these theories rejected, the general implication is that motivational practices should facilitate goal setting rather than control behavior.  

This has obvious implications for using badges to motivate. Rather than as a reward for meeting some criteria, badges and badge systems should be used to help learners set appropriate goals and then meet them. Most obviously, badges should help learners activate relevant goals for completing a task. Perhaps more importantly, the badges and the corresponding curriculum should help ensure that goals that are activated are clear, compelling, and presented in such a way that learners understand what they need to do to accomplish them. This is the feature that I suspect many of the less successful badge systems that we studied overlooked. Some of them had a list of very specific competencies and very ambitious plans for getting experts and peers to validate the evidence of these competencies. Consider, for example, the experience of badge system at the Young Adult Library Services Association. Their advisory panel insisted that their badge should be "really valuable" and need to contain convincing evidence that the earner had mastered the Competencies for Librarians Serving Youth. However, given that the badges do not have any pre-defined value, it turns out that very few librarians apparently have the motivation to slog through the many activities and then find a peer to approve the activity. According to project director Linda Braun in 2015, just a handful of the badges had been issued and she had to personally serve as the peer reviewer. Generally speaking, research of the content of goals points to other ways badges might motivate learners. One of these is helping learners set multiple goals. Thus, an Entrepreneurial Mindset badge might help students set proximal goals for completing a project as well as a more distal goal of landing a job that values that mindset. 

The influential goal theorist Carol Dweck is responsible for much of the current interest in Mindset. She helped organize the field around the distinction learning-oriented and performance-oriented goals (Dweck and Legget, 1988, Elliott and Dweck, 1988).  Hundreds of psychologists took up this distinction; have labeled them "ego-involved" vs. "task-involved," but most have settled on mastery-orientation vs performance-orientation:

• Mastery goal orientation is defined as a focus on learning and mastering a task according to self-set standards or self-improvement, learning new skills, trying to accomplish something challenging, or trying to gain insight.
• Performance goal orientation is defined as a focus on demonstrating competence or ability, or how ability will be judged by others, striving to be the best in the class, and avoiding judgement of poor ability.

This distinction became perhaps the most important idea in all of educational psychology. It is so widely appreciated that it does not call for more elaboration here (but see Anderman, 2015). Hundreds of studies have shown that mastery goals lead to desirable behavior, attribution patterns, cognition, and affect, while performance goals lead to undesirable counterparts (here is a nice video about this). One important extension of this work the distinction between approach performance goals (desire to outperform others and demonstrate superiority) and approach avoidance goals (desire to avoid failure and look incompetent) (Elliott, 1999).  

Perhaps the most important extension of this research is studies by Dweck (2000) and others showing that people who are more oriented by mastery goals tend to express an incremental theory of intelligence (the belief that intelligence is malleable) while people who are more oriented by performance goals tend to express an entity theory of intelligence (the belief that intelligence is fixed). Much of the current focus on "Growth Mindsets" aims to help students develop an incremental view of ability so that they set mastery goals for themselves. There are now countless books, movies, and articles about doing so in classrooms, workplaces, and beyond.  One of the most popular motivation textbooks (Schunk and Meece, 2012) presents a widely cited list of ten. Some of these are so obvious that they are unlikely to impact the design of a badge system or curriculum:

• Focus on meaningful aspects of learning activities
• Design tasks for variety, diversity, and interest
• Design tasks that are challenging in terms of learners' capabilities

But, some of the other guidelines are less intuitive. One is give learners choice and control over their activities. Some badge systems are associated with highly organized sequences of activities that give students few choices. But others envision badges as a way to help learners find and pursue learning opportunities that suit them.  Thus one application of goal theory to badges for entrepreneurial mindsets is that a badge system that offers students choice and control over how they develop and demonstrate that mindset are likely to lead to a desirable incremental view of the knowledge of entrepreneurship and the establishment of mastery-oriented goals towards being an entrepreneur.

Two additional implications of goal theory that are closely related are are focus on individual improvement, learning, progress, and mastery and recognize student effort. Western (as apposed to Asian) cultures tend to recognize success over effort, and this has been shown to underlie widespread embrace of entity beliefs and the establishment of performance goals in Western classrooms. It seems that some of the badge systems are likely to emphasize final achievement rather than effort. One solution might be stacking smaller "effort-oriented" badges within a larger "achievement-oriented" badge. These assumptions are implicit in current efforts to define badge "pathways" and "playlists."

One of the implications of goal orientation theory is potentially rather problematic. Shunk and Meece suggest that educators strive to make evaluation private, not public. They argue, "Although it is impossible to make all evaluation private (classrooms are public places), teachers can minimize public evaluations."  As introduced above, the public nature of evaluation of evidence is one of the defining features of digital badges. That was certainly one of the things that celebrated motivation writer Alfie Kohn emphasized in his blanket dismissal of nearly all of the features of open digital badges in his keynote in 2014. 

This gets at something that has guided the design of badges for my own open online courses (as described here). As I elaborated in Hickey and Rehak (2013) and Hickey (2015), this "participatory" approach to assessment distinguishes between public, local, private, and discreet interactions. Nearly all of the work in my classes is carried out in public, via "wikifolios" that everyone in the class can see and comment on. Most of the interaction is "local" discussions that take place between individuals (mostly via threaded comments) that take place between individuals. Most of the points in class are awarded for local reflections that students place directly on their wikifolios. These are carefully worded prompts that ask students to reflect on their engagement rather than their success or achievement. But, each assignment includes a private self-assessment that students can use to see if they are ready to move on. Finally, I characterize my exams as discrete assessments by making them worth relatively few points (perhaps 20-30% of the grade) and I use them to motivate and reward prior engagement. Most importantly, I never teach to the tests and discourage students from trying to memorize things that might appear on the tests. Nonetheless, I still let students include evidence that they met criteria on the module exam in their badge to help increase the value of that badge.

Other implications of goal theories in Schunk and Meece (2012) seem worth exploring when designing badge systems.  One is help students see mistakes as an opportunity for learning. One of the suggestions I had for Rebecca and Helen in creating badges is letting students include examples of their early less-polished work as well as later more-polished work. Imagine two graduates from Rose-Hulman with similarly-impressive senior year entrepreneur badges. But, imagine if one of those students had a similarly polished freshman-year badge while the other was far less polished. A savvy recruiter might recognize that the second student actually learned more during his or her time at Rose-Hulman and might therefore learn more on the job. Other implications worth exploring are use heterogeneous cooperative groups to foster peer interaction; use individual work to convey progress and adjust time requirements for students having trouble completing work; allow students to plan work schedules and timelines for progress. As we are seeing, this is getting complicated pretty quickly. One straightforward framework for fostering growth mindsets is Epstein's (1989) TARGET model that focuses on tasks, authority, recognition, grouping, evaluation, and time (see the last paragraph of Anderman, 2015).

Summary of Goal Theory and Entrepreneurial Mindsets

This post is already overly long.  But, it did let me explore some really fundamental issues about badge design and the challenges of motivating and recognizing disciplinary practices and mindsets. There is certainly a lot more worth exploring but the key point is that any efforts to use badges to motivate and recognize entrepreneurial mindsets should think carefully about what kind of mindset they create. Personally, I don't think that goal orientation theories are going to provide the full guidance that is needed. This is because they unquestioningly embrace modern constructivist theories of learning. These theories are so antithetical to behaviorist theories that they don't even consider the possibility that "extrinsic" incentives can motivate desirable forms of engagement. I am confident that the primary motivator of students at Rose-Hulman is getting the best job possible. Most of the students graduated at the top of their high school class, and many of them probably are striving to be in the top of their Rose-Hulman class. Any badge system that ignores this or tries to counter it is unlikely to be taken seriously. Constructivist theories also focus almost exclusively on goals as they reside in individuals and dismiss the idea of "collective" goals. As I will write later, Connie Yowell, the Education Director at MacArthur and the driving force behind open badges has written eloquently about newer sociocultural theories of motivation (Yowell & Smilie, 1998). In addition to recognizing these more collective forms of motivation, I believe that these theories also provide a coherent way of resolving the tensions between behaviorist and constructivist theories of motivation (Hickey, 2003; 2015). 

References

Casilli, C., & Hickey, D. (2016). Transcending conventional credentialing and assessment paradigms with information-rich digital badges. The Information Society, 32(2), 117-129.

Dweck, C. S. (2000). Self-theories: Their role in motivation, personality, and development. Psychology Press.

Dweck, C. S., & Leggett, E. L. (1988). A social-cognitive approach to motivation and personality. Psychological Review, 95(2), 256.

Elliot, A. J. (1999). Approach and avoidance motivation and achievement goals. Educational Psychologist, 34(3), 169-189.

Elliott, E. S., & Dweck, C. S. (1988). Goals: an approach to motivation and achievement. Journal of Personality and Social Psychology, 54(1), 5.

Epstein, J. (1989). Family structures and student motivation: A developmental perspective. Research on Motivation in Education, 3, 259-295.

Hickey, D. T.  (2003)  Engaged participation vs. marginal non-participation: A stridently sociocultural model of achievement motivation.  Elementary School Journal, 103 (4), 401-429.

Hickey, D. T. (2015). A situative response to the conundrum of formative assessment. Assessment in Education: Principles, Policy & Practice, 22(2), 202-223.

Maslow, A. H. (1943). A theory of human motivation. Psychological Review, 50(4), 370.

Messick, S. (1994). The interplay of evidence and consequences in the validation of performance assessments. Educational Researcher, 23(2), 13-23.

Murray, H. A. (1938). Explorations in personality.

Schunk, D. H., Meece, J. R., & Pintrich, P. R. (2012). Motivation in education: Theory, research, and applications. Pearson Higher Ed.

Yowell, C. M., & Smylie, M. A. (1999). Self-regulation in democratic communities. The Elementary School Journal, 469-490.