tag:blogger.com,1999:blog-4778608909198518776.post6605905594519318792..comments2024-03-04T17:08:30.238-05:00Comments on re-mediating assessment: On MOOCs, BOOCs, and DOCCs: Innovation in Open CoursesAnonymoushttp://www.blogger.com/profile/11170497669879668928noreply@blogger.comBlogger8125tag:blogger.com,1999:blog-4778608909198518776.post-27998305207915285952013-09-17T15:26:46.325-04:002013-09-17T15:26:46.325-04:00Mark--
Great Question. Mary-- Yup. It gets pretty...Mark--<br />Great Question. Mary-- Yup. It gets pretty messy pretty quickly. I am actually not a very big fan of standardization because I think learning and knowing are highly contextual. Case in point, somebody on the Facebook page for the Educational Assessment BOOC questioned whether somebody who completed the course (which is a trimmed down version of a pretty comprehensive grad-level course) could be deemed an "expert." I suspect this question came from an assessment researcher or scholar. Our course is intended to make someone an expert in their specific context. In that past students who complete my course have told me that even before the course is over that they are deemed the residential assessment expert at their school or program. This is because they know more about assessment than anybody else in that particular setting or domain. Decades of failed assessment reforms have convinced me that most of the assessment solutions designed and promoted by external "experts" fail because the ignore local expertiseDaniel Hickeyhttps://www.blogger.com/profile/09885916528215868949noreply@blogger.comtag:blogger.com,1999:blog-4778608909198518776.post-88640747799405426582013-09-17T14:39:42.198-04:002013-09-17T14:39:42.198-04:00Daniel wrote, "At more than a dozen participa...Daniel wrote, "At more than a dozen participating colleges, professors will offer their own courses for credit and create their own assignments and assessments."<br /><br />I'm wondering how fractured this could become and whether, in some cases, it amounts to duplicated and/or wasted effort in the area of assessment. It would make sense for professors to share their assessments with one another, with the understanding that they are all free to use what's useful to them.<br /><br />I also wonder what happens when it comes to yet another term in the proliferation of acronyms: OLAs, or "online learning activities" (http://bit.ly/15K6BF6). If we start taking bits and pieces of MOOCs (or other forms of online learning) and fitting them Lego-like into personalized courses, then how do we devise proper assessments? Is there a way to pull them together so that bits and pieces can be used for customization? It feels as if the hard part is finding the right balance of centrifugal and centripetal forces in all this.Mark Vickershttps://www.blogger.com/profile/03379731144200212097noreply@blogger.comtag:blogger.com,1999:blog-4778608909198518776.post-43293097205324504722013-09-16T09:23:20.435-04:002013-09-16T09:23:20.435-04:00Nate--
I don't think you have to choose one si...Nate--<br />I don't think you have to choose one side of the fence or the other! The point is that open content curation will more appropriate in some contexts while instructor defined content will be more appropriate in other contexts. The real questions is when and for what? <br /><br />In the Assessment BOOC, we will open up the content significantly when we get to assessment policies because they differ by state and and role, and because there is a lot of controversy. We can't even scratch the surface of the policies about standardized testing for example so there is no point in even trying. Rather we want students to begin engaging with professional peers. And we really want non-peers like teachers and adminstrators learning how each other think about policies.<br /><br />This is all very different than, say the guidelines for making good performance assessments. Our text has a nice straightforward set of guidelines that students should practice using before we open things up.Daniel Hickeyhttps://www.blogger.com/profile/09885916528215868949noreply@blogger.comtag:blogger.com,1999:blog-4778608909198518776.post-22049524235499098622013-09-06T16:37:19.247-04:002013-09-06T16:37:19.247-04:00I was excited about this model of online course wh...I was excited about this model of online course when I saw the announcement. As you noted, Dan, it does come a lot closer to Siemens' and Downes' connectivist model than the common pattern of 2012-13's institutional MOOC. As someone who usually instigates trying to "connectivize" my experience of centralized MOOCS that I'm in by spreading the discussion out from the closed forum of each MOOC site, I was excited to see a university-sponsored effort try a move away from centralization as well.<br /><br />As far as the distribution of content curation goes, I sit on both sides of the fence. I like giving individual learners and guides the leeway to decide that a particular resource should be included or not in the course, but I also recognize the value of an expert's curation of a course of study through the literature of a field. As a learner, I don't trust myself to know in advance what the important readings are in a field, and I know that I will never get to that level of expertise in most of the fields I enter. A well tailored reading list is my friend in these contexts. The connectivist model of an online course doe not completely do away with an instructor-curated syllabus, it just admits that participants will naturally bring in their own networks, resources, and examples to the discussion. For the Assessment BOOC, decentralizing the reading list wouldn't mean abandoning a textbook, it just might mean bringing in student-identified examples of blog posts about assessment techniques in the wild.<br /><br />I do have a separate curiosity about the Distributed Open Collaborative Course. How is this model open? When I saw the announcement, I looked around for signup links or spaces where discussion would be occurring, and it seems that there are no open spaces associated with the course. There is also value in closed, protected discussion spaces, but it leaves me wondering what sense of "open" the organizers were using when they chose it. Stephen Downes frequently writes about how the American institutional model of the MOOC (which he calls an xMOOC to distinguish from his connectivist cMOOCS) has tended to attempt to break down the MOOC acronym one word at a time. In reaction to the MOOC, some providers are delivering experiences that are either not free (open), massive (the BOOC, perhaps), online (with in-person components), or even courses (see Harvard's new thinking about breaking down courses into modules: http://news.yahoo.com/napster-moment-education-130900966.html ). So it's interesting to see a breakdown of the MOOC actually move closer to the original connectivist model, even though they would criticize this DOCC for lack of openness.Nate Ottohttps://www.blogger.com/profile/06488791026991936169noreply@blogger.comtag:blogger.com,1999:blog-4778608909198518776.post-42966971806487282942013-09-03T21:03:53.944-04:002013-09-03T21:03:53.944-04:00Sharon--
My theoretical orientation always points ...Sharon--<br />My theoretical orientation always points me to search for evidence of productive forms of disciplinary engagement. I define productive primarily in terms of engagement in discourse that connects the disciplinary concepts of the course with the varied contexts and experiences of the participants. You can interpret that by looking at discourse, but it is rather hard to formally assess whether or not that discourse has occurred by looking at portfolios or other artifacts. We use portfolios (actually wikifolios) extensively in my assessment BOOC, but we will also have formal assessments because they helpful at ensuring (and motivating) transfer of individual understanding from those interactions. My main point is that you have to be very carefully assessing artifacts like portfolios as you can really screw up the discourse that they can support. If the discussion around portfolios is not disciplinary but rather about what you have to do to get a good grade, that is a bad sign.Daniel Hickeyhttps://www.blogger.com/profile/09885916528215868949noreply@blogger.comtag:blogger.com,1999:blog-4778608909198518776.post-18489172952070490022013-09-03T16:57:52.060-04:002013-09-03T16:57:52.060-04:00For sure, "participation" means many dif...For sure, "participation" means many different things, as you note. But still, for every learner, it would be helpful to know if the DOCC is really contributing to the following goals, and how--acknowledging the roles of place, institutional cultures and student backgrounds. How do we know if we have:<br /> -developed ethical and equitable practices for more socially just global communities [we have to define these phrases, of course]<br /> -developed innovative uses for digital technologies that serve important cultural and social needs, anywhere and everywhere [this seems somewhat measurable through portfolios and projects]<br /> -identified and preserved the history of feminist contribution to technological innovation [I guess we count the quality and numbers of contributions to Wikipedia for example]<br /> -advance feminist principles of social justice in creating educational models and pedagogies for the future [I guess here we need to specify what outcomes we think align with these principles]<br /><br />In any case, I am encouraged that you think portfolios and contracts make sense!<br /><br />Sharon Irishhttp://sharonirish.orgnoreply@blogger.comtag:blogger.com,1999:blog-4778608909198518776.post-55899253406115304362013-09-03T16:13:51.033-04:002013-09-03T16:13:51.033-04:00Sharon--
Thanks for your comment and suggestions. ...Sharon--<br />Thanks for your comment and suggestions. I (sheepishly) revised the post to reflect your point about the broader organization behind this effort. <br /><br />As for the issue about assessment.... I did not really get into the issue of assessing outcomes, but you are right in showing how they loom. I think that your suggested assessment practice makes perfect sense. It would be pretty silly to standardize the assessment for something that is so distributed. That actually is a pretty good basis for evaluating whether a particular set of instructional goals can be distributed. In some cases like my assessment course, it really is possible to define a particular set of knowledge that can be meaningfully assessed and possibly even measured. But in the case of FemTechNet (and in some of the other courses I teach)the learning goals are fundamentally about learning to participate in the practices of a community. And in the case of a community like FemTechNet, both the community and the practices are nascent and emergent. So contracts, portfolios, papers, all make sense because they provide evidence of participation. Perhaps most importantly they can reflect the fact that participation means different things to different participants. even at the same site.<br /><br />Good luck with your endevour. I look forward to seeing how things go! Daniel Hickeyhttps://www.blogger.com/profile/09885916528215868949noreply@blogger.comtag:blogger.com,1999:blog-4778608909198518776.post-35879251771009811002013-09-03T14:12:34.330-04:002013-09-03T14:12:34.330-04:00Hello, I understand your point about distributed a...Hello, I understand your point about distributed assessment being a problem, particularly if you are concerned about everyone acquiring core knowledge in a particular field. I am one of the instructors teaching a Distributed Open Collaborative Course (DOCC), in my case, a graduate seminar at the University of Illinois, Urbana-Champaign. For our course, we are writing contracts with each student to align with their goals and needs, rather than having a set of standard assignments. That is part of the DOCC idea: using thematic materials, we have designed 16 different courses at 16 different institutions--from small liberal arts colleges to large research universities, from "Hispanic-serving institutions" to community-based courses. While Anne Balsamo has been a key leader for the DOCC and for FemTechNet, it isn't accurate to say "Balsamo's DOCC." Follow us on femtechnet.org, or on Twitter @DOCC2013. Thanks!<br />Sharon IrishSharon Irishhttp://sharonirish.orgnoreply@blogger.com