MOOCs

Learning Objectives in MOOCs

We are learning more and more about who enrolls in Massive Open Online Classes (MOOCs) and how those students behave.  For example, Harvard and MIT recently released de-identified data from their first 16 MOOCs that ran in 2012-2013 (read more about the Harvard and MIT data sets here and access the actual data here).  The data set includes several variables relating to student activities – for example, whether students visited the course website, watched videos, or completed exams.  These types of measures can tell us a lot about what students do, but it is not clear how much they learned as a result of those actions.

We were interested to find out how much students gained in specific learning objectives as a result of participating in a MOOC.  To do this, we asked students to rate their learning gains in five areas that roughly correspond to Bloom’s taxonomy of learning outcomes.  Bloom’s taxonomy is a classification of learning outcomes that includes both lower-level outcomes (remembering, understanding) and a progression towards higher-level outcomes (evaluating, creating).

We sent a post-course survey to students in Dorian’s Canelas’ Introduction to Chemistry course asking them to indicate the extent to which the course contributed to their progress on five different learning objectives.  The survey was completed by 382 students, and it should be noted that this does not represent a random sample of students in the course.  Rather, the findings generated from this survey are indicative of what some of the more-engaged students experienced.

Of the students who completed the survey, 62% earned a Statement of Accomplishment.  However, even those who did not earn a Statement reported having a very positive experience in the course.  As shown below, when asked to rate their overall experience with the course, the overwhelming majority of all students rated it highly.

The graph below shows student’s self-reported progress on the five learning objectives we asked about.  The percents reported are the percent of students who said that the course contributed “highly” or “very highly” to their progress (other options were “not at all”, “a little”, and “moderately”).

As is typical in a traditional class, most students made significant progress on the lower-level outcomes of gaining knowledge and understanding basic concepts.  However, over half the students reported that they made progress with the higher-level outcomes of applying knowledge to other situations and synthesizing information.  Finally, 42% of students made progress learning to conduct their own inquiry, an objective at the top of the taxonomy.  This group does not include only those who completed the course and earned a Statement.  Of those who did not earn a Statement, 33% said that they made significant progress learning to conduct inquiry.

These numbers highlight the need to think carefully about how we define success in a MOOC.  It is increasingly clear that students who do not earn certificates at the end and who do not meet the traditional metrics of completion are still having meaningful engagements with the course material and accomplishing learning gains.

Kimberly Manturuk