Since “the year of the MOOC” in 2012, the effectiveness of massive open online courses (MOOCs) has been widely debated. Some argue that MOOCs are not an effective mode of instructional delivery because of low completion rates. In the interest of developing alternative indicators of performance this study draws from recent efforts to measure engagement in social media, as well as from research on indicators of student engagement in traditional college courses. Using data from 16 Coursera MOOCs offered by the University of Pennsylvania we calculate standardized access rates for lectures and assessments. While these indicators have clear limitations as measures of educational progress they offer a different, more nuanced understanding of the level and nature of users’ engagement with a MOOC. This paper shows that a very small share of users takes up available opportunities to access course content but notes that the standardized access rates compare favorably with those for social media sites and with response rates to large-scale direct mail marketing programs. For MOOC providers and platform managers, indicators like the ones developed in this study may be a useful first step in monitoring the extent to which different types and combinations of activities may be providing better opportunities for learning.

Keywords

MOOC measurement · MOOCs · post secondary education