The OER Knowledge Cloud makes use of cookies. By continuing, you consent to this use. More information.
Global times call for global measures: Investigating automated essay scoring in linguistically-diverse MOOCs
Reilly, Erin · Wiliams, Kyle · Stafford, Rose · Corliss, Stephanie · Walkow, Janet · Kidwell, Donna

PublishedJune 2016
JournalOnline Learning
Volume 20, Issue 2, Pages 1-18

This paper utilizes a case-study design to discuss global aspects of massive open online course (MOOC) assessment. Drawing from the literature on open-course models and linguistic gatekeeping in education, we position freeform assessment in MOOCs as both challenging and valuable, with an emphasis on current practices and student resources. We report on the findings from a linguistically-diverse pharmacy MOOC, taught by a native English speaker, which utilized an automated essay scoring (AES) assignment to engage students in the application of course content. Native English speakers performed better on the assignment overall, across both automated- and human-graders. Additionally, our results suggest that the use of an AES system may disadvantage non-native English speakers, with agreement between instructor and AES scoring being significantly lower for non-native English speakers. Survey responses also revealed that students often utilized online translators, though analyses showed that this did not detrimentally affect essay grades. Pedagogical and future assignment suggestions are then outlined, utilizing a multicultural-lens and acknowledging the possibility of certain assessments disadvantaging non-native English speakers within an English-based MOOC system.

Keywords assessment · automated essay scoring · massive open online courses · multicultural assessment · technology-enhanced language learning

Other number2
RightsOpen Access Policy This journal provides immediate open access to its content on the principle that making research freely available to the public supports a greater global exchange of knowledge.
Export optionsBibTex · EndNote · Tagged XML · Google Scholar

638-4274-1-PB.pdf · 200.1KB29 downloads

Viewed by 37 distinct readers


The evaluations below represent the judgements of our readers and do not necessarily reflect the opinions of the Cloud editors.

Click a star to be the first to rate this document


Evaluating the validity and applicability of automated essay scoring in two massive open online courses
Reilly, Erin; Stafford, Rose; Williams, Kyle; Corliss, Stephanie; et al.
The use of massive open online courses (MOOCs) to expand students’ access to higher education has raised questions regarding the extent to which this course model can provide and assess authentic, higher level student ...
Match: Reilly, Erin; Stafford, Rose; Corliss, Stephanie; assessment

MOOCs in Europe: Overview of papers representing a collective European response on MOOCs as presented during the HOME conference in Rome November 2015
Bang, Joergen; Dalsgaard, Christian; Kjaer, Arne; O’Donovan, Maria; et al.
Table of contents Foreword Part 1: Regional MOOC initiatives Building OOC layers on top of existing courses (M)OOCs in Iceland: Language and learning communities EduOpen network in Italy Part 2: Role media ...
Match: assessment

MOOCs - A review of the state-of-the-art
Yousef, Ahmed Mohamed Fahmy; Chatti, Mohamed Amine; Schroeder, Ulrik; Wosnitza, Marold; Jakobs, Harald
Massive open online courses (MOOCs) have drastically changed the way we learn as well as how we teach. The main aim of MOOCs is to provide new opportunities to a massive number of learners to attend free online courses ...
Match: assessment

MOOCS: What The Open University research tells us
Ferguson, Rebecca; Coughlan, Tim; Herodotou, Christothea
This quality enhancement report recommends priority areas for university activity in relation to massive open online courses (MOOCs). It does this by bringing together all The Open University’s published research work ...
Match: assessment

Developing a strategic approach to MOOCs
Ferguson, Rebecca; Scanlon, Eileen; Harris, Lisa
During the last eight years, interest in massive open online courses (MOOCs) has grown fast and continuously worldwide. Universities that had never engaged with open or online learning have begun to run courses in these ...
Match: assessment

Assessment challenges in open learning: Way-finding, fork in the road, or end of the line?
Conrad, Dianne
Growing global commitments to open learning through the use of Open Educational Resources (OERs) are accompanied by concerns over what “to do” with that learning when learners present it to traditional institutions ...
Match: assessment

Unpacking MOOC scholarly discourse: A review of nascent MOOC scholarship
Ebben, Maureen; Murphy, Julien S.
The rapid rise of MOOCs (Massive Open Online Courses) signals a shift in the ways in which digital teaching and learning are engaged in and understood. Drawing upon a comprehensive search of nine leading academic ...
Match: assessment

Assessment in Massive Open Online Courses
Admiraal, Wilfried; Pilli, Olga; Huisman, Bart
Open online distance learning in higher education has quickly gained popularity, expanded, and evolved, with Massive Open Online Courses (MOOCs) as the most recent development. New web technologies allow for scalable ...
Match: assessment

Presentation to UNCTAD's advisory group on "Developing skills, knowledge and capacities through innovation: E-Learning, M-Learning, cloud-Learning"
Downes, Stephen
Match: assessment

Is peer review an appropriate form of assessment in a MOOC? Student participation and performance in formative peer review
Meek, Sarah E.M.; Blakemore, Louise; Marks, Leah
Many aspects of higher education must be reconceptualised for massive open online courses (MOOCs). Formative and summative assessment of qualitative work in particular requires novel approaches to cope with the numbers ...
Match: assessment