Global times call for global measures: Investigating automated essay scoring in linguistically-diverse MOOCs
Published | June 2016 |
Journal | Online Learning Volume 20, Issue 2, Pages 1-18 |
ABSTRACT
This paper utilizes a case-study design to discuss global aspects of massive open online course (MOOC) assessment. Drawing from the literature on open-course models and linguistic gatekeeping in education, we position freeform assessment in MOOCs as both challenging and valuable, with an emphasis on current practices and student resources. We report on the findings from a linguistically-diverse pharmacy MOOC, taught by a native English speaker, which utilized an automated essay scoring (AES) assignment to engage students in the application of course content. Native English speakers performed better on the assignment overall, across both automated- and human-graders. Additionally, our results suggest that the use of an AES system may disadvantage non-native English speakers, with agreement between instructor and AES scoring being significantly lower for non-native English speakers. Survey responses also revealed that students often utilized online translators, though analyses showed that this did not detrimentally affect essay grades. Pedagogical and future assignment suggestions are then outlined, utilizing a multicultural-lens and acknowledging the possibility of certain assessments disadvantaging non-native English speakers within an English-based MOOC system.Keywords | assessment · automated essay scoring · massive open online courses · multicultural assessment · technology-enhanced language learning |
ISSN | 2472-5730 |
Other number | 2 |
Refereed | Yes |
Rights | Open Access Policy This journal provides immediate open access to its content on the principle that making research freely available to the public supports a greater global exchange of knowledge. |
DOI | 10.24059/olj.v20i2.638 |
URL | https://olj.onlinelearningconsortium.org/index.php/olj/article/view/638 |
Export options | BibTex · EndNote · Tagged XML · Google Scholar |
AVAILABLE FILES
Viewed by 55 distinct readers
CLOUD COMMUNITY REVIEWS
The evaluations below represent the judgements of our readers and do not necessarily reflect the opinions of the Cloud editors.
![](interface/images/icons/star.png)
![](interface/images/icons/star.png)
![](interface/images/icons/star.png)
![](interface/images/icons/star.png)
![](interface/images/icons/star.png)
![](interface/images/icons/star.png)
![](interface/images/icons/star.png)
![](interface/images/icons/star.png)
![](interface/images/icons/star.png)
![](interface/images/icons/star.png)
Click a star to be the first to rate this document
▶ POST A COMMENT
SIMILAR RECORDS
Evaluating the validity and applicability of automated essay scoring in two massive open online courses
Reilly, Erin; Stafford, Rose; Williams, Kyle; Corliss, Stephanie; et al.
The use of massive open online courses (MOOCs) to expand students’ access to higher education has raised questions regarding the extent to which this course model can provide and assess authentic, higher level student ...
Match: Reilly, Erin; Stafford, Rose; Corliss, Stephanie; assessment
Peer-to-peer recognition of learning in open education
Schmidt, Jan Philipp; Geith, Christine; Håklev, Stian; Thierstein, Joel
Recognition in education is the acknowledgment of learning achievements. Accreditation is certification of such recognition by an institution, an organization, a government, a community, etc. There are a number of ...
Match: assessment
Assessment challenges in open learning: Way-finding, fork in the road, or end of the line?
Conrad, Dianne
Growing global commitments to open learning through the use of Open Educational Resources (OERs) are accompanied by concerns over what “to do” with that learning when learners present it to traditional institutions ...
Match: assessment
Open recognition and its enemies (2) — No informal learning without informal recognition
Ravet, Serge
Match: assessment
Redrawing the boundaries in online education through media literacy, OER, and web 2.0: An experience from Brazil
de Siqueira, Alexandra Bujokas; Rothberg, Danilo; Prata-Linhares, Martha Maria; Connolly, Teresa; et al.
This chapter presents lessons learnt after reflecting on a distance learning course based on Web 2.0 tools, which was promoted in order to teach visual communication concepts to students on a teaching degree programme ...
Match: assessment
Flexible paths to assessment for OER learners: A comparative study
Conrad, Dianne; McGreal, Rory
This paper highlights the preliminary findings of a one-year research project (2011) that investigated the fit of recognizing prior learning (RPL) practice and related assessment and transfer protocols to projected OER ...
Match: assessment
Open Education Resources (OER) for assessment and credit for students project: Towards a logic model and plan for action
Mackintosh, Wayne; McGreal, Rory; Taylor, J.
The Open Education Resource (OER) for assessment and credit for students initiative aims to develop a “parallel learning universe” to augment and add value to existing post-secondary education provision by creating ...
Match: assessment
MOOCs in Europe: Overview of papers representing a collective European response on MOOCs as presented during the HOME conference in Rome November 2015
Bang, Joergen; Dalsgaard, Christian; Kjaer, Arne; O’Donovan, Maria; et al.
Table of contents
Foreword
Part 1: Regional MOOC initiatives
Building OOC layers on top of existing courses
(M)OOCs in Iceland: Language and learning communities
EduOpen network in Italy
Part 2: Role media ...
Match: assessment
MOOCS: What The Open University research tells us
Ferguson, Rebecca; Coughlan, Tim; Herodotou, Christothea
This quality enhancement report recommends priority areas for university activity in relation to massive open online courses (MOOCs). It does this by bringing together all The Open University’s published research work ...
Match: assessment
What impacts do OER have on students? Students share their experiences with a health psychology OER at New York City College of Technology
Cooney, Cailean
This article reports findings from a study conducted with students in three sections of a Health Psychology course that replaced a traditional textbook with open educational resources (OER) as the primary course ...
Match: assessment