Gradescope Proof of Concept

Gradescope: A pilot solution for time consuming and troublesome marking

Western is piloting an application that can ease the burden of marking.  This is particularly useful for STEM subjects that use a range of quiz type questions and technical reports.  Gradescope, developed by Turnitin, can manage quiz questions and formats that require a handwritten response in areas such as mathematics, science and engineering.  It can facilitate the development of a rubric for marking these types of responses and provide analytics data on student performance.  This can be accomplished with time saving efficiencies relating to marking, improved effectiveness of assessment, and enhanced learning guidance for students.

Key features are:

  • Time saving for markers. Gradescope can group student responses so that each marker can grade all of the responses to a single question if necessary. This also applies to handwritten responses to questions and questions that requires students to draw a graph or diagram or submit an image of their work.
  • A rubric can be developed that provides students with feedback and partial marks if parts of the question are not answered correctly.  The rubric allows markers to only have to click once on a response to provide a grade and feedback.  Additional comments that are unique to the student can also be added.
  • Rubrics can be developed dynamically while the test is being marked if additional examples of misinterpretation emerge.  Prior marking is auto-corrected if this occurs.
  • Analytics data from marking can be used to refine the effectiveness of the test.

The value of Gradescope in assisting with marking and feedback has led to wide adoption in the tertiary sector.  The system’s capability to provide analytical data has been used by academic staff to further enhance and refine assessments. For example, Stephens (2017) used the technology to assist with marking for a very large class.  Additionally, Stephens used Gradescope data was analysed to help consider the alignment between assessment questions and subject learning outcomes (SLOs).  He found that most assignments only addressed a few SLOs, however the exam addressed most.  He also found that one outcome was not addressed by any questions.  It was decided that this topic needed to be introduced earlier in the subject to ensure it was assessed.  Taylor et al (2020) took Gradescope’s analytic capability a step further.  Their project was to develop a computer science concept inventory.  A range of test quiz questions were developed following open ended question development and testing to clarify understanding of the fundamental concepts.  Gradescope was used to code data from student answers to test questions, so that issues could be followed up relating to reliability and validity of questions.  This was an important step in the development of the concept inventory.

An important issue with assessment is the reliability of assessment results.  This is undermined if there are allegations of cheating.  Gehringer et al (2021) discuss the use of Gradescope to detect cheating.  The highlight a tool called the Examinator, developed by Apoorv et al (2020), uses Gradescope data to detect similarities in wrong answers that are unlikely to occur by chance.  This tool successfully detected (by student admission) examples of cheating in tests.  Work on this aspect has not been widely developed and applied, however it demonstrates the potential capability of Gradescope analytics.

Gradescope: How I Use It and Why I Love It (opens in a new window) – Yale University

Are you interested in testing this tool in your subject? Please fill in our EOI form (opens in a new window).

If you have any further questions, feel free to contact Stephanie Bourke.

References

Apoorv, R., et al. (2020).  Examinator: A Plagiarism Detection Tool for Take-Home Exams.  https://dl.acm.org/doi/abs/10.1145/3386527.3406723

Gehringer, E. F., Menon, A., & Wang, G. (2021, July 26). Tools for Detecting Plagiarism in Online Exams. 2021 ASEE Virtual Annual Conference Content Access. https://peer.asee.org/tools-for-detecting-plagiarism-in-online-exams

Stephens, A.W. (2019).  Assessing Student Learning Using a Digital Grading Platform.  Applied Economics Teaching Resources 1:1.

Taylor, C., et al. (2020).  The Practical Details of Building a CS Concept Inventory. SIGCSE ’20, March 11–14, 2020, Portland, OR, USA.  https://dl.acm.org/doi/pdf/10.1145/3328778.3366903