Mona Information Technology Services, The University of West Indies, Mona

The focus here has always been on finding ways to mark and analyze Multiple Choice Examinations as quickly and as accurately as possible. The analysis of results is based on the individual questions as well as on the examination as a whole. Lecturers can easily spot “weak” questions from this analysis.

Originally, data from all of the examination scripts were put through a manual double-entry and automated verification process. More recently, students’ scripts were scanned electronically with the intention of speeding up the process by removing the manual data entry step. This has not worked as well as expected because students appear to have problems with shading ID numbers and responses:-

Some forget to both shade the ID and write their ID in the space provided, in case there was an error in the coded ID
Some write the ID and do not shade it, so it is a manual process to get that ID into the system
Some do not follow instructions for shading. They shade multiple areas in the same column / row for their ID
Some shade such wide areas as to overlap into the next option, registering as unclear responses
While erasing is acceptable, some students leave too many large, untidy smudge marks on the script, again resulting in the need for manual interpretation and correction
Also on the issue of erasing, some students re-make the pre-printed marks if they happen erase it, and this mark is also read as an input
Some students begin answering on Side 2 (/Paper 2) of the answer form.

The result of these problems is that considerable manual effort is required to locate the scripts in error, interpret the data and make the corrections manually. See correct method for shading. One of the major examinations processed by this method is the English Language Proficiency Test, which further incorporates analysis of pilots for the papers, the results of the examination and the generation of final letters to candidates.

Also in an attempt to make improvements in the speed of the examination marking service offered, an Automated Test Scoring application was set up. It was envisaged that Lecturers/Administrators could use this walk-in service particularly to mark smaller examinations, rather than wait in the queue for MITS staff to run the process for them. However, the service has not yet been made available as from experience in marking these papers, the large number of errors made by students would make for a frustrating experience in marking by the Lecturer or Administrator.

This examination is processed from the collection of demographic data for candidates from all regions of the Caribbean and other parts of the world, through the generation of pre-examination reports, the processing of the candidates’ responses, final ranking based on criteria specified by Outreach and Awards, and the generation of post-examination reports.