Utilising question pools within Blackboard online Exams
Rachel Cunliffe – School of Forensic and Applied Sciences
Students are assessed on bone identification. Previously we have conducted these tests in the lab, but it generates a large demand on staff time as it requires multiple repeats to accommodate all the students and their academic support needs, in addition to preparation and marking. The test is made of simple questions (e.g. what bone is this? What side of the body does this bone come from?); and is short (less than 30 minutes); this results in very limited take-up of summer resits – students do not wish to travel to Preston simply to undertake a 25 minute test in the lab. We needed an alternative method of assessment that reduced staff time needs and was easy for students to access.
An online multiple choice assessment using photographs of bones was developed on the Blackboard platform. A pool of questions was developed, and the test options were set to randomly select questions from the pool for each test, so no student had the same questions. The order of questions was randomised, too, so in the unlikely event the same questions were used, they would not be in the same order anyway. The test parameters were set to be equivalent to the lab version; questions appeared one at a time, and students could not go back to previous answers. Pool options allowed for the same proportion of question types as the original test, so there were no concerns that one area of the skeleton would be over-assessed and another omitted, while maintaining the randomisation of question selection. Time constraints are easily adjustable for students who need extra time, and the test was deployed for a 3-day window so the students could complete it at a time that suited them.
Since this test is multiple choice it can now be graded automatically, reducing staff input. The test is deployed online so there is no staff time needed for organising or invigilating multiple repeats. There are some issues with providing image description tags for screen readers and other assistive technologies, but these cases occur minimally and are easily dealt with by arranging a face-to-face test format similar to the previous lab version.
The online pool question format means it has been easy to set up a mock test, which has not been available before; this enhances student experience. Feedback from the students is positive and grades have increased. There has been a small increase in engagement with summer resits, which was one of the reasons this format was used – and the saving in staff time is a huge benefit