Various forms of automated assessment including the familiar MCQs are available through systems such as Canvas and are used daily across the STEM subjects. Up to 40% of assessed work in a given year of a STEM degree is laboratory based, and existing automated assessment systems require correct answers to be pre-determined and therefore can’t account for the variability of real experimental data. Previous approaches have aimed to remove this uncertainty by giving students a known dataset to work with. However, allowing students to record and use their own experimental data is very important for instilling good laboratory practice and encouraging them to take ownership of their work.
MöbiusLab, developed in-house at the UoB’s Collaborative Teaching Laboratory by the EPS EdTech Team, aims to solve this problem. Automatic grading and intelligent feedback ensure that students are rewarded for their efforts and given the opportunity to learn from their mistakes, while also reducing marking load. We have recently completed a successful 1 year pilot of MöbiusLab, handling the majority of post-lab assessment for 2nd year Chemistry.
How does it grade if it doesn’t know the answer?
To put it simply: MöbiusLab doesn’t need to know the answer, it can calculate it. By linking individual answer boxes together MöbiusLab calculates correct answers on-the-fly based on student input. Each answer can have its own criteria for precision and accuracy, and any partial marking scheme can be applied based on these criteria. Allowances can be made for error-carried-forward, equipment error and rounding errors if required. MöbiusLab can also adapt how answers are calculated based on student input. This can be particularly useful when there is some ambiguity in how the student may be handling their data, for example with respect to units.
A simple example of MöbiusLab functionality. The students must use the temperature of a water bath to calculate subsequent values. MöbiusLab uses the temperature entered by the student to calculate correct answers on-the-fly.
What about feedback?
Students often stress the value of personalised feedback, but many assessment systems lack flexibility in this area. We always endeavour to deliver high-quality, response-specific feedback and MöbiusLab has been developed with this in mind. Response-specific feedback can be delivered automatically mid-assignment upon student input, triggered by clicking a button, or delivered after submission.
Automatic feedback: Useful for giving students hints and quick feedback regarding answer format, units, decimal places etc.
Elective feedback: Students choose when (or whether) to receive feedback by clicking a button. Useful for allowing students to check a method or value before continuing.
Enabling new pedagogies
By combining the features described above we’ve been able to not only replicate existing paper-based assessments but design and implement new pedagogical approaches.
Combined In-Lab / Post-Lab Assessment (collaboration with Sarah Horswell, School of Chemistry):
There is often a disconnect between the activities that students complete in-lab and post-lab. Hours of work in the lab can be reduced to a few data points, and context is often lost. By given students a reason to access post-lab worksheets during the lab session, we aim to restore context and reinforce the connection between the experimental and theoretical aspects of chemistry. We have developed a worksheet which not only allows students to enter experimental data and receive immediate feedback, but can also suggest sources of error based student input. Immediate feedback is only available for the first measurement from a set of 5 and students can re-enter their data as many times as they like without penalty, receiving new feedback each time. This allows them to act on feedback before they’ve even left the lab.
Impact and future work
In the coming months we will be working to assess the impact that MöbiusLab has had on our students and the academic community more broadly through a case study, which will include surveys, interviews and user accounts. The wider impact of our work on MöbiusLab is already being felt through our close working relationship with DigitalEd. We are currently in conversation with DigitalEd about incorporating some of MöbiusLab’s features into their base product. Through our own experimentation and by collaborating with teaching staff from across the sciences we are developing new features all the time. The EPS EdTech team will have a stand at the 2019 HEFi conference, so please free to come and ask about MöbiusLab.
If you would like to learn more about MöbiusLab, please contact Dr Thomas Murphy (t.s.murphy@bham.ac.uk).
Follow us on twitter @UoBEdTechEPS