As we begin marking our first major assessment items for 202130 just a reminder that an updated BJBS moderation guidance was approved by the Faculty Board late last 2020. It is now in the Policy Library as a Local Instruction and applies to all BJBS moderation activity from 202130 onwards. As outlined below the updated version provides guidance on sampling method, applying moderation adjustments, and on evidencing moderation and evidencing any moderation adjustments.
At the request of the Faculty Assessment Committee (FAC), a paragraph has very recently been inserted to ensure procedural fairness for those suspected of student academic misconduct (SAM). That updated paragraph (in italics below) has been endorsed by the Executive Dean and is now reflected in the Local Instruction in the Policy Library.
Guidance on sampling method
Although the selected sampling method should give all scripts in the marker cohort an equal opportunity of being selected the only method that guarantees this is random sampling when a random number generator is used to make selections, but this is time-consuming and complicated. Other methods include personal judgment, block, haphazard, stratified, and systematic sampling.
The sampling method approach to be used is a modified form of block and stratified sampling. Moderators select scripts for moderation in proportion to each of the substantive grades awarded (stratified sampling). For example, if 40% of the cohort have been awarded a PS grade then 40% of the sample is selected from those awarded a PS grade. Within each passing grade sample, moderators select the scripts closest to the grade below (block sampling). Within the FL and AE grade samples, moderators select the scripts with the highest marks.
Guidance on marking and moderating scripts in suspected SAM cases
To ensure procedural fairness for students suspected of student academic misconduct (SAM) their script must be marked in full prior to its referral for investigation. In all cases where the allegation is not upheld, or the student receives a caution or a penalty that constitutes less than 50% of the total available marks for the assessment (e.g. zero marks for 1 of 5 questions), the script must be included in the moderation sample and subject to the moderation processes outlined below.
Guidance on evidencing the sample selected
To evidence the sample selected, the moderator picks the sample (as outlined above) and then indicates, using the notes fields in Grade Centre, which scripts are to be selected for moderation for each marker cohort.
Guidance on applying moderation adjustments
If moderation of the selected sample identifies inconsistencies against the marking criteria of:
- <5% of the total mark: no adjustments to any scripts are required
- >5% of the total mark across much of the sample selected: remark of all scripts in the marker cohort
- >5% of available marks in specific section(s): remark of section(s) in all scripts in the marker cohort
- >5% of the total mark in a minority of scripts selected: adjust the marks only in identified scripts
For all scripts (or parts of scripts) that are remarked, repeat the moderation process until the moderator is satisfied that marking is consistent with the marking criteria. The moderator should not expand the moderation sample size unless they believe additional samples will provide new information about the consistency of marking against the marking criteria. The moderator cannot apply a blanket percentage adjustment to all scripts.
Guidance on evidencing moderation and moderation adjustments
There are two equally important aspects to evidencing moderation and moderation adjustments:
- Evidence that the moderator has moderated the selected sample: for each script selected, the moderator records enough information (e.g. student name(s), student number(s), assessment of marking quality, marks awarded by marker and moderator) to facilitate a reperformance of the moderation by an alternate. This information must either be fully recorded in QUASAR or a hyperlink/file path provided in QUASAR that links to the detail, in a format decided by each HoS for their school.
- Evidence that any moderation adjustment mark is reflected on scripts: any adjustments to marks as a result of the moderation process (which includes remarks) must be adjusted on each affected script. Adjusted marks must be internally consistent, and in the case of exams, the coversheet must reconcile with the moderated mark recorded for the task in Grade Centre.