Definition of Anonymous Marking for Online Assessment
QSC policy describes anonymous marking as being a process whereby “student assessments are marked without the student’s name or identity being made known to the marker or moderator of the work at the time of marking.”
What Proxy Identifier Should be Used Instead of the Student Name?
Why is a proxy identifier needed at all? Because staff, sometimes working offline with downloaded files, need a robust, convenient way to refer to any given student, reconcile their markings with those of other markers, and update the student’s record in the absence of a name.
Until a technical solution becomes available, students should be instructed to include this proxy identifier in both the title and file name of their submission.
The recommended proxy identifier is the Student Number.
The proxy identifier isn’t generated automatically, so students need be instructed to title their submission and file name so as to include it.
Enabling Anonymous Marking
Anonymous marking allows you to grade student’s work without seeing any identifiable student information. In a situation where you are required to reveal a student’s name, such as instances of suspected plagiarism, an audit trail will be created, accessible by your account administrator.
Anonymous marking is set at assignment level. To enable anonymous marking for an assignment, you will find the option during assignment creation. Anonymous marking cannot be enabled after a submission has been made; however, if there have been no submissions to the assignment, you are able to edit this option in the settings.
- Upon login, you will be greeted with a list of your courses in the Moodle Dashboard. Select the course that you wish to create an assignment with anonymous marking enabled.
- From the Administration menu, or from the top of the page, remember to Turn editing on.
- Select the Add an activity or resource button for the relevant time period.
- An Activities and Resources pop-up box will appear. Select the Turnitin Assignment 2.
- Continue by clicking the Add button.
- Enter the assignment details, ensuring you select Yes alongside Anonymous Marking.
- To save, select Save and return to course or Save and display.
Enabling Anonymous Marking in Existing Assignments
- Upon login, you will be greeted with a list of your courses in the Moodle Dashboard. To enable anonymous marking in an existing assignment, select the course in which the assignment resides.
- From the Administration menu, or from the top of the page, remember to Turn editing on.
- Click the Edit button alongside the assignment you wish to edit.
- Click Edit Settings from the top of the drop-down menu.
- Edit any necessary settings, ensuring you select Yes alongside Anonymous Marking.
- To save, select Save and return to course or Save and display.
Identifying an Anonymous Student
Once a student has submitted to an assignment, you may reveal their name, providing you have reason to do so. Revealing a students name is irreversible and will be audited to ensure that student information is revealed for an acceptable reason. Account administrators will receive notice of your anonymous marking deactivation.
In the assignment inbox, you’ll find that all student names are filled with a link titled Anonymous Marking Enabled.
- If for any reason you must reveal a student’s name, select Anonymous Marking Enabled from the assignment inbox.
- You’ll be asked to provide a reason for unmasking your student’s name. State your reason and click the Reveal button.
- Your student’s name will now be visible in the Student column of the Submission Inbox.
Notifying Non-Submitters
Turnitin’s non-submitter notification feature gives you the ability to contact students who have not yet submitted to an assignment. You can send a reminder at any time during the submission process – including once the post date has passed.
- Navigate to the submission inbox for the class you would like to send a notification to.
- Select Notify Non-Submitters from the submission inbox.
- Enter a subject line and message for the notification and select Send email to distribute the message to all students who have not yet submitted. Please consider the wording of the message so not to cause concern or stress to those with extensions as the communication will go to all who have not submitted regardless of whether they have an extension or not.
Marking an Assignment as a Multiple Marker
Multiple Markers (Beta) introduces the ability for integration users to allow multiple instructors to mark an assignment. Multiple Markers functionality appears within the Feedback Studio and gives each instructor their own marking layer on an assignment. This marking layer can then be toggled on and off, just like any other layer in Turnitin Feedback Studio, to provide multiple sources of feedback for a student.
Marker Layers
In opening a student’s submission, the Turnitin Feedback Studio will activate your layer by default. Your layer will be retained only if you leave a comment on the paper.
Your Layer
Once you have left feedback within your layer, the feedback will then be made available for other instructors to view. You must wait for other instructors to also leave feedback in order for their layers to appear within your Feedback Studio.
To view all marker layers for a paper, each marker must have accessed the paper and left feedback. This will then cause the marker’s name to appear within the Active Layers side panel.
Your initials will appear alongside any comments you leave, helping your students to differentiate between the multiple markers who have left feedback on their piece of work.
Multiple Marker Layers
Any other marker layers will appear in the same location as your own. You can turn these layers on and off like any other by checking and unchecking the relevant checkbox.
Enabling other marker layers will make their comments appear alongside your own.
Allocating Assignments between Multiple Markers
If your team split up marking between multiple markers within the same assignment, we would recommend allocating via the Turnitin paper Id number.
Click on your assignment link where you find all the assignments, and then click on the Paper Id column. This should order the paper Id from smallest to largest. You can then allocate numbers from **** to ***** to each marker.
Guides
- Marking Workflow-Marker Allocation-Marker Layers
- Moodle Assignment – Restricting Access
- Grader Report Export
Moderation
The following section includes guidance on how different forms of moderation can take place when using a Moodle Assignment for anonymous marking. Please refer to the University Code of Practice for Assessment for definitive guidance on the approaches you should be adopting.
Open Second Marking
Full Open Second Marking
Since second markers need to view the markings of first markers, the order of marking is important.
First markers access the submission inbox and can carry out marking using Turnitin’s Grademark. At an agreed date and time (before the Post Date), the second marker accesses the same inbox. Depending on local policy, second markers can use Grademark to add their markings alongside the first marker’s.
One consideration here is whether, on the Post Date, students are expected to see the second marker’s comments as well as the first marker’s. If only the first marker’s, then the second marker accesses Turnitin to view the submission and first marking, but records all their own markings outside Turnitin on a marksheet provided (this enables record keeping, which the alternative practice of deleting second marker comments in advance of the Post Date does not).
Another consideration is whether there is more than a single first and single second marker. Where a group of markers is involved, there is a need to divide the work between them. The anonymised inbox allows sorting by submission title, so if students are instructed to begin the title their submission with their Student Number, it should be possible to sort by these (inevitably a few students will make errors, but Moodle provides ways to reconcile work with a mistyped Student Number). Students should also include their Student Number in the name of their uploaded file.
Where individual markers are allocated marking based on subject specialism, students may be instructed to include a very short identifying phrase – perhaps the question number – in the submission filename and submission title, so that markers can view the contents of the inbox and identify the work they need to mark.
Sampled Open Second Marking
Where Grademark has been used and second marking precedes the Post Date, the submission inbox allows the second marker to apply the sampling criteria by applying the sort function the numeric mark column so as to easily find firsts, fails, upper borderlines, etc. In addition their attention could be directed to any particular submissions with reference to the PaperID.
Where Grademark has been used and second marking needs to happen after the Post Date, then an intermediary accesses the submission inbox and sorts it to identify a sample which is then downloaded and securely passed, outside Turnitin, to the second marker. If this can be arranged after marking is complete but before the Post Date de-anonymises student work, this both potentially saves a great deal of time for whomever carries out this task, and also enables a marker to do it (because the work is still anonymous). However, if the Post Date has passed then an intermediary needs do it, manually re-anonymising all submissions by removing or redacting the student name Turnitin appends after the Post Date.
Open Second Marking After the Post Date
Currently in Turnitin, the Post Date permanently removes student anonymity.
As for sampled open second marking, where Grademark has been used, it saves time to download the still-anonymous inbox after marking is complete but before the Post Date de-anonymises student work. However, if the Post Date has passed, an intermediary needs to carry out the task, re-anonymising all submissions manually by removing or redacting the student name Turnitin appends after the Post Date.
Where marking has taken place outside Turnitin, the process is more convoluted. If this happens before the Post Date then the original files may be downloaded from Moodle – before securely passing to the second marker it is important to check that the Student Number is present in the filename. If after the Post Date then an intermediary downloads the original files, re-anonymises these by removing the name Turnitin appends to the submission filename, ensures the Student Number is present in the filename, and securely passes these to the second marker along with the first marker’s markings.
Blind Second Marking
Full Blind Second Marking
Since the order of marking is irrelevant here, markers can take advantage of the online environment by both accessing the work they need to mark at the same time; the first need not wait for the second and both can use the entire marking period. Markers will need to toggle marker layers on/off in Grademark in order to mark blind.
It is now possible to bulk-download the Turnitin inbox of anonymised work, so one marker can give feedback via Turnitin, while the other looks at the work outside Turnitin.
Sampled Blind Second Marking
The guidance is similar to that for ‘Full Blind Second Marking’, except that it is on selected submissions. So markers would either need to be instructed about how to sort the inbox to take the sample, or an intermediary would need to download and re-anonymise the original submissions and send these securely to the blind second marker outside Turnitin, along with any marksheets required.
Blind Second Marking After the Post Date
See the section on ‘Open Second Marking After the Post Date’ – the difference would be that the intermediary would pass anonymised original submissions (not the marked ones) to the second marker with a blank marksheet (not a completed one).
Moderation
The moderator is tasked with checking the assignation of marks on a full set of submissions. If Grademark has been used and moderation happens before the Post Date, moderators can view the contents of the submission inbox on Turnitin, including the markings. If markings have been made outside Turnitin, then these need to be sent to the moderator either by the markers or via an intermediary.
Sampled Moderation
External Examiners sometimes undertake sampled moderation. Unless a window of opportunity could be made before the Post Date lifts anonymity, an intermediary would need to pass the Examiner(s) a downloaded, re-anonymised sample of marked work as set out in ‘Open Second Marking after the Post Date’ or ‘Blind Second Marking After the Post Date’, above.
Moderation After the Post Date
The guidance here is the same as for ‘Open Second Marking After the Post Date’.
External Examining
Since External Examiners are routinely provided with a manually-created Moodle account, they should be given the ‘External Examiner’ role in a Moodle course area, which allows them access to its respective submission inboxes.
Sampled External Examining
The guidance here is the same as for ‘Open Second Marking After the Post Date’. The submission inbox allows sorting by various columns including numeric mark, allowing Examiners to easily find firsts, fails, upper borderlines, and also allows submissions to be randomly selected. In addition their attention could be directed to particular submissions with reference to the PaperID.
External Examining After the Post Date
The guidance here is the same as for ‘Open Second Marking After the Post Date’.
Frequently Asked Questions
What do markers need to know about avoiding allegations of intentional, rather than unconscious, bias?
Along with Equality and Diversity initiatives and other governance structures, this guidance is intended to help staff avoid allegations of unconscious bias. In the case of an allegation of intentional bias, the student would need to provide supporting evidence according to the Appeals & Complaints Procedure. Where there are sufficient grounds to warrant further investigation, Moodle may be able to provide some records which inform the case, since it keeps track of page views for all its users. The implication here is that if, during the time they are marking, markers avoid consulting the Participation Report, or Logs for the assignment being marked or previous de-anonymised assignments, they will not have been able (on Moodle) to associate names with work.
Can’t staff who are supposed to be anonymously marking after the Post Date simply look up students’ names on turnitinuk.com?
This is technically possible where a marker has a Tutor role on the Moodle course area. However, it would require a purposeful decision, on the part of a marker, to flout the anonymous marking policy. Similar opportunities for malpractice exist in many areas of academia as a consequence of the high degree of workplace autonomy academia requires. To design an entire EMA approach round an assumption of deliberate malpractice on the part of academic markers would be extreme and bring many and negative side effects.
Is it possible to re-anonymise student submissions after the Post Date?
The short answer is no, even though Moodle’s Turnitin integration has unintentionally allowed this in the past. As such if this capability reappears, it is vulnerable to being removed by Turnitin without notice, and cannot be relied upon. Moodle Assignment prevents re-anonymisation but it allows download of a currently-anonymous submission inbox, which permits anonymous copies to be retained.
Do markers need to be excluded from a Moodle area containing an assignment after that assignment’s Post Date?
No, there is no expectation that student work remains anonymous after marking because, without reference to work and markings, markers are likely to experience more obstacles to discussing the feedback with individual students, and may be hampered in considering students’ development over time. According to the definition of anonymous marking in use here, only markers who will be marking after the Post Date would need to be temporarily excluded from a Moodle area containing an assignment (because students names will have been revealed by the time those markers access the submission inbox). Otherwise markers may remain in their usual roles in those Moodle areas.
Doesn’t reuse of the Student Number compromise anonymous marking? Doesn’t the appearance, after the first Post Date of the first assignment, of the Student Number alongside the student name compromise anonymous marking?
Not according to the definition of anonymous marking in use here – as long as the student name does not appear on any piece of anonymous work still to be marked.
Technical limitations oblige us to balance competing goods of anonymous marking and assessment feedback. A more defensive view of anonymity would seek a technical solution which treated assessors as if they were deliberately intending discrimination, rather than simply subject to unconscious bias. Since there is a grievance process for students who wish to allege intentional discrimination, it is considered sufficient here to take measures to disrupt unconscious bias. There is a recognition that the Student Number may be used on multiple occasions over the year since (because of technicalities of updating each student’s record) its repeated use may be the only feasible means of communicating assessment feedback to students in circumstances of anonymous marking.
Doesn’t Tutor access to the Participation Report compromise anonymous marking?
Not according to the definition of anonymous marking in use here. By default, Moodle course areas allow staff in a Tutor role access to a Participation Report which exists to allow teaching staff to make pastoral interventions, send selective reminders to students, and keep records. A defensive view of anonymity might seek to exclude these since they could allow staff to anecdotally associate student names with late work. It is possible for staff in a Moodle course Administrator role to remove access to these reports from Tutors – however, this is a blunt and potentially resource-intensive measure which would incidentally prevent teaching staff from viewing reports on any other activity, or being able to send out selective reminders. Reports being an important aspect of student feedback to staff, the benefits for anonymity need to be weighed against the educationally undesirable side effects.
Doesn’t Tutor access to the Logs compromise anonymous marking?
Not according to the definition of anonymous marking in use here. By default Moodle course areas allow staff in a Tutor role access to logs, and these incidentally include records of student names and dates and times of submission. As above, it is possible for Administrators to remove access to the logs from Tutors, and as above this is a blunt and potentially resource-intensive measure with far reaching side effect of preventing the Tutor from seeing any logs at all.
Should we set the Post Date a year in advance and amend it when we are ready, to ensure that names are not revealed before all students’ work has been marked?
We recommend that the Post Date is the same as a the date on which students have been told to expect feedback. If anonymous marking needs to happen after that, the original unmarked submissions can be downloaded by an intermediary, re-anonymised and sent to the marker. Students can be apprised that their mark is provisional, and Moodle’s Turnitin Integration allows marks to be changed after the Post Date.