Pink

Developing a feed-forward assessment using the peer marking facility on Turnitin


Dr Alex Dymock, School of Law and Martin King, Educational Development

College Excellence Teaching Commendation 2018

Aims

The aims of this project are varied and reflect the needs of the two departments involved: to enhance, promote and streamline pedagogically sound approaches to assessment; to provide assessment-based opportunities for reflective practice and to provide feed-forward; and to expand the range of credible assessment models at Royal Holloway. 

 

Development & practice

Assessment need not be passive (Dochy et al, 1999) and where students mark work, they are often both accurate and, in so doing, reflect on their own performance – often more than once - with the result of achieving better outcomes in future assignments (Gentle, 1994). Pedagogically, peer-assessment improves student learning (Falchikov & GoldFinch, 2000) through “a sense of ownership and responsibility, motivation, and reflection of the students’ own learning” (Saito & Fujita 2009), and has proven to be an effective example of giving students the opportunity to ‘feed forward’ (Wimhurst and Manning, 2013), improving participants’ conception of quality, and hence improving the quality of summative work. From an academic integrity perspective, peer marking activities can positively impact upon eradicating the possibility of plagiarism (Davis (2004).

The second year Criminology module, Key Perspectives and Debates in Criminology (30 credits) has since 2015 involved a peer marked formative essay. In previous years, the assessment was a stand-alone, peer-marked in hard copy and then moderated by the module convenor. For 2017-18, Alex Dymock revalidated the module to incorporate the peer marking exercise into a two-stage summative assessment to improve student engagement. Davis (2004) found evidence in support of previous claims that awarding a ‘mark for marking’ rewards the demonstration of higher order skills of assessment. The peer marking exercise is now rewarded with an automatic 5% towards the final module mark. Alex also introduced a further summative feed-forward activity. Duncan (2007) notes that some students only read qualitative comments if the quantitative mark is outside their expectations, failing to recognise their potential value. To mitigate this problem, students completed a 300 word reflective paragraph, also with a reward of an automatic 5%, submitted with their summative coursework. The aim of this was to push students to reflect on what steps have been taken to improve the quality of their work based on both peer feedback and feedback from Alex, and how taking part in peer marking has changed their conception of quality.

While efforts have been made in previous years to streamline the peer marking activity and enhance student engagement with it, for 2017-18 Alex worked intensively with Martin King in the E-Learning Team to pilot the peer marking facility in Turnitin. This is the first time across the college that this facility has been used for assessment, so bespoke materials were produced by Martin to give students guidance on how to access essays, how to mark them, and how to submit reviews. We trialled the facility using a dummy assessment, which allowed us to evaluate the tool and develop an appropriate and sustainable workflow. Students were provided with an anonymised sample essay (with permission) from a previous year on the same topic to get a sense of what an outstanding essay might look like. They were also provided with substantive guidance on completing the reflective paragraph, including some FAQs, and guidelines on the benefits of reflective writing.

There is substantial evidence that the use of peer marking technology, and the feed forward activity, improved student engagement and added a reflective component to their learning.

1)     The use of the Turnitin PeerMark facility allowed Alex and Martin to set a number of open or closed questions to guide peer feedback, and set a minimum word count for responses to those questions before feedback can be submitted. Using the School of Law marking rubric, questions were set such as ‘Is there evidence of wide independent reading, beyond lecture content and core texts’? and ‘Does the essay demonstrate a critical understanding of the topic, supported by relevant evidence, and respond fully to the question? How could the writer improve this?’ The use of PeerMark technology ensured that students left substantial feedback and were required to engage directly with the marking rubric.

2)     Essays can be distributed anonymously, and Turnitin allows tutors to automate the assignation of one essay to each student. Students who did not submit formative work (though this only accounted for 1 or 2 students – a great improvement on previous years) were not given the opportunity to peer mark and receive the automatic 5%. That the use of technology to streamline and partially-automate the previously manual, paper-based processes coincides with increased engagement is perhaps explained by Ashenafi (2015), who highlights the challenges that have hampered the advance of peer-marking, which was a concern with this exercise in previous years. Most of these challenges arise from the manual nature of peer assessment practices, which prove intractable as the number of students involved increases.

3)     The use of technology responds to changing student expectations and needs by streamlining the assessment process from submission to providing and subsequently receiving peer feedback, and then marks and feedback from the convenor; with all associated activities consolidated in the Moodle course. That students can use their own device at a location and time of their choosing (within the due dates set by the convenor) is a better fit with current student expectations and lifestyles (Johnson et al, 2016). This contrasts with the first iteration of this peer marking exercise, in which students previously completed the activity in a guided seminar.

4)     Essays can also be marked by the convenor, as usual, using Turnitin Feedback Studio (formerly known as GradeMark). This gave Alex the opportunity to add any correctives as to the peer feedback, assign essays a final mark, and also write additional feedback. Students therefore receive two sets of feedback on their formative work, which they are able to feed forward into their summative work.

5)     The use of the Turnitin PeerMark facility improves the inclusivity of peer marking activities. The anonymity of the technology creates a ‘safe space’ for students to provide peer feedback. The open and closed questions balance prescriptive and autonomous elements to the assessment. Using technology also accommodates students with a broad range of learning needs. For example, students who experience visual impairment are able to more easily participate when essays are not distributed in hard copy. The prescriptive element also assists students who experience social/communication impairment.

6)     The use of PeerMark considerably reduced the volume of administrative work required in ensuring essays and feedback were returned to students, and ensuring the exercise remained entirely anonymous.

7)     Qualitative evidence produced by Wimhurst and Manning (2013) suggests combining peer-marking with a feed-forward activity does enhance student engagement. Our findings are congruent with this. The reflective paragraphs, and other feedback collected from students, confirms that the opportunity to review the work of a peer and reflect on their own learning and skills development, offered students clarity and better understanding of what constitutes essay quality.

8)     The benefits of peer-teaching and assessment to enhance cognitive and social congruence between students have been recognised (Yu et al, 2011), and were the basis for introducing the peer-marking exercise on the module. However, our findings suggest that the prescriptive facilities on PeerMark based on marking rubric further enhanced this, because it requires students to look for the same qualities in work across the cohort. 

 

 PeerMarkFeedback

 

Feedback from students

“Having just submitted our CR2013 Summative, I’d like to pass on my thanks and appreciation for setting the peer mark exercise both for our formative and summative. It really helped me understand explicitly what a marker is looking for in an essay, and I often referred to both my peer’s feedback and your feedback from my last formative when writing the recent summative essay.”

 “By marking someone else’s essay, I was able to extensively use the marking criteria as an examiner would. I was able to look for specific criteria in their work, which ultimately guided me for what I should be including in future essays, such as showing excellent as opposed to good understanding of the topic, and to do this by defining key concepts and providing examples/evaluations.”

”I feel that it has been useful having feedback from a peer as I know that they have been through the same process in writing their assignment and so can therefore use their own experience to feedback on my work. Being able to receive praise from another student is very motivational as I feel that they are on the same level as I am within this degree. The fact that Dr Dymock’s feedback stated similar things to the peer feedback was also interesting as it made me realise that when looking at my work and judging its quality, it is possible for me, as a student, to give a similar perspective on whether it is of a good standard or not, as a lecturer would. This has allowed me to further my ability to properly check over my work and make changes after finishing to ensure I can get the best possible mark.”

 

 

See other case studies on our Assessment and Feedback page