Pink

Designing peer-assessed coursework for students engaging in complex creative tasks


Dr Carlos Matos and Mr Nuno Barreiro, Department of Computer Science

College Excellence Teaching Commendation, 2019

 

The Human-Computer Interaction (HCI) course has been running for four academic years. It covers user interface (UI) design, development and testing. HCI is a third-year course for Computer Science and a second-year course for Digital Media Culture and Technology; it has 40-50 students with very different backgrounds; 60% of the assessment is done with practical work, most of it in teams.

With a view of appropriately preparing students to becoming productive professionals in this growing area, we decided to mimic real-world workplace conditions. We designed a module with focus on creative practical work, also providing the corresponding theorical support that spans from some basic elements of biology and psychology, to interaction design and statistics. To understand the challenges of designing and building an UI, students must undertake the task themselves. However, prototyping is a long and iterative process, requiring both collaborative and introspective stages.

We have simulated the prototyping process by using an iterative assignment model, where students go from teamwork to related individual assignments, converging towards a final product. Considering student diversity and the breadth of taught material, we have designed tasks that encompass a range of activities, suiting different kinds of students and allowing them to work at their full potential. This also fosters engagement, ensuring a high level of participation.

 

Assessment design

The assessment is designed as follows:

  • Group Project I (20% of the final grade) – students build a low-fidelity prototype using paper, scissors and other physical media; they submit a report and a video; they live-demo the prototype in the class. This is a team activity, and an example can be found here:

 

 

  • Individual Report I (10% of the final grade) – students individually evaluate the low-fidelity prototypes of two groups; they submit a report to each group, which is anonymised by a custom Python script and published on Moodle; during the presentations, students have the opportunity to engage with the evaluated groups, to clarify some aspects of the prototype.
  • Group Project II (20% of the final grade) – given the peer feedback, groups redesign the UI, build a detailed Web prototype, and test it; they submit a report and the prototype code, as well as the test results; they demo the prototype in the class and present the tests.
  • Individual Report II (10% of the final grade) – students individually evaluate the prototype during the demos; they also perform the statistical analysis of the tests from two groups; they submit a report with the analysis and suggestions for improvement.
  • Exam (40% of the final grade) – students are evaluated on the material that is not assessed in the prototype, they are also asked about prototype development and test analysis.

 

To accommodate this kind of coursework, we have adopted a flexible delivery structure with two two-hour sessions per week, where we can do any of the following: introduction of concepts (lecture-style delivery); lab sheets with practical work (often a direct application of the new concepts); work on the projects; peer interaction; presentations/demos. This coursework results in meaningful output, with real-world applications, providing important professional skills. Students also develop several transferable skills: presenting, observing, interacting, assessing, reflecting, etc.

 

Our peer feedback cycle has several learning outcomes.

  1. Being confronted with different approaches to the same problem, the student engages in self-reflection about their own solution and its level of detail.
  2. Students evaluate their colleagues, which reinforces self-reflection and provides ideas for the student’s prototype.
  3. The student, in turn evaluated, receives a few reports with feedback, sometimes contradictory, and selects the relevant information. This selection has two important aspects: students learn to accept change with which they may disagree when it is suggested by several different sources; students acquire the ability to discard outliers from the pool of comments.
  4. The student can go back to the drawing board. Being able to act on feedback is highly appreciated, and we ensure that students engage with each group to discuss the received feedback.

 

Evaluation and feedback

The success of this assessment, which has been praised by one of our external examiners and colleagues from other departments, is also measured by the output quality of coursework and student satisfaction. We have consistently observed good results in the exams, friendly competition and a high level of engagement.

Written student feedback includes many good references on teamwork and the practical nature of the course: “Paper prototype, it was fun.” Other students recognise the importance of peer feedback: “The individual reports on other people's assignments helped, as we saw other people's ideas and good/bad aspects.”

 

 

See other case studies on our Assessment and Feedback page