Examining the Effects of Automatic Comment Classification on Comment Types in Peer Review for Graduate Students' Academic Writing

Although previous studies have confirmed the effect of peer review on academic writing, graduate students are usually unable to provide appropriate comments in peer review. As a result, their comments cannot effectively help authors revise academic papers. For this reason, in order to improve the quality of students' comments, this study aims to provide peer reviewers with feedback of the automatic classification of their comments in an existing peer review system, so that reviewers can improve their comments for writers. In order to examine its effect, this study designs an experiment in an academic writing course for first-year graduate students in two semesters. In the experimental group, peer reviewers are provided with the feedback, while students in the control group did not receive any feedback. The results showed that the graduate students in the experimental group consciously improved their comments after receiving the feedback, especially on the cognitive dimensions (e.g., increasing guidance) and the metacognitive dimensions (e.g., increasing evaluating comments). The findings may provide a direction for students to write high-quality comments and improve writing proficiency.