TYPES OF FEEDBACK IN PEER REVIEW AND THE EFFECT ON STUDENT MOTIVATION AND WRITING QUALITY By Christopher Colgan Sloan A DISSERTATION Submitted to Michigan State University in partial fulfillment of the requirements for the degree of Educational Psychology and Educational Technology – Doctor of Philosophy 2017 ABSTRACT TYPES OF FEEDBACK IN PEER REVIEW AND THE EFFECT ON STUDENT MOTIVATION WRITING QUALITY By Christopher Colgan Sloan In writing classrooms, peer review has been shown to have numerous practical and cognitive benefits. However, little research has been done that examines how different types of peer feedback motivate adolescent students and the effects that different types of feedback have on writing quality. While the literature has indicated that peers give each other much more positive feedback than their teachers do, the research has yielded conflicting findings on whether comments about strengths or comments about weaknesses are more helpful. For this study high school seniors in an AP English Language and Composition class were randomly assigned to one of three treatments: a group directed only to provide comments about the strengths in the writing of their peers, a second group directed only to provide comments about weaknesses, and a “business as usual” group who were directed to provide helpful feedback (typically a mix of strength and weaknesses comments). There were no treatment group differences in students’ writing quality and motivation (attributions, belongingness, and sociocognitive conflict regulation). However, there were clear indicators that suggest students saw statistically significant differences in how helpful some types of comments were perceived to be. Implications for the theory and practice of peer-based feedback in writing instruction are discussed. ACKNOWLEDGEMENTS I owe many thanks to Dr. Matthew J. Koehler who was my dissertation director and mentor for much of my work at Michigan State. Dr. Cary Roseth inspired me to continue to understand student motivation in all its manifestations. Dr. Jeff Grabill was a tremendous help due to his work in peer review and his co-development of Eli Review, along with his colleagues Melissa Meeks and Michael McCleod, who were always available with my many questions about the interface. I am also thankful for the Michigan State hybrid Ph.D. program in Educational Psychology and Educational Technology. It takes vision and courage to launch an innovative higher ed program with a completely new delivery system. If it weren’t for this hybrid approach, I would not have been able to pursue a doctoral degree while remaining a classroom teacher. The support of cohort members Andrea Zellner, Dr. April Niemela, Molly Frendo was also much appreciated. Finally, and most importantly, I would like to thank my family for their support. My wife and love of my life was eternally patient with my doctoral pursuits. And I could not have completed all this without the support of my own children, my parents, and my in-laws. iii TABLE OF CONTENTS LIST OF TABLES vi INTRODUCTION 1 LITERATURE REVIEW Peer Review and Writing Instruction The Eli Review Interface Writing Quality and Peer Review The weakness approach to peer review The strength approach to peer review Key gaps in the literature Motivation and Peer Review Attribution theory Belongingness Sociocognitive conflict theory Purpose and Research Questions 4 4 7 8 8 10 12 13 13 14 16 17 METHODS Participants Design Procedure Measures Motivation – Attributions Motivation – Belongingness Motivation – Sociocognitive conflict regulation Writing quality – Instructor scores of drafts and revisions Review quality – Helpfulness Manipulation check Data Analysis Strategies for Validating Findings 19 19 19 20 22 22 22 23 23 23 24 24 25 RESULTS Research question #1 – Impact of feedback type on student motivation 1(a) – Relationship between feedback type and student attributions? 1(b) – How do the different types of feedback affect belongingness? 1(c) – Relationship between types of feedback and sociocognitive conflict regulation? Research question #2 – Perceived quality of the feedback received Research question #3 – Types of feedback and writing quality 27 27 27 28 DISCUSSION Limitations 30 38 iv 28 29 29 Implications 39 APPENDICES APPENDIX A: Argumentative Writing Assignment and Traits APPENDIX B: Motivation survey APPENDIX C: Tables 42 43 45 47 REFERENCES 53 v LIST OF TABLES Table 1: Coding scheme adapted from Cho and MacArthur (2011) 47 Table 2: Mean and standard deviation by condition and measure 48 Table 3: Correlations for measures used in this study (by student) 49 Table 4: Means and standard deviations (by comment) for measures of helpfulness 50 Table 5: Frequency of comment traits received per treatment group 51 Table 6: Tukey post hoc comparison of comment traits 52 vi INTRODUCTION The practice of peer review is one of the most common feedback activities in many disciplines, and studies have shown that students improve their writing through peer review activities (Cho & Cho, 2011; Cho & MacArthur, 2011; Nelson & Schunn, 2009). Peer review is becoming even more prominent, as these activities have been implemented into popular existing online Learning Management Systems like Canvas and Blackboard. Dedicated online peer review applications like Eli Review (Elireview.com) and Peerceptiv (peerceptiv.com) have been created to scaffold and structure the peer review process for students and instructors. Despite the fact that peer review is such a well-established activity and is recommended by standards documents like the Common Core State Standards (CCSS) and the Advanced Placement writing program, there are several gaps in the research supporting its widespread use. First, although studies have examined peer review processes and outcomes from a cognitive perspective (ex. Cho & MacArthur, 2011; Nelson & Schunn, 2009), no studies have looked at peer review from a motivational perspective; this is an important area to examine since decades of research have shown that if students aren’t motivated to learn, negative consequences follow (Ames, 1992; Brophy, 2008; Juvonen, 2006; Weiner, 1985). Second, very few studies to date have examined the effects of different types of peer review on the writing quality of high school students; this is important because writing quality is a key component of what counts as being “college and career ready” in standards documents. Third, this study explores what type of feedback adolescent students find helpful when revising. 1 This randomized experimental control study aimed to address these gaps in the literature in two ways. First, this study examines the differential impact of two types of evaluation comments in peer review – strength comments and weakness comments (Cho & Cho, 2011; Cho & MacArthur, 2011; Nelson & Schunn, 2009). Strength comments identify strengths in the source material, such as detecting strengths, explaining positive feedback, and suggesting further implementation of successful strategies and techniques. Weakness comments identify weaknesses in the writing such as detecting mistakes, explaining problems, and offering solutions to those problems. Studies have shown that students give higher frequencies of strength comments to their peers compared to when teachers give feedback to students (Beason 1993; Cho, Schunn, & Charney, 2006; Patchan, Charney, & Schunn, 2009), and standards documents and trade books on the teaching of writing encourage the use of strength comments. And yet, research on the effect of these different types of comments is inconclusive. Cho and MacArthur (2011) and Nelson and Schunn (2009) found that strength comments in peer review do not positively impact student writing quality when compared with weakness comments. However others (Daiker, 1989; Elbow, 1998; Gee, 1972; Seidman, 1968) found that information about student strengths regarding writing is an effective form of feedback, and approaches such as “praiseworthy” feedback of student writing (Dragga, 1988) have been found to be more helpful than focusing on weaknesses like error identification. Second, this study also examined motivational outcomes of the different types of peer review from three perspectives: attribution theory, belongingness needs, and sociocognitive conflict regulation. The first motivational perspective examined was through the lens of attribution theory. Weiner’s (1985) attribution theory explains how 2 and why students attribute success and failure in academic settings, which may well explain why strength and weakness comments have a differential impact. This study also examined peer review from the belongingness needs perspective (Baumeister and Leary, 1995), which argues that the need to belong is a fundamental human need that extends to students of all ages and classroom settings. Different kinds of feedback may well differentially contribute to students’ sense of belongingness. Finally, this study examined motivation through the lens of the sociocognitive conflict regulation perspective (Buchs, Pulfrey, Gabarrot, & Butera, 2010; Saltarelli & Roseth, 2014) which holds that the way that students regulate conflict causes them to either focus on understanding the task or on social comparison. Different kinds of feedback might induce social comparison processes that decrease writing quality. This study attempted to further writing research by clarifying the role of different types of feedback, and by establishing a link between the types of feedback students find helpful with motivational outcomes and writing quality outcomes. This study also offers potentially important implications for practice. If educators can better understand what motivates student writers to engage in substantive revision and improve writing quality, they will be better equipped to instruct their students and make the most of peer review activities. 3 LITERATURE REVIEW This literature review discusses how audience awareness and feedback from peers became an integral part of writing process classrooms. It is generally agreed that information received from helpful peer reviews can increase writing quality. However what remains unclear is the impact that different types of feedback have on student writing. Additionally, while the research to date has documented numerous cognitive and practical benefits of peer review, studies have largely overlooked (or have generally dismissed) the motivational influences on students in peer review activities. This section explains why a motivational perspective on peer review can expand our understanding of what students find to be helpful feedback when revising. Peer Review and Writing Instruction Feedback is an essential part of instruction and has been found to be one of the most critical influences on student learning. Hattie and Timperly (2007) define feedback as “information provided by an agent regarding aspects of one’s performance or understanding” (81). Feedback provides information specifically relating to the task or process of learning that fills a gap between what is understood and what is aimed to be understood through affective or cognitive processes (Hattie & Timperly, 2007). It can fill this gap through affective processes, such as increased effort, motivation, and engagement. This gap may also be reduced “through a number of different cognitive processes, including restructuring understandings, confirming to students that they are correct or incorrect, indicating that more information is available or needed, pointing to directions students could pursue, and/or indicating alternative strategies to understand particular information” (Hattie & Timperly, 2007, p.82). Hattie (1999) found that 4 feedback had twice the average effect size (.79) as some other influences on achievement, right behind direct instruction (.93) and reciprocal teaching (.86). Hattie and Timperly (2007) found that feedback is more effective: 1. when it is specific, 2. when it provides information about progress or how to proceed, and 3. when it provides information on correct rather than incorrect responses. Hattie and Timperly’s (2007) work focuses on feedback from teachers to students, but another source of feedback, known as peer response or peer review, has been a staple of many writing classrooms for decades. In the 1960s and ‘70s educators like Donald Murray (1972) introduced the process approach to teaching writing, and an integral part of this approach is writing for an authentic audience of one’s peers. In his 1973 book Writing Without Teachers, Peter Elbow argued that a sympathetic reader’s response to a piece of writing was the most effective form of feedback a writer can receive and that the most important peer feedback anyone could give is to tell the writer what was going on inside their head as they were reading, relaying “movies in the mind” (85). Elbow argued that what a writer needed was a sympathetic reader who would “like” their partner’s writing or at least recognize the potential in it, reasoning that if writers do not like a piece of writing they are not as motivated to work on it. Others felt that that student writing improved with peers as “spontaneous apprentices” (Smith, 1983, p. 562) and through creation of a community of knowledgeable peers (Bruffee, 1984). Graham and Perin (2007) found that “collaborative arrangements where students help each other with one or more aspects of their writing had a strong and positive impact on writing quality” (p. 463). 5 Online peer review and traditional face-to-face peer review share similar pedagogical assumptions about writing – it is a process, it is a social act, and it is studentcentered (Breuch, 2004). However there are three noteworthy differences between online peer review and the way peer review was performed before computers –interactivity, textuality, and anonymity. Grabill and Hicks (2005) argue that one of the strengths of online peer review is its increased interactivity whereby writers get feedback, “sometimes immediately, from readers both inside and outside the classroom. Therefore, audiences and writers are related to each other more interactively in time and space” (p. 305). Textuality is another key feature of online peer review. Face-to-face peer review tends to highlight social interactions in terms of oral exchanges with class members whereas online peer review happens most frequently via written exchanges. Studies have found that students participating in online peer review provide more directive comments than in face-to-face settings, and writers were more likely to integrate peer comments into their final revisions because peer review that happens in face-to-face settings is perceived as idea sharing rather than directive (Hewett, 2000; Mabrito, 1991). However, research has shown that students are often reluctant to give directive feedback to their peers because providing critical feedback has been shown to lead to rejection by peers and, additionally, students are afraid of being wrong in front of their peers (Emerson, 1954; Lauderdale, 1976; Levine, 1980; Wiggins, Dill, & Schwartz, 1965). Anonymity has been touted as a feature that might help in this regard. Zhao (1998) found that anonymity allowed participants to be more honest and critical; Cho and Schunn (2007) found that anonymity reduced reviewer bias and led to more helpful feedback. 6 The Eli Review interface Since Eli Review was used for this study, a brief look at the interface will illustrate how it supports process writing pedagogy and also why it was ideal for collecting the data needed for this study. Peer review is becoming even more prominent with the growth of digital writing environments, as these activities have been implemented into popular existing online Learning Management Systems like Canvas and Blackboard. Dedicated online peer review applications like Eli Review (Elireview.com) and Peerceptiv (peerceptiv.com) have been created to scaffold and structure the peer review process for students and instructors. Eli Review incorporates many of the features of process writing classrooms and collaborative learning in general, for example, reducing the cognitive load of complex writing tasks (Kellogg & Whiteford, 2009). From a practical standpoint an affordance of an online peer review system is that it provides teachers additional student engagement data on writing quality and perceptions of review quality not readily available in face-toface settings. For example ratings of helpfulness are an important indicator of the quality of the feedback. In Eli Review, after writers receive the reviews of their peers, they rate the helpfulness of that feedback. Atwater, Waldman, Atwater, and Cartier (2000) found that feedback was less effective for those who had a negative perception of it. Cho, Schunn, and Charney (2006) found that praise feedback positively influenced the perceived helpfulness of the peer review, and critical feedback negatively influenced perceived helpfulness. 7 Writing Quality and Peer Review Research has shown that feedback is helpful to students when revising their writing. Nelson and Schunn (2009) conducted a correlational study that focused on how peer review benefited undergraduates in a survey of history course and identified several feedback features that led to increased writing quality: providing solutions, giving a summary of the performance, and identifying the location of the problem. These results illustrate some of what is already known about effective feedback – that it is timely and provides specific information about how to proceed in order to improve (Hattie & Timperly, 2007; Nelson and Schunn, 2009; Hart-Davidson, McLeod, Klerkx, and Wojcik, 2010). Although peer review is commonly practiced in education settings, there is no general agreement regarding what type of feedback is most helpful when they revise. Cho and MacArthur (2011) studied the effects of two specific types of peer feedback on the writing of undergraduate students: strengths and weaknesses (see Table 1). This distinction guided the framing of the design of this study. Accordingly, each approach (strength and weakness) is described in Table 1. The weakness approach to peer review. Braaksma, Rijlaarsdam, Van den Bergh, and van Hout-Wolters (2004) and Siegler (2002) found that students learn by explaining weaknesses in their writing. Commenting on the weaknesses of peer drafts may prompt reviewers to reflect on their less effective writing strategies in their own writing and to develop knowledge of what to avoid on subsequent writing. The research findings of Gick and McGarry (1992) indicate that if students encounter writing problems they have already addressed in peer drafts, they may be able to effectively solve those problems in their own writing. 8 Cho and Cho (2011) employed a quasi-experimental design to study the writing of 72 undergraduates in an introductory physics course, exploring the hypothesis that giving comments improves the development of the reviewer’s own writing skills. They analyzed comments in two dimensions: evaluation (strength vs. weakness) and scope (surface level, or meaning levels – i.e. micro-meaning level, and macro-meaning level). Cho and Cho found that when student reviewers commented on weaknesses at the micro-meaning level (or paragraph level), the revision qualities of their own drafts improved. However, providing weakness comments at the macro-meaning level and on surface features did not improve the reviewer’s own writing quality. Cho and MacArthur (2011) found that students learn to write better through the act of reviewing the writing of peers. They examined three types of feedback in this randomized experimental control study: strength, weakness, and off-task comments. There were three types of weakness comments categorized: problem detection, defined as a “statement about what is wrong or weak in peer writing,” problem diagnosis, defined as “explanation on why the problem happened” and solution suggestion, “statement about how to improve the problem.” They found that weakness comments coded as problem detection and solution suggestion positively correlated with writing quality. Nelson and Schunn (2009) examined the relationship between various types of feedback, potential internal mediators, and the likelihood of implementing feedback. They looked specifically at three types of feedback: praise, problem/solution, and summary. They examined understanding of feedback and agreement with feedback as potential internal mediators of actually implementing the feedback and found that understanding feedback was the only significant mediator of implementation. 9 Furthermore they found that problem understanding was especially important when implementing feedback, and it led to improved writing quality. The strength approach to peer review. While some research has indicated that weakness comments are a more helpful form of peer feedback, other research has found that feedback on the strengths in student writing has a positive effect (Braaksma, et al., 2004; Cho, Schunn, & Charney, 2006; Seidman, 1968; Elbow, 1998; Gee, 1972) and “praiseworthy” comments improve writing quality (Dragga, 1988). Chi and Bassok (1989) found that students increase their knowledge when they explain correct problemsolving strategies, therefore, strength comments may improve understanding of what makes good writing. Additionally trade books on the teaching of writing (Strong, 2012) and standards documents like the AP program and the Common Core State Standards also encourage the use of strength comments in peer review. Studies of peer review examining strength and weaknesses have yielded conflicting results. In Cho and Cho’s study (2011) if a reviewer described or explained what was good about surface aspects of their peer’s draft or at the meaning level (i.e. the paragraph level or the essay as a whole), then the idea unit was coded as a strength comment. Cho and Cho found that when writers received more praise from peer reviewers on surface features, the revised writing resulted in a lower quality. However when reviewers made strength comments at the macro-meaning level, the revision qualities of their own drafts improved. Strength comments at the micro-meaning level had no significant impact on later writing quality. The only kind of strength comment that Cho and MacArthur (2011) examined was praise, defined as “good remarks on what constituted the strength.” They found that the number of strength/praise comments were 10 unrelated to later writing quality. Nelson and Schunn (2009) defined praise as “complimentary comment or identifying a positive feature in the paper.” They also found that praise was unrelated to improvement in writing quality. Cho, Schunn, and Charney (2006) found that undergraduate students value praise comments more than graduate students. They examined the difference between the types of comments on student writing from peers compared to experts, looking at the perceived helpfulness of comments from those different sources. They found that experts do not give much praise, however peers gave much more praise comments to each other. They also found that the undergraduates in this study considered praise comments to be helpful, and critical comments negatively influenced the perceived helpfulness of the review. While not specifically looking at peer review, previous research with regard to teacher praise of student writing is instructive. In a quasi-experimental design, Dragga (1988) found that student writing improved in classrooms whose instructors used “praiseworthy” grading, a method of evaluating students’ writing in which the instructor comments only on the praiseworthy characteristics of a given essay. By explaining their praise comments, writing instructors in this study had to struggle with putting into words their implicit and intuitive appreciation of the students’ writing. According to Dragga such comments required more thought and more time than simple weakness detection. Others have found that strength comments have benefits as well. Gee (1972) found that high school students who received more praise turned in longer final drafts. Seidman (1968) found that high school students who consistently received positive feedback also experienced improved writing quality when compared to those who received negative feedback. 11 Considering the emphasis of this study, it is worth noting that after reviewing the thousands of studies on student achievement Hattie and Timperly (2007) issued this caveat about a wholesale dismissal of praise as a form of feedback: “It is important to distinguish between praise that directs attention away from the task to the self (because such praise has low information value to achievement and learning) and praise directed to the effort, self-regulation, engagement, or processes relating to the task and its performance (e.g., ‘You’re really great because you have diligently completed this task by applying this concept’). This latter type of praise can assist in enhancing self-efficacy and thus can be converted by students back into impact on the task, and hence the effects are much greater. It seems likely from reading these meta-analyses, however, that reviewers do not always distinguish between praise as a reinforcer or reward (for which it has zero to limited effect on achievement) and praise accompanied by information about the processes or performance, which has more, but still limited, effect” (96-97). Some studies have found that strength comments are not related to writing quality (Cho & Cho, 2011; Cho & MacArthur, 2011). Yet others found that student writers have found strength comments to be more helpful than weakness comments (Cho, Schunn, & Charney, 2006). Hattie and Timperly (2007) argued that praise accompanied with information may have a limited effect on achievement. Dragga (1988) argued that praiseworthy comments are more effective than comments on the weaknesses. Key gaps in the literature. Peer review is one of the most common activities in many high school classrooms yet there are several gaps in the research regarding the effects that different types of feedback might have on writing quality. First of all, the vast majority of the research on peer review has been situated in undergraduate and graduate classrooms. Second, I know of only one other randomized experimental control study that has been conducted with regard to writing quality and peer review (Cho & MacArthur, 2011). Third, previous studies have presented a promising distinction that differentiates 12 between strength and weaknesses comments (Cho & Cho, 2011; Cho & MacArthur, 2011). These straightforward characterizations, however, do not represent the nuance of many of the strength comments that students actually provide to each other. After surveying the empirical evidence regarding strength and weakness comments as a whole, questions remain. For example, what is still unresolved is whether strength comments from peers (accompanied by information) might be considered to be as effective a form of feedback than weakness comments. Psychologically, it also remains unclear why these differential effects occur. In the next section the possibility that different types of peer review comments may impact student motivation is discussed. Motivation and Peer Review While much of the research on peer review has examined cognitive processes, the motivational aspects of peer review have been largely overlooked. In terms of motivation, this study specifically examined peer review through the perspectives of belongingness needs, attribution theory, and sociocognitive conflict regulation. Attribution theory. Weiner (1974, 1985) developed a theoretical framework that explains how and why students attribute success and failure in academic settings. What students attribute as the causes of their successes or failures in school can be characterized in terms of three dimensions: locus, stability, controllability. The locus of an attribution is the source of the success or failure. For example in peer review if students receive only weakness comments on their writing and they might attribute this to poor strategy, which has an internal locus. If on the other hand weakness comments are attributed to task difficulty, then this locus is said to be external. The stability of an attribution is its relative permanence. If for example students attribute the cause of 13 weakness comments on their writing to be low effort, then this is said to be unstable; if on the other hand the cause is attributed to ability, it is stable. The controllability of an attribution is the extent to which the individual can influence it. If students attribute strength comments to the amount of effort put into that assignment, then the source of success is said to be controllable. But if students attribute comments to the “luck of the draw” as to whom the teacher assigned to be their peer reviewer, then the source is uncontrollable. Weiner (1974) identified ability, effort, task difficulty, and luck as the most important factors affecting attributions for achievement. Perry, Stupinsky, Daniels, and Haynes (2008), examined the multiple causes of attributional thinking about failure of Canadian college students transitioning from high school. Perry, et al. found that even when multiple reasons for failure were cited by five different cohorts spanning a thirteenyear period, students identified the causes of failure with remarkable consistency. All cohorts identified low effort as the most important attribution for failure, followed by test difficulty, poor strategy, professor quality, natural ability, and bad luck. “This ordering of causal attributions has positive implications for motivation because the three most important attributions imply that poor performance is controllable (low effort, poor strategy) and is external to the student (task difficulty)” (469). While attribution theory may explain some aspects of student motivation in peer review, it does not consider social concerns, which are very important to high school students. Belongingness needs and sociocognitive conflict theory might help further explain other aspects of motivation in peer review activities. Belongingness. Baumeister and Leary (1995) argue that human beings are 14 fundamentally and pervasively motivated by a need to belong and found multiple links between the need to belong and cognitive processes, health and well-being. Connell and Wellborn (1991) claimed that relatedness, or the need to experience oneself as worthy and capable of love and respect, is a basic human need. In a construct similar to relatedness, Juvonen (2006) argued that the need for belongingness affects student behaviors in ways that in turn influence their ability to form and maintain relationships in school, and that motivation to learn increases if students feel that they are valued by their learning community. Support from friends and peers have been found to promote higher levels of motivation, involvement in the classroom, and academic achievement (Van Ryzin, Gravely & Roseth, 2009). Baumeister, Campbell, Krueger, and Vohs (2003) found that students need to develop significant and positive relationships with their teachers and peers; if these needs aren’t met, students will experience a decrease in motivation for learning and other maladaptive behaviors. Correlational studies have found that children who are rejected by their peers care less about the welfare of others, or in other words, they act less prosocially than do their peers (Asher & Coie, 1990; Gest, Graham-Bermann, & Hartup, 2001; Wentzel & McNamara, 1999). It can be argued that weakness comments might lead to more asocial feelings and a decreased sense of belongingness. Other studies have found that prosocial actions are highly correlated with social acceptance (Parkhurst & Asher, 1992; SchonertReichl, 1999). Studies of peer review suggest that strength comments might be more positively received by writers and therefore might potentially lead to an increased sense of belongingness (Atwater, Waldman, Atwater, and Cartier, 2000; Cho, Schunn, & Charney, 2006). 15 Sociocognitive conflict theory. According to Piaget (1952) cognitive development progresses through stages, and conflicts are essential for growth. Conflicts represent contradictions between what the child expects and the observed results, and they can lead children to higher levels of development. Subsequent research (Darnon, Butera, Harackiewicz, 2007; Doise, Mugny, & Pérez, 1998; Mugny, DePaolis, & Carugati, 1984) has shown that when an individual’s response diverges from that of a partner, progress can result. Since this conflict is both social (i.e. disagreement between peers) and cognitive (i.e. each partner doubts his/her own response), Mugny, et al. (1984) labeled this “socio-cognitive conflict” and found it beneficial in many settings. However, sociocognitive conflict does not always lead to progress; how students regulate this conflict that has important implications. Conflict regulation may either be focused on the task and understanding of the problem (epistemic conflict regulation) or focused on social comparison or on demonstrating self-competence (relational conflict regulation). Whether students engage in epistemic or relational conflict regulation depends on whether they perceive the task or partner as threatening to their sense of competence. Doise and Mugny (1984) found that learning progress resulted from epistemic conflict regulation but that no progress resulted from relational conflict regulation. Of particular interest for this study is how students regulate sociocognitive conflict when they are confronted by someone who disagrees with them or has an opposing point of view (i.e. the weakness condition). On the one hand, this conflict resolution could be relational. Research has shown that conflict enhances uncertainty by making students doubt their knowledge and competence (Darnon, Butera, and Harackiewicz, 2007; Pérez & Mugny, 1996; Butera & Mugny, 1995; McGarty, Turner, 16 Oakes, & Haslam, 1993). Festinger (1954) found that when uncertainty in learning situations is high, people feel the need to compare themselves to others. Attention to social comparison and competence differential between peers focuses attention to selfworth and not on the task (Mugny et al., 1984). On the other hand, the conflict resolution that may arise in the weakness condition might be epistemic. Tjosvold and Johnson (1977) found that partners who were confronted with an opposing position were able to identify their partner’s reasoning more precisely. Johnson and Johnson (1995) found that concurrence-seeking, where confrontations and conflicts are avoided, is not as positive for cognitive outcomes as is controversy when different positions are explored. Purpose and Research Questions The purpose of this randomized experimental control study was threefold. First, this study examined how strength or weakness comments (or combinations of the two) affect students’ motivation, defined in this study in terms of attributions, sense of belonging, and sociocognitive conflict regulation. Second, this study explored what students perceive to be the most helpful type of feedback. Third, this study looked at how different types of feedback affect writing quality. Participants were randomly assigned to be in one of three groups providing anonymous peer review via Eli Review. The first group focused on identifying strengths in student writing, such as praise accompanied with information about the task, explaining positive feedback, and suggesting further applications of successful elements. The second group focused on identifying weaknesses in the writing such as correcting mistakes, labeling and explaining problems, and offering solutions. The third group (a 17 control group) were instructed to provide feedback that would be considered helpful to the writer. Using data collected from Eli Review, a student survey, and instructor evaluation of student writing, the following questions were explored: 1. How do strength comments and weakness comments in peer review impact motivation? a. How do the different types of feedback affect the attributions students make about why they received the kind of feedback they did? b. How do the different types of feedback affect belongingness? c. From a sociocognitive conflict regulation perspective, does the receiving of different types of peer feedback induce performance goals? 2. How do the different types of feedback impact students’ perceived helpfulness of the comments they receive? 3. How do the different types of feedback impact student writing quality? 18 METHODS This study explored the impact of different types of feedback on student motivation to write and on the quality of writing in a 12th grade Advanced Placement context. Participants Eighty-two 12th-grade students were enrolled in four face-to-face sections of AP English Language and Composition class in a primarily middle-class, college-bound student population. There were 29 males and 53 females in a predominately Caucasian student population. Although the institution is a private school, nearly two-thirds of the students are on some kind of financial assistance, with 15% of the students on free and reduced lunch. Seven percent of the participants were non-native English speakers. Sections sizes ranged from 19-25 students. The author taught three of the face-to-face sections and another teacher with seven years of AP English Language teaching experience taught the other face-to-face section of the class. Students were given the opportunity to opt out of the study when the initial permission forms were distributed, but none elected to do so. Design The study followed a randomized experimental design with three different peer feedback writing conditions. All three sections composed the same argumentative writing assignment (Appendix A), and completed the same measures (Appendix B). Students in each condition differed on the type of the format and type of feedback offered to fellow students within the group. Students were randomly assigned to one of the following three conditions. No student was in the same online response group with 19 someone in their face-to-face section. Response groups were comprised of three or four students. Feedback was conducted anonymously in Eli Review; student writers did not know who gave them feedback, and reviewers did not know who rated their feedback. The three randomly assigned groups were as follows: • A control group – Students in this group were instructed to provide whatever feedback that they think the writer would consider most helpful • A strength group – Students were taught (and instructed) to provide strength comments in their feedback to writers • A weakness group – Students were taught (and instructed) to provide weakness comments in their feedback to writers In this design, motivational and writing quality outcomes were analyzed according to group differences (i.e., the type and format of feedback given). Procedure This study took place over the course of four weeks in the spring of the 2015-16 academic year. Students wrote an argumentative research essay on a topic of their choosing (Appendix A). Before the study began students first completed an in-class training activity where they were instructed to only provide comments about the weaknesses of a sample AP essay. Students provided comments to a partner, and examples of weakness comments were written on the board in the classroom. On a subsequent in-class training activity students were given a different AP sample essay and were instructed to provide only comments about the strengths of the drafts. Again students were instructed only to provide comments about the strengths of the essay to a partner. Strength comments were written on the board for all to see during the discussion 20 of the strength comments. A practice peer review activity then took place in Eli Review where the instructors emailed the directions per treatment group to all students the night before the activity was to be conducted in class. Those same instructions were posted again in Eli Review as part of the directions for the classroom activity as students provided peer feedback. On day 1 of the study they uploaded drafts of their research essay to Eli Review. Students then provided peer feedback on drafts (day 2); rated the feedback they received on their drafts (day 3), revised drafts and resubmitted to Eli Review (day 4). Before the final peer review activity in Eli Review students were emailed the instructions the night before the activity, and those same instructions were embedded in the Eli Review interface. As an attempt to reinforce the conditions for the study, students were provided with those same instructions on paper in class before they began the final online review activity, according the condition they were assigned. They were also instructed not to talk about the assignment with classmates as they worked on the online reviews during class time. Next students rated the feedback they received on their revisions (day 5), and revised and resubmitted the final draft of the essay to Eli Review (day 6). Collection of data for this study took place in three ways: SurveyMonkey, Eli Review, and instructor ratings of the initial draft and revision (Appendix D). SurveyMonkey was used to gather student self-reported motivation levels for attributions, belongingness, and conflict regulation. Eli Review was used to gather engagement and achievement data for both reviewers and writers. During the four-week period of this study, the students also participated in classroom activities pertinent to the course, like AP test preparation. 21 Measures The motivation survey used a 7-point Likert scale to measure student attributions, belongingness, and sociocognitive conflict regulation as a result of the type of peer review feedback they received. Eli Review was used to collect student engagement and achievement data (e.g. helpfulness ratings). Instructor ratings were used to determine writing quality. The motivation survey can be found in Appendix B. Questions 1-7 of the survey (attributions) are adapted from Perry et al. (2008). Questions 8-15 (belongingness) are adapted from Deci and Ryan (1985). Questions 1621(sociocognitive conflict regulation) are adapted from Saltarelli and Roseth (2014), and Darnon, Muller, Schrager, Pannuzzo, and Butera (2006). Motivation - Attributions. Items in the first section of the motivation survey (Appendix B) regarding attributions are adapted from Perry et al. (2008). In the Perry study, college students were asked the question, “when you perform poorly in your Introduction to Psychology course, to what extent do the following factors contribute to your performance (ability, assignment difficulty, effort, luck, strategy, and teacher quality).” In this study, the data for causal attributions will come from student responses to the following item: “indicate the extent to which the following factors contributed to your performance on this assignment (ability, assignment difficulty, effort, luck, strategy, teacher quality, feedback from peer reviewers, and structure of the peer review activity).” The attribution subscale consisted of eight items (α=0.70). Motivation - Belongingness. Items in the second section of the motivation survey regarding belongingness are adapted from Deci and Ryan’s IMI measure of relatedness. These items ask students specifically for their thoughts about the comments they received 22 with regard to belongingness issues. For example, an item from the IMI read, “I felt like I could really trust this person”; for this study, this item was revised to read, “I felt like I could really trust the people who reviewed my letter.” The IMI also contains reversescaled items that are phrased in the semantically opposite direction; these items were denoted with an “(R)” – for example, item #5, “I felt really distant from the people who reviewed my letter.” As per the instructions for scoring that accompany the IMI, items 18 in the Comments survey were averaged to find the belongingness measure (α=0.65). Motivation - Sociocognitive conflict regulation. Items in the third section of the motivation survey (sociocognitive conflict regulation) are adapted from Saltarelli and Roseth (2014), and Darnon, Muller, Schrager, Pannuzzo, and Butera (2006). Saltarelli and Roseth asked subjects about their conflict regulation with regard to an acceptance, mild rejection and control condition; Darnon, et al. asked subjects about their conflict regulation when disagreements occurred. Questions 18-20 pertain to relational conflict regulation. Questions 21-23 check for epistemic regulation. The sociocognitive conflict regulation subscale consisted of six items (α=0.75). Writing quality - Instructor scores of draft and revision. Students uploaded their essays in Eli Review without their names on them and then assigned a random number in the Eli Review interface. The writing quality of de-identified essays was scored using a 7-point scale (see Appendix A). The instructor ratings subscale consisted of two items (α=0.77). Review quality - Helpfulness. These are scores assigned by an individual student writer to rate the helpfulness of the feedback they received from a peer on their writing. In Eli Review this is scored on a 1-5 scale, with five stars being the most helpful 23 feedback, one star being the least helpful. The helpfulness ratings were normally distributed (M=3.42, SD=0.89, p > 0.05). Manipulation check. Before the peer review activity took place on the revised research essay, a random sample of 100 comments were coded for an introductory assignment in Eli Review. This resulted in 77% weakness comments in the weakness condition, and 71% of strength comments in the strength condition. For the final revision 86% of the comments students in the weakness condition were coded as weakness comments and 81% of the comments provided by students in the strength condition were coded as strength comments. Data Analysis To answer research question #1 a one-way analysis of variance (ANOVA) was performed. Questions 1-7 were used to determine levels of attribution. As per the instructions for scoring that accompany Deci and Ryan’s Intrinsic Motivation Inventory, questions 8-15 were averaged to find the belongingness measure. Questions 16-18 were averaged to determine a relational conflict regulation measure. Questions 19-21 were averaged to determine the epistemic conflict regulation measure. These averages will be checked to see if the differences are significant among the groups. Additionally, because students did not receive only strength comments in the strength condition, or weakness comments in the weakness condition, a measure of the proportion of strength comments was computed for each participant using a Pearson correlation. To answer research question #2 a one-way analysis of variance (ANOVA) was performed to determine whether the means for helpfulness of the reviews are significant between the control group, the weakness group, and the strength group. A measure of the 24 proportion of strength comments was computed for each participant using a Pearson correlation. Additionally, an ANOVA was performed at the comment level using the coding scheme of MacArthur and Cho (2011). To answer research question #3 – how the different types of feedback impact student writing quality – an analysis of covariance (ANCOVA) was performed to determine whether the means for the quality of the instructor’s scores on the drafts and revisions were significantly different between the groups. Strategies for Validating Findings This study employed two strategies for strengthening the internal validity of quantitative findings. First, randomization and anonymity of students took place at the individual level. Students from four face-to-face sections were randomly distributed into three online sections, comprised of an equal number of participants from each of the four face-to-face sections. Additionally the identity of those students were not known to one another because students names were hidden from others in the Eli Review interface; instead, students were assigned randomly generated numbers as identifiers. Therefore, reviewers did not know the identity of the writers, and writers did not know the identity of their reviewers. The second strategy took place through a norming session with the author and a colleague with seven years experience teaching the same AP English Language and Composition. Then reliability tests for both the writing traits and scores on the essay drafts were conducted for a sample set of essays and comments. A Cohen’s Kappa of 0.71 was recorded for the instructors’ scores for nine randomly chosen essays. Thirty-one randomly chosen comments were also tested for reliability, and the Kappas ranged from 25 0.73-1.0, which are acceptable levels, according to Landis and Koch (1977) and Fleiss, Levin, and Paik (2013). After these levels were attained the author scored the remaining essays and tabulated the writing traits. 26 RESULTS The means and standard deviations (by student) for the measures used in this study are presented in Table 2. Correlations between all the measures used for this study are presented in Table 3. Research question #1 – Impact of feedback type on students’ motivation 1(a) –Relationship between feedback type and student attributions? Students rated eight attributions about why they received the feedback that they received. The means and standard deviations for each of these attributions are presented in Table 2, along with the results of the ANOVA testing for group differences on each of these eight attributions. There were no significant differences in the attributions that students made according the treatment they received (strength comments, weakness comments, or control group). The ANOVA results and effect size for each measure are as follows: ability F(2, 79) = 0.21, n.s., η2 = 0.01; assignment difficulty F(2, 79) = 1.02, n.s., η2 = 0.03; effort F(2, 79) = 1.30, n.s., η2 = 0.03; feedback from peer reviewers F(2, 79) = 1.61, n.s., η2 = 0.03; luck F(2, 79) = 1.36, n.s., η2 = 0.03; strategy F(2, 79) = 2.90, n.s., η2 = 0.07; teacher quality F(2, 79) = 0.77, n.s., η2 = 0.01; structure of peer review activity feedback from peer reviewers F(2, 79) = 0.01, n.s., η2 = 0.01. Because students did not receive only strength comments in the strength condition or weakness comments in the weakness condition, a measure of the proportion of strength comments was computed for each participant. Thus students who received all strength comments (praise, strength detection, strength explanation, strength suggestion) would have a proportion of strength comments of 1.0, whereas students who received only weakness comments (censure, weakness detection, weakness explanation, weakness 27 suggestion) would have a ratio of 0.0. Most students fell somewhere in between these two extremes (e.g., 0.90, 0.83, 0.11). The proportion of strength comments was used as a measure of the treatment students actually received. When differences in attribution were explored using the proportion of strength comments, there was a positive correlation between one of the attributions (strategy) and strength comments (r = 0.22, n = 82, p < .05). All other attributions were not correlated with feedback type. In sum, there was no evidence of significant differences between conditions on the attributions that students gave when asked about the factors that contributed to their performance on this writing assignment. A relationship between the strength type of feedback and strategy attribution was found. 1(b) – How do the different types of feedback affect belongingness? Students rated their sense of belongingness, or their need to feel connected with others. The means and standard deviations for each of these attributions are presented on Table 2, along with the results of the ANOVA testing for group differences on belongingness. There were no significant differences between the groups sense of belongingness, F(2, 79) = 0.67, n.s., η2 = 0.02. The proportion of strength comments was used as a measure of the treatment students actually received. When differences were explored using the proportion of strength comments, there was no relationship between belongingness and treatment group (r = 0.03, n = 82, n.s.). 1(c) – Relationship between types of peer feedback and socio-cognitive conflict regulation? Students next rated how they regulated conflict. The means and standard deviations for each of these attributions are presented on Table 2. There were no significant differences between the groups with regard to whether they focused on social 28 comparison, F(2, 79) = 0.39, n.s., η2 = 0.01 or on understanding the task, F(2, 79) = 0.56, n.s., η2 = 0.01. The proportion of strength comments was used as a measure of the treatment students actually received. When differences were explored using the proportion of strength comments, there was no relationship between either social comparison (r = 0.01, n = 82, n.s.) or understanding the task (r = 0.15, n = 82, n.s.). Research question #2 – Perceived quality of the feedback received After students submitted their writing for review by their peers, they rated the helpfulness of each comment they received on their writing. This rating of helpfulness is used to measure students’ perceived quality of the reviews they received. The means and standard deviations for the average ratings of helpfulness of feedback are presented in Table 2. There were no significant differences between the groups, F(2, 79) = 1.35, n.s., η2 = 0.03. Again the proportion of strength comments was used as a measure of the treatment students actually received. When differences in helpfulness were explored using the proportion of strength comments, there was a non-significant correlation between the type of feedback students received in their groups and helpfulness (r = 0.41, n = 82, p=n.s.). Research question #3 – Types of feedback and writing quality Writing scores assessed by the instructor were used to determine how the different types of feedback provided in the groups impacted student writing quality. An ANCOVA was run using the instructor score on the revised draft and the instructor score on the draft as a co-variate. No significant differences were found (F(2, 79) = 0.26, n.s.) between the writing quality of the three groups based on the feedback they received. 29 DISCUSSION The purpose of this study was to examine how peer feedback on the strengths and weaknesses in high school writers impacted student motivation and writing quality. This study found, unexpectedly, that there were no treatment group differences in students’ writing quality and motivation. In other words, strength comments from peers (accompanied by information) were considered no more or less helpful a form of feedback when compared to weakness comments, both from a motivational perspective and in terms of writing quality. One reason for these unexpected outcomes might be that there was contamination in this study. As Table 5 shows, students in all conditions received a combination of strength and weakness comments. I therefore coded each of the 461 individual comments provided to students on the final revision according to one or more of eight characteristics, or traits (see Table 5). Students in the control group provided 63% weakness comments and 37% strength comments. In the weakness condition 86% of the comments students provided were coded as weakness comments, while 14% were coded as strength comments; 81% of the comments provided by students in the strength condition were coded as strength comments, while 19% were coded as weakness comments. If this contamination had not taken place, perhaps this study may have found that the type of comments students received had an impact on student motivation and writing quality. Future research should consider using “confederates” posing as students in the class in the order test the pure effects of strength and weakness comments. Even though there was contamination, however, most of the students did provide the type of feedback they were instructed to, and all students received the majority of the 30 types of feedback intended for their group. Because students did not receive only strength comments in the strength condition or weakness comments in the weakness condition, a measure of the proportion of strength comments was computed for each participant, and the results also showed no significant differences between writing quality and motivation with the type of feedback received. The means of the study are generally in line with previous research on attributions. For example as in this study Weiner (1974) also found that effort and ability were two of the highest factors affecting attributions for achievement. Perry, Stupinsky, Daniels, and Haynes (2008) found that students rated effort and strategy as two of the most important attributions when explaining their success and failures in school. One specific finding of this study is that strength comments did not impact student writing quality. This finding contradicts some prior research but is consistent with others. For example Dragga (1988) found that “praiseworthy” comments improved writing quality. According to Dragga praiseworthy comments required more thought and more time than simple weakness detection. On the other hand, Cho and MacArthur (2011) found that strength comments, compared to weakness comments, did not lead to improved writing quality. However, upon closer examination these findings may be due to the fact that these studies conceptualized strength comments in very different ways. Dragga argued that praiseworthy comments were more helpful for writers because it forced reviewers into more explanations compared to “simple weakness detection.” Cho and MacArthur (2011) grouped all strength comments into one category, whether it was praise unrelated to the task or elaboration of strengths; as will be discussed in more detail 31 later in this section, students in this study found the latter to be significantly more helpful than the former. This study’s finding that weakness comments did not impact writing quality is also inconsistent with most of the prior research. For example, Cho and Cho (2011) found that when student reviewers commented on weaknesses, especially at the paragraph level, the revision qualities of their own drafts improved. Nelson and Schunn (2009) found that when student writers understood the problems in their drafts, writing quality improved. Furthermore a number of studies have found that students learn effectively by explaining weaknesses in their work (e.g. Braaksma, et al. 2004; Gick & McGarry, 1992; Siegler, 2002). Another explanation for why the different types of peer review in this study did not impact student writing quality may be due to the study’s focus on the ratings of feedback rather than the actual feedback that was used by writers when revising. Supporting this idea, Sommers (1980) found that novice writers understand the revision process merely to be rewording what they already have written, while expert writers make changes at the structural level and make sentence level changes to support those structural changes. Within ELI Review, how students implement feedback in their revision plans may be more instructive than analyzing helpfulness ratings because “revision determines writing quality, not precisely feedback.” (J. Grabill, personal communication, April 10, 2017). Future studies might focus on the feedback that results in structural changes made to students’ revision plans in Eli Review and that subsequently appear in their revised writing. 32 Another unexpected finding of this study is that that strength comments (as well as the proportion of strength comments) did not impact motivational measures. This null finding is contrary to the idea that motivation to learn increases if students feel that they are valued by their learning community (Juvonen, 2006). For this study it was hypothesized that a higher proportion of strength comments would be equated with an increased sense of belongingness, while feedback consisting only of weakness comments would lead to feelings of rejection, which others have found to lead students to act less prosocially than peers who don’t feel rejected (e.g. Gest, Graham-Bermann, & Hartup, 2001; Wentzel & McNamara, 1999). Since Weiner (1974) identified ability as one of the most important factors affecting attributions for achievement, it was also hypothesized that weakness comments would be equated with lower levels of self-perceptions of ability, which Weiner argued have detrimental effects on achievement. With regard to conflict regulation it was hypothesized that a higher proportion of weakness comments would focus student attention on self-worth and not focus on the task and understanding the problem (Johnson & Johnson, 1995; Mugny, et al., 1984; Tjosvold & Johnson, 1977). There are a few reasons why motivation may not have not differed by strength and weakness comments. For example, many previous studies have measured levels of belongingness and attributions at transitional times in schooling, from elementary to middle or junior high school, from junior high to high school, or from high school to college (e.g. Juvonen & Cadigan, 2002; Perry, et al., 2008). The vast majority of participants in this study had been at the same school for four years, therefore the treatments most likely would have had less of an effect than on students who are new to a learning environment and therefore more sensitive to the judgments of their peers. Future 33 studies might examine the effects on belongingness and attribution of peer review in transitional periods of schooling (like ninth graders in high school or first-year college students). With regard to sociocognitive conflict regulation Buchs, et al. (2004) found that threats to student competence led to decreased performance on academic tasks, and cognitive gains are disrupted when partners simply impose their will without explanation (i.e. directive comments in peer review where writers are simply advised to fix problems). The fact that the majority of feedback in this study provided by all groups did not involve solutions, explanations, and suggestions might also explain why weakness comments did not cause relational conflict regulation, or why strength comments did not lead to epistemic conflict regulation. Future research could test this idea by comparing conditions in which one group of students elaborates on weakness comments, provides explanations, solutions and suggestions against another group in which students provide only directive comments on what to fix in a draft. Another finding of this study indicates that the adolescents seem to have firmly held notions that peer review consists of a combination of strength and weakness comments. This is consistent with the findings of Cho, Schunn, and Charney (2006) who determined that students have a “clear comment-giving script that includes giving praise feedback” (p. 276). Consequently it was difficult for participants to abandon these notions for the purposes of this study. For example sometimes students in the weakness condition apologized for their comments and often softened their criticism with praise (e.g. “I'm sorry if this seemed harsh, [instructor’s name] wanted me to point out weaknesses only in my feedback, so I didn't say anything about all the things you did 34 right, though you did plenty. Keep up the good work!). Upon reflection comments like these might simply be illustrating Juvonen’s finding (2006) that adolescents adapt to the norms of group they wish to affiliate with. Similarly, students in the strength condition sometimes began their comment by discussing strengths as a form of mitigation, but then quickly followed up with a weakness comment (e.g. “You improved your essay a lot, however I’m still not sure what your main claim is”). A study design that incorporated the use of confederates posing as students in the class might obviate the contamination that inevitably arises from students need to mix strength and weakness comments. This study also found that it was not strength or weakness comments per se that students found helpful, but rather some of the traits of the strengths or weakness comments. Each of the 461 individual comments that students made on the final revision were coded according to the scheme shown in Table 1. The means and standard deviations (by comment) for the measures of helpfulness used in this study are presented in Table 4. A statistically significant difference was found between groups for the helpfulness ratings and the eight comment traits F(7, 848) = 13.93, p < .01, η2 = 0.10), and a post hoc Tukey test showed that the comment traits of Strength Suggestion, Strength Explanation, Weakness Suggestion, and Weakness Explanation were statistically different from each of the four other comment traits (see Table 6). Therefore the comment traits that were found to be most helpful were explanations of strengths and weaknesses, suggestions for further applications of successful strategies, and solutions to problems identified in the peer review. This finding is consistent with that of Nelson and Schunn (2009), for example, where strength comments that merely identified a positive feature were unrelated to writing quality. 35 This study also found that statements of praise unaccompanied by information on how to proceed are not considered very helpful by students (see Table 4); statements of criticism unaccompanied by information (i.e. censure) were not considered helpful either. Table 5 shows that the majority of feedback students received did not have the qualities that students found most helpful (explanations, suggestions, and solutions). In fact, 52% of the total comments students received did not have the traits that students rated as being most helpful. The finding that students prefer feedback with explanations and suggestions is also supported by a strong positive correlation between helpfulness and the length of comments (r = 0.00, n = 82, p < .01). This may explain why Dragga (1988) found that praiseworthy feedback that required elaboration was preferred by students, as compared to simple error identification. Future research comparing comments requiring elaboration comments versus directive comments or the use of confederates supplying these kinds of comments might help in understanding. One of the most curious findings of this study was that writing quality was not related to student ratings of the helpfulness of the comments received (see Table 3); there are some plausible explanations. First, student attributions in Table 2 indicate that peer feedback was not among the top reasons students identified when asked what factors they felt contributed to their grade on this writing assignment; instead, the most frequent reasons students gave were effort, ability, teacher quality, and strategy. Given that the participants were second semester high school seniors, it can be argued that these students felt reasonably confident of the skills needed to succeed on this writing assignment without much help from others. The second explanation might be, as Tables 4 and 5 indicate, that if majority of the feedback students received was not considered to be 36 the most helpful type, it is reasonable to assume that the preponderance of unhelpful feedback did not impact writing quality. Even though these participants were good writers (approximately 80% of the students in this study passed the AP Composition Exam), these results indicate that adolescent writers in general might need more guidance on how to provide the most helpful feedback for their peers so that writing quality improves. Another explanation might be due to the fact that previous research has found that giving feedback is more helpful to writers than receiving feedback (Lundstrom & Baker, 2009). One more reason might be the length of the time between when the feedback was given and when received. Roseth, Saltarelli, and Glass (2011) found that synchronous forms of computer-mediated cooperative learning resulted in greater achievement and motivation when compared to asynchronous forms; the feedback given in this study sometimes was not received by students until more than two class periods had elapsed. A study design that incorporates a shorter time frame between the receiving of feedback and the implementation of that feedback into a revision plan would be worth exploring. The correlations were generally in line with the predictions about the way the variables should relate to one another. First, not surprisingly there was a strong correlation between effort and strategy. Second, the findings on relational conflict regulation of Doise and Mugny (1984) support the strong negative correlation that was found in this study between students perceived helpfulness of feedback and social comparison. Third, Van Ryzin, Gravely, and Roseth (2009) found that support from friends and peers promotes higher levels of motivation which might explain why there was a strong positive correlation in this study between belongingness and feedback from 37 peers; however, somewhat surprising was the fact that helpfulness had no relationship with the structure of the peer review activity itself. Finally, some findings about the relationship between helpfulness and motivational factors reported in Table 3 are worth noting. A positive correlation was found between helpfulness and belongingness (r = 0.32, n = 82, p < .05). Relationships were also found between helpfulness and the sociocognitive conflict regulation measures; a positive correlation between helpfulness and task understanding (r = 0.28, n = 82, p < .05) and a negative correlation between helpfulness and social comparison (r = -0.34, n = 82, p < .01). Whether students with prosocial tendencies (for example, an orientation toward understanding the task) perceive peer feedback to be more helpful, or whether helpful feedback engenders prosocial behaviors (like belongingness) can’t be answered here. Nevertheless this study found relationships between helpfulness and some motivational factors. Limitations There are some important limitations to this study. First, students mixed strength and weakness comments, despite multiple attempts to not have them do so. Second, the four-week study period is a relatively short amount of time to detect change in writing quality and motivation levels. A third limitation is the sample of students. The participants were Advanced Placement students in a school where 99% go on to higher education or military academies. The vast majority of the students were native-English speakers (93%). As such this sample is not representative of American high school seniors. The fourth limitation has to do with the structure of the peer review activity itself. Since not all students had reliable access to the Internet at home, the study took 38 place during class time. The fact that students were providing feedback to students outside of the classroom via Eli Review and not in the far more common type of face-toface feedback they were used to doing in school, may not have been as natural for them. Additionally the fact that one student providing feedback on strengths, for example, was most likely sitting next to another student who was providing a different type of feedback may have caused some confusion for some student reviewers. Finally, a limitation of the study was the timing; it was conducted during the last few months of the students’ senior year in high school. Since the participants in this study were at the end of their fourth year at this same school, it can be argued that the reasons for success or failure on this assignment did not stem primarily from this peer review activity. Implications As very little research to date has examined the motivational aspects of peer review, this study has implications for the research and practice of peer-based feedback in writing instruction. Given this study found that there were no treatment group differences in students’ motivation and the fact that high school students seem to have a need to provide peer feedback that includes praise, future research might examine the impact of comment traits instead strengths versus weaknesses. With regard to attributions, much of the previous research has taken place where students are transitioning to new learning environments (e.g. Perry, et al., 2008), therefore future research on the attributions of adolescent writers should be situated in settings as students move from junior high to high school or from high school to college. With regard to belongingness and sociocognitive conflict regulation future studies might instead examine more long-range effects of weakness comments on student feelings of rejection or might induce social 39 comparison processes that decrease writing quality over an entire course or academic year. This study examined peer review from the motivational perspectives of attribution theory, belongingness, and sociocognitive conflict regulation, however future research might look at peer review from other motivational lenses such as theories of intelligence (Dweck, 2000), mastery/performance goals (Midgley, Kaplan, & Middleton, 2001), and individualistic/cooperative perceptions (Johnson & Norem-Hebeisen, 1977). One goal of this study was to shed light on what types of feedback adolescent writers found helpful, and this has implications for practitioners. Helpfulness has been found by others to be a key factor in the implementation of peer feedback (e.g. Schunn & Charney, 2006; Hart-Davidson, et al., 2010). Future research might explore the connection between student ratings of helpfulness and implementation of helpful feedback in the revision plan of students. In Eli Review students construct a revision plan based on peer feedback. In this way helpfulness could be measured by what feedback is actually used in student revision plans, and the feedback that is not used would be considered not helpful. Another finding in this study has implications for future writing research. Since there were clear indicators that suggest students saw statistically significant differences in how helpful some types of comments were perceived to be, writing instructors might consider requiring peer reviewers to explain their feedback in more detail and to spend more time suggesting ways for writers to improve. Of particular note was the strength suggestion category, which was the most highly rated type of feedback. Although strength suggestion was the most highly rated type of feedback, it was used less frequently (4%) than the other highly rated feedback traits of strength explanation, 40 weakness explanation, and weakness suggestion. Practitioners might consider this when describing helpful types of peer feedback to student writers. Looking at peer review from both cognitive and motivational perspectives can only expand our understanding of what effective and helpful feedback is for students. 41 APPENDICES 42 APPENDIX A Argumentative Writing Assignment and Traits Writing assignment: Upload the draft of your argumentative essay to Eli Review. DO NOT put your name anywhere on your essay. Copy and paste the draft into the space provided. Based on the Structure for Argument (p. 153 in Curious Researcher) your draft should • • • • • introduce the research question or problem clearly provide an identifiable central claim or thesis explain your reasons for believing what you do support your claims and reasons with sufficient evidence/examples have smooth and logical transitions between ideas Peer review activity Group 1 instructions (control group), Helpful comments (online via Eli Review): Directions: Give the writer feedback that you think would be helpful to consider as they revise the draft of the essay. Here are some things you might consider: 1. The traits that are identified in the assignment itself (for example, evidence and transitions) 2. Other things you know about what makes writing effective in general. 3. Things you know about effective argumentation and research writing in particular. Group 2 instructions, Weakness comments: Directions: Give the writer feedback that will help them improve the essay by commenting on the weaknesses of the draft. 1. Identify what's not working with some of traits of the essay, describing them specifically. 2. Explain what's not working and why. 3. Suggest solutions to any problems you identified that would help the writer improve their essay. 43 Group 3 instructions, Strength comments: Directions: Give the writer feedback that will help them improve the essay by commenting on the strengths of the draft. 1. Identify what's working with some of traits of the draft, describing them specifically. 2. Explain what's working in detail. 3. Suggest places in the draft where the writer could use a similar technique or strategy that would improve their essay. All groups: Peer review response types in Eli Review (trait identification, lLikert scale, contextual comments) Response 1. Trait identification: Are the following traits present? [yes or no] 1. 2. 3. 4. the research question or problem is introduced clearly the argument or central claim of the essay is clear the writer adequately explained the reasons for believing what they do enough convincing evidence/examples are provided to support the writer's claims and reasons 5. transitions between ideas are smooth and logical Response 2. Likert scale: Rate the quality of this initial draft by saying whether you agree with the following statement: "This draft is excellent in every way. It exceeded my expectations for all of the assignment traits." 1. 2. 3. 4. 5. 6. 7. strongly disagree disagree slightly disagree neutral slightly agree agree strongly agree Response 3. Contextual Comments: Support your rating of the initial draft by explaining specific parts of the essay. Consider the following when providing helpful feedback: • • • • • the research question or problem is introduced clearly the argument or central claim of the essay is clear the writer adequately explained the reasons for believing what they do enough convincing evidence/examples are provided to support the writer's claims and reasons transitions between ideas are smooth and logical 44 APPENDIX B Motivation survey All questions are rated on a 7-point scale of the type listed below. Your responses to items 1-7 ask about the causes of your performance on this writing assignment; items 725 gauge your attitude based on the feedback you received on your letter. 1 not at all 2 3 4 5 somewhat 6 7 very much so Attributions Indicate the extent to which the following factors contributed to your performance on this assignment: 1. Ability 2. Assignment difficulty 3. Effort 4. Luck 5. Strategy 6. Teacher quality 7. Feedback from peer reviewers 8. Structure of the peer review activity Belongingness/Relatedness 9. It is likely that the people who reviewed my letter and I could become friends if we interacted a lot. 10. I felt really distant from the people who reviewed my letter. (R) 11. I really doubt that the people who reviewed my letter and I would ever be friends. (R) 12. I felt like I could really trust the people who reviewed my letter. 13. I’d like a chance to interact with the people who reviewed my letter more often. 14. I'd really prefer not to interact with the people who reviewed my letter in the future. (R) 15. I don't feel like I could really trust the people who reviewed my letter. (R) 16. I feel close to the people who reviewed my letter. Sociocognitive Conflict Regulation 45 When you received feedback from your partners, to what extent did you... 17. try to show you were right? 18. try to resist by maintaining your initial position? 19. try to show your partner was wrong? 20. try to think about the text again in order to understand better? 21. try to examine the conditions under which each point of view could help you understand? 22. try to think of a solution that could integrate both points of view? 46 APPENDIX C Tables Table 1. Coding scheme adapted from Cho and MacArthur (2011) Comment type & traits Praise (P) Strength detection (SD) Strength explanation (SE) Strength suggestion/application (SS) Definition Positive affect, no specific suggestion to how to improve writing statement about what is strong/right in peer writing explanation of why it’s good Weakness explanation (WE) statement of another application of the strength/strategy to improve the writing Negative affect, no specific suggestion on how to improve writing statement about what is wrong or weak in peer writing explanation on why the problem happened Weakness suggestion/solution (WS) statement about how to improve the problem Censure (C) Weakness detection (WD) 47 Example I like the essay. It’s pretty interesting The opening paragraph is more fluid in this revision Showing transitions through history was a good technique because it kept the paper interesting from start to finish Use diction like this in some of your other transitions to act as signals. Your essay was a bit strange. You didn’t cite your sources You mention the rods from the reactor, but I feel like you somewhat draw it out too long and that caused me to lose focus when reading. Clarify your central question in your opening paragraph Table 2. Mean and standard deviation by condition and measure Measure Strength Weakness Control Test of Group Differences F(2,79) significance Effort 6.11 (0.83) 5.70 (1.33) 6.04 (0.71) 1.30, n.s. Ability 5.68 (0.67) 5.59 (1.31) 5.78 (1.12) 0.21, n.s. Teacher Quality 5.61 (1.13) 5.48 (1.85) 5.78 (1.45) 0.77, n.s. Strategy 5.61 (1.07) 4.96 (1.51) 5.63 (0.79) 2.90, n.s. Feedback from peer reviewers 5.32 (1.02) 4.89 (1.78) 4.78 (1.53) 1.05, n.s. Structure of peer review activity 5.18 (1.06) 4.85 (1.75) 4.89 (1.65) 0.39, n.s. Assignment Difficulty 4.86 (1.08) 4.56 (1.70) 5.07 (1.17) 1.02, n.s. Luck 3.36 (1.47) 3.74 (1.53) 4.07 (1.82) 1.36, n.s. 4.61 (0.94) 4.36 (0.92) 4.35 (1.01) 0.67, n.s. Social Comparison 3.50 (1.36) 3.49 (1.11) 3.23 (1.31) 0.39, n.s Task Understanding 5.67 (0.56) 5.47 (0.75) 5.54 (0.79) 0.56, n.s 3.60 (0.84) 3.24 (0.95) 3.42 (0.88) 1.11, n.s Draft 3.79 (1.62) 3.75 (1.51) 4.26 (1.46) 0.87, n.s Revision 4.57 (1.50) 4.36 (1.62) 4.70 (1.64) 0.38, n.s Student Attributions Belongingness Conflict Regulation Helpfulness Writing Quality 48 Table 3. Correlations for measures used in this study (by student) Measure 1 1. ability — 2 3 4 2. assignment difficulty .34** — 3. effort .42** .32** — 4. luck -.04 .15 -.20 5. strategy 6. teacher quality 7. feedback from peer reviewers 8. structure of peer review activity ** 5 6 7 8 9 10 11 12 13 — .17 .13 .32 .02 — .36** .43** .36** .04 .19 — .07 .33** .19 .03 .25* .28* — .15 .42** .24* .10 .24* .49** .73** ___ .05 .29** .17 .13 .14 .25* .54** .50** — -.13 .05 -.02 -.08 -.21 -.16 -.24* — . 9. belongingness 10. social comparison 11. task understanding -.22* -.05 -.00 .35** .23* -.13 .20 .42** .40** .44** .29** -.13 — 12. helpfulness -.02 .19 .14 .15 .00 -.03 .40** .15 .23* -.34** .28* — .07 .02 .18 -.26* -.10 .04 -.01* -.05 -.23* .06 -.03 -.05 — -.06 .06 .09 -.16 .08 .03 .07 .01 .01 .00 .13 -.02 .63** 13. instructor draft rating 14. instructor revision rating 49 Table 4. Means and standard deviations (by comment) for measures of helpfulness Comment trait Mean Std. Dev. N Strength Suggestion 4.22 0.72 36 Strength Explanation 3.91 0.96 92 Weakness Explanation 3.88 1.09 83 Weakness Suggestion 3.87 1.17 198 Strength Detection 3.43 1.11 207 Praise 3.33 1.27 84 Weakness Detection 3.01 1.40 151 Censure 1.40 0.89 5 3.56 1.23 856 Total 50 Table 5. Frequency of comment traits received per treatment group Comment trait Control Weakness Strength Praise 36 18 30 Strength Detection 48 10 149 Strength Explanation 28 0 64 Strength Suggestion 0 1 35 112 19 278 Censure 2 3 0 Weakness Detection 59 68 24 Weakness Explanation 39 37 7 Weakness Suggestion 88 76 34 Total Weakness Comments 188 184 65 Total traits 300 213 343 Total Strength Comments 51 Table 6. Tukey post hoc comparison of comment traits Comparison p Comparison p SS – SE .879 SD – WS .003* SS – SD .004* SD – WE .055 SS – P .003* SD – WD .019* SS – WS .703 SD – C .003* SS – WE .821 P – WS .010* SS – WD .000* P – WE .051 SS – C .000* P – WD .441 SE – SD .019* P–C .008* SE – P .023* WS – WE 1.000 SE – WS 1.000 WS – WD .000* SE – WE 1.000 WS – C .000* SE – WD .000* WE – WD .000* SE – C .000* WE – C .000* SD – P .999 WD – C .051 Note. SS = Strength Suggestion; SE = Strength Explanation; SD = Strength Detection; P = Praise; WS = Weakness Suggestion; WE = Weakness Explanation; WD = Weakness Detection; C = Censure. 52 REFERENCES 53 REFERENCES Ames, C. (1992). Classrooms: Goals, structures, and student motivation. Journal of Educational Psychology, 84, 261–271. Asher, S. R., & Coie, J. D. (1990). Peer rejection in childhood. CUP Archive. Atwater, L. E., Waldman, D. A., Atwater, D., & Cartier, P. (2000). An upward feedback field experiment: Supervisors’ cynicism, reactions, and commitment to subordinates. Personnel Psychology, 53(2), 275-297. Baumeister, R. F., Campbell, J. D., Krueger, J. I., & Vohs, K. D. (2003). Does high selfesteem cause better performance, interpersonal success, happiness, or healthier lifestyles?. Psychological science in the public interest, 4(1), 1-44. Baumeister, R. F., & Leary, M. R. (1995). The need to belong: desire for interpersonal attachments as a fundamental human motivation. Psychological bulletin, 117(3), 497. Beason, L. (1993). Feedback and Revision in Writing across the Curriculum Classes. Research in the Teaching of English, 27(4), 395–422. Braaksma, M. A., Rijlaarsdam, G., Van den Bergh, H., & van Hout-Wolters, B. H. M. (2004). Observational learning and its effects on the orchestration of writing processes. Cognition and Instruction, 22(1), 1-36 Breuch, L. A. K. (2004). Virtual peer review: Teaching and learning about writing in online environments. SUNY Press. Brophy, J. (1981). Teacher Praise : A Functional Analysis. Review of Educational Research, 51(1), 5–32. Brophy, J. (2008). Developing students' appreciation for what is taught in school. Educational psychologist, 43(3), 132-141. Bruffee, K. A. (1984). Collaborative learning and the conversation of mankind. College English, 635-652. Buchs, C., Butera, F., Mugny, G., & Darnon, C. (2004). Conflict elaboration and cognitive outcomes. Theory Into Practice, 43(1), 23-30. Buchs, C., Pulfrey, C., Gabarrot, F., & Butera, F. (2010). Competitive conflict regulation and informational dependence in peer learning. European Journal of Social Psychology, 40(3), 418-435. 54 Butera, F., & Mugny, G. (1995). Conflict between incompetences and influence of a low expertise source in hypothesis testing. European Journal of Social Psychology, 25, 457–462 Chi, M. T., Bassok, M., Lewis, M. W., Reimann, P., & Glaser, R. (1989). Self‐ explanations: How students study and use examples in learning to solve problems. Cognitive science, 13(2), 145-182. Cho, K., & Cho, Y. H. (2011). Peer reviewers learn from giving comments, Instructional Science, 39(5), 629–643. Cho, K., & MacArthur, C. (2010). Student revision with peer and expert reviewing. Learning and Instruction, 20(4), 328–338. Cho, K., & MacArthur, C. (2011). Learning by reviewing. Journal of Educational Psychology, 103(1), 73–84. Cho, K., & Schunn, C. D. (2007). Scaffolded writing and rewriting in the discipline: A web-based reciprocal peer review system. Computers & Education, 48(3), 409426. Cho, K., Schunn, C. D., & Charney, D. (2006). Commenting on writing typology and perceived helpfulness of comments from novice peer reviewers and subject matter experts. Written Communication, 23(3), 260-294. Cho, K., Schunn, C. D., & Wilson, R. W. (2006). Validity and reliability of scaffolded peer assessment of writing from instructor and student perspectives. Journal of Educational Psychology, 98(4), 891-901. Connell, J. P., & Wellborn, J. G. (1991). Competence, autonomy, and relatedness: A motivational analysis of self-system processes. In R. Gunnar & L.A. Sroufe (Eds.), Self processes in development: Minnesota Symposium on Child Psychology, (pp.43-77). Chicago: Chicago University Press. Daiker, D. A. (2011). Learning to praise. Concepts in Composition: Theory and Practice in the Teaching of Writing, 168-174. Darnon, C., Butera, F., & Harackiewicz, J. M. (2007). Achievement goals in social interactions: Learning with mastery vs. performance goals. Motivation and Emotion, 31(1), 61-70. Darnon, C., Muller, D., Schrager, S. M., Pannuzzo, N., & Butera, F. (2006). Mastery and performance goals predict epistemic and relational conflict regulation. Journal of Educational Psychology, 98(4), 766. 55 Deci, E. L., & Ryan, R. M. (1985). The general causality orientations scale: Selfdetermination in personality. Journal of research in personality, 19(2), 109-134. Deci, E. L., & Ryan, R. (2000). The “what” and “why” of goal pursuits: Human needs and the self-determination of behavior. Psychological Inquiry, 11(4), 227-268. Doise, W., & Mugny, G. (1984). The social development of the intellect. Oxford: Pergamon Press. Doise, W., Mugny G., & Perez, J. A. (1998). The social construction of knowledge: Social marking and socio-cognitive conflict. In U. Flick (Ed.), The psychology of the social (pp. 77–90). Cambridge: Cambridge University Press. Dragga, S. (1988). The effects of praiseworthy grading on students and teachers. Journal of Teaching Writing, 7(1), 41–50. Dweck, C.S. (2000). Self-theories: Their role in motivation, personality and development. Philadelphia: Taylor & Francis. Elbow, P. (1993). Ranking, Evaluating, and Liking: Sorting out Three Forms of Judgment. College English, 55(2), 187-206. Elbow, P. (1998). Writing without teachers. New York: Oxford University Press. Emerson, R. M. (1954). Deviation and rejection: An experimental replication. American Sociological Review, 19(6), 688-693. Festinger, L. (1954). A theory of social comparison processes. Human Relations, 7(2), 117–140. Fleiss, J. L., Levin, B., & Paik, M. C. (2013). Statistical methods for rates and proportions. Hoboken, NJ: John Wiley & Sons. Gee, T. (1972). Students’ responses to teachers’ comments. Research in the Teaching of English, 6(2), 212-221. Gest, S. D., Graham-Bermann, S. A., & Hartup, W. W. (2001). Peer experience: Common and unique features of number of friendships, social network centrality, and sociometric status. Social development, 10(1), 23-40. Gick, M. L., & McGarry, S. J. (1992). Learning from mistakes: Inducing analogous solution failures to a source problem produces later successes in analogical transfer. Journal of Experimental Psychology: Learning, Memory, and Cognition, 18(3), 623-639. 56 Grabill, J. T., & Hicks, T. (2005). Multiliteracies meet methods: The case for digital writing in English education. English Education, 37(4), 301-311. Graham, S., & Perin, D. (2007). A meta-analysis of writing instruction for adolescent students. Journal of educational psychology, 99(3), 445-476. Hart-Davidson, W., McLeod, M., Klerkx, C., & Wojcik, M. (2010). A method for measuring helpfulness in online peer review. In J. Anecleto (Ed), Proceedings of the 28th ACM International Conference on Design of Communication (pp. 115121). Sao Paulo, Brazil: Association for Computing Machinery. Hattie, J. (1999). Influences on student learning. Retrieved from http://xn-wwwrp0a.teacherstoolbox.co.uk/downloads/managers/Influencesonstudent.pdf Hattie, J., & Timperley, H. (2007). The power of feedback. Review of educational research, 77(1), 81-112. Hewett, B. L. (2000). Characteristics of interactive oral and computer-mediated peer group talk and its influence on revision. Computers and Composition,17(3), 265288. Johnson, D. W., & Johnson, R. T. (1995). Creative controversy: Intellectual challenge in the classroom. Interaction Book Company. Johnson, D. W., & Johnson, R. T. (2009). Energizing learning: The instructional power of conflict. Educational Researcher, 38(1), 37-51. Johnson, D. W., & Norem-Hebeisen, A. (1977). Attitudes toward interdependence among persons and psychological health. Psychological Reports, 40(3), 843-850. Juvonen, J. (2006). Sense of belonging, social bonds, and school functioning. Handbook of educational psychology, 2, 655-674. Juvonen, J., & Cadigan, J. (2002). Social determinants of public behavior of middle school youth: Perceived peer norms and need to be accepted. In T. Urdan (Ed.), Adolescence and education. Vol. 2: Academic motivation of adolescents (pp. 277–297). Greenwich, CT: Information Age Publishing. Kellogg, R. T., & Whiteford, A. P. (2009). Training advanced writing skills: The case for deliberate practice. Educational Psychologist, 44(4), 250-266. Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 159-174. Lauderdale, P. (1976). Deviance and moral boundaries. American Sociological Review, 41, 660-676. 57 Levine, J. M. (1980). Reaction to opinion deviance in small groups. In P. B. Paulus (Ed.), Psychology of group influence (pp. 101-123). Hillsdale, NJ: Erlbaum. Lundstrom, K., & Baker, W. (2009). To give is better than to receive : The benefits of peer review to the reviewer’s own writing. Journal of Second Language Writing, 18, 30–43. Nelson, M. M., & Schunn, C. D. (2009). The nature of feedback: How different types of peer feedback affect writing performance. Instructional Science, 37(4), 375-401. Mabrito, M. (1991). Electronic Mail as a Vehicle for Peer Response Conversations of High-and Low-Apprehensive Writers. Written Communication, 8(4), 509-532. McGarty, C., Turner, J. C., Oakes, P. J., & Haslam, S. A. (1993). The creation of uncertainty in the influence process: The roles of stimulus information and disagreement with similar others. European Journal of Social Psychology, 23(1), 17–38. Midgley, C., Kaplan, A., & Middleton, M. (2001). Performance-approach goals: Good for what, for whom, under what circumstances, and at what cost?. Journal of Educational Psychology, 93(1), 77. Mugny, G., De Paolis, P., & Carugati, F. (1984). Social regulations in cognitive development. In W. Doise & A. Palmonari (Eds.), Social interaction in individual development (pp. 127–146). Cambridge: Cambridge University Press. Murray, D. (1972). Teach writing as a process not product. The Leaflet, 71(3), 11-14. Nelson, M. M., & Schunn, C. D. (2009). The nature of feedback: How different types of peer feedback affect writing performance. Instructional Science, 37(4), 375-401. Parkhurst, J. T., & Asher, S. R. (1992). Peer rejection in middle school: Subgroup differences in behavior, loneliness, and interpersonal concerns. Developmental Psychology, 28(2), 231. Patchan, M. M., Charney, D., & Schunn, C. D. (2009). A validation study of students ’ end comments : Comparing comments by students , a writing instructor , and a content instructor, Journal of Writing Research, 1(2), 124–152. Perez, J. A., & Mugny, G. (1996). The conflict elaboration theory of social influence. In E. Witte, H., & J. Davis, H. (Eds.), Understanding Group Behavior: Small Group Processes and Interpersonal Relations, (pp. 191–210). New Jersey: Lawrence Erlbaum Associates. 58 Perry, R. P., Stupnisky, R. H., Daniels, L. M., & Haynes, T. L. (2008). Attributional (explanatory) thinking about failure in new achievement settings. European Journal of Psychology of Education, 23(4), 459-475. Piaget, J. (1952). The origins of intelligence in children. New York: International Universities Press. Renkl, A., Stark, R., Gruber, H., & Mandl, H. (1998). Learning from worked-out examples: The effects of example variability and elicited self-explanations. Contemporary educational psychology, 23(1), 90-108. Roseth, C. J., Saltarelli, A. J., & Glass, C. R. (2011). Effects of face-to-face and computer-mediated constructive controversy on social interdependence, motivation, and achievement. Journal of Educational Psychology, 103(4), 804. Saltarelli, A. J., & Roseth, C. J. (2014). Effects of synchronicity and belongingness on face-to-face and computer-mediated constructive controversy. Journal of Educational Psychology, 106(4), 946-960. Schonert-Reichl, K. A. (1999). Relations of peer acceptance, friendship adjustment, and social behavior to moral reasoning during early adolescence. The Journal of Early Adolescence, 19(2), 249-279. Schunn, C. D., & Charney, D. (2006). Commenting on writing typology and perceived helpfulness of comments from novice peer reviewers and subject matter experts. Written Communication, 23(3) 260–294. Seidman, E. (1968). Marking students’ compositions: Implications of achievement motivation theory. (Doctoral dissertation). Dissertation Abstracts International, 28, 2605-A. Siegler, R. S. (2002). Microgenetic studies of self-explanation. In N. Granott & J. Parziale (Eds.), Microdevelopment: Transition processes in development and learning, 31-58. New York: Cambridge. Smith, F. (1983). Reading like a writer. Language arts, 60(5), 558-567. Sommers, N. (1980). Revision strategies of student writers and experienced adult writers. College composition and communication, 31(4), 378-388. Strong, W. (2012). Coaching writing in the content areas: Write-for-Insight strategies, grades 6-12. Boston, MA; Pearson. Tjosvold, D., & Johnson, D. W. (1977). Effects of controversy on cognitive perspective taking. Journal of Educational Psychology, 69(6), 679-685. 59 Valkenburg, P. M., Peter, J., & Schouten, A. P. (2006). Friend networking sites and their relationship to adolescents' well-being and social self-esteem. CyberPsychology & Behavior, 9(5), 584-590. Van Ryzin, M. J., Gravely, A. A., & Roseth, C. J. (2009). Autonomy, belongingness, and engagement in school as contributors to adolescent psychological wellbeing. Journal of youth and adolescence, 38(1), 1-12. Weiner, B. (1974). Achievement motivation and attribution theory. Morristown, N.J.: General Learning Press. Weiner, B. (1985). An attributional theory of achievement motivation and emotion. Psychological review, 92(4), 548. Wentzel, K. R., & McNamara, C. C. (1999). Interpersonal relationships, emotional distress, and prosocial behavior in middle school. The Journal of Early Adolescence, 19(1), 114-125. Wiggins, J. A., Dill, F., & Schwartz, R. D. (1965). On “status-liability.” Sociometry, 28(2), 35-43. Zhao, Y. (1998). The effects of anonymity on computer-mediated peer review. International Journal of Educational Telecommunications, 4(4), 311-345. 60