By Ted Murcray In a nutshell Short answer: Maybe, but not likely.\u00a0 A recent study found that easy grading was not associated with increased SET scores. Increasing clarity of expectations for the course is a more solid path to increasing course evaluation scores. Longer answer: When studied as an independent variable, grades were not predictive of SET scores.\u00a0 Instructors who are \u201ceasy\u201d graders were less likely to have strong correlations between their grades and their SET scores, indicating that you can\u2019t \u201cbuy\u201d high SET scores by giving students better grades as some researchers have suggested (Marsh, 1984). In fact, only 4.8% of the variance in SET scores can be explained by grading reliability, which means final grades in a class have very little to do with how students rate the class on course evaluations. The same study then compared five items from the SET to grading practices of the instructors and found that high ratings in areas of the course and instructor clarity explained 77% of the variation in SET scores.\u00a0 In other words, instructors can grade easily to get a little bump in course evaluation scores, but if they want big changes in scores, they will need to work on the level of clarity found in the course. How do we know if someone is an easy grader or a hard grader? In 2018, Millet studied the grading practices of instructors in higher education by comparing the grades that an instructor assigned in one section of a class to the students\u2019 overall GPA.\u00a0 If an instructor assigned a grade that was consistent with the students\u2019 GPA, then that instructor was considered reliable.\u00a0 Lenient graders assigned grades that were higher than the students\u2019 GPA, and tough graders assigned grades that were lower than the students\u2019 GPA.\u00a0 To recap: Grading reliability: When the grade the student receives in the course matches the student\u2019s GPA, the grade would be considered reliable because it is consistent with other data points.\u00a0 This study didn\u2019t measure reliability at the student level but at the course level, so student grades were averaged and compared to the average GPAs of the same students. Grading leniency: When the grade the student receives in the course is higher than the student\u2019s GPA, the grade might be considered lenient. Tough Grading, or Strict Grading: When the grade the student receives in the course is lower than the student\u2019s GPA, the grading would be considered tough or strict. What is the leniency hypothesis and where did it come from? Millet\u2019s (2018) study did not include SET scores, so he was not able to make any connections in his article.\u00a0 However, in his discussion, he suggested that easy graders may be doing so to get higher SET scores, which is a theory posited by prior researchers, such as Marsh (1984).\u00a0 Marsh suggested that instructors \u201cbuy\u201d high SET scores by giving high grades. Brockx, Spooren, and Mortelmans (2011) noted that those who debate whether easy grading leads to high SET scores fall into two camps: the Leniency Hypothesis or the Validity Hypothesis. Leniency Hypothesis: lenient grading causes higher SET scores Validity Hypothesis: strong teachers teach so well that grades go up and students feel good about their learning, which causes higher SET scores.\u00a0 Millet suggested that future research should be conducted to see which of these hypotheses is true.\u00a0 He reasoned that a researcher could replicate his study on grading reliability and use the results as an independent variable to evaluate the correlations between easy grading and SET scores. So, do easy graders get higher SET scores? Calkins, et.al. (2022) conducted the study that Millet suggested by comparing grading reliability data with SET scores.\u00a0 This study included data from a large comprehensive university with eleven (11) years\u2019 worth of data.\u00a0 They looked to see if there was a positive correlation between grades and SET scores (high grades and high SET scores go together; low grades and low SET scores go together).\u00a0 Then they compared the results using grading reliability as an independent variable (Does being an easy grader correlate with high SET scores? Does being a tough grader correlate with low SET scores?) What they found was that grading reliability is not predictive of SET scores.\u00a0 Only 4.8% of the variance in SET scores can be explained by grading practices.\u00a0 Instructors who were rated as lenient graders had low correlations between their grades and their SET scores.\u00a0 That means even though their grades were high, the SET scores were all over the place.\u00a0 Instructors who were tough graders were more likely to have strong correlation between their grades and their SET scores, but that doesn\u2019t necessarily mean higher scores.\u00a0 For example, if a tough grader gave all F\u2019s, it is likely the SET scores would be correlated with that, which means the SET scores would also be low.\u00a0 In short, this study refutes the Leniency Hypothesis.\u00a0 Instructors cannot \u201cbuy\u201d higher SET scores by giving out higher grades.\u00a0 Then, how do I raise my course evaluation scores? Calkins and her team compared the items from the SETs administered with the grading reliability indicators and the overall SET scores.\u00a0 They found that five items in the course evaluation, taken together, account for 77% of the variance in SET scores.\u00a0 Those items are: Objectives were clearCourse provided sufficient opportunity to learnGrades adequately reflect the quality of my performanceCourse challenged me intellectuallyCourse increased my knowledge This aligns with the literature on strong course design.\u00a0 Well-designed courses coupled with intentional course delivery results in higher course evaluation scores, regardless of the grades that the students earn. Other interesting tidbits Some other interesting findings from these two ground-breaking research studies: Both studies found that instructor experience was negatively correlated with grading reliability.\u00a0 This indicates that we become less consistent with our grading the longer we work in higher education.Both analyses found a positive and significant influence on grading reliability by course level.\u00a0 The more advanced courses are associated with greater grading reliability. Research Questions Interested faculty may want to continue this study.\u00a0 Here are some additional questions that could be explored and might spark additional research questions. What are factors that contribute to or explain the decrease in grading reliability as instructors gain experience?\u00a0 Both qualitative and quantitative studies could be done to explore instructor perceptions about their own grading practices and correlated with grading reliability measures.Why are instructors more consistent in grading upper-division courses? Is this a sign that they are giving more constructive feedback to majors within their program? Or is this an indication that grades become more even at the top levels because students have selected into the work?\u00a0 Is something else contributing to this phenomenon?How can we test the Validity Hypothesis of the relationship between SET scores and grades? Refuting the Leniency Hypothesis does not necessarily mean the Validity Hypothesis is accurate.\u00a0 What could be done to determine the accuracy of that hypothesis? If you are interested in any of these questions, feel free to grab them and begin your study!\u00a0 If you feel you need assistance, please contact the TLC for more information! References Brockx, B., Spooren, P., & Mortelmans, D. (2011). Taking the grading leniency story to the edge. The influence of student, teacher, and course characteristics on student evaluations of teaching in higher education.\u00a0Educational assessment, evaluation, and accountability,\u00a023(4), 289-306. Calkins, C., Crooker, J., & Shi, Q. (2022, April 19-27). The influence of grading reliability and grading leniency metrics on student evaluations of teaching [Conference presentation]. AERA 2022 Conference, San Diego, CA, United States. Marsh, H. W. (1984). Students' evaluations of university teaching: Dimensionality, reliability, validity, potential biases, and utility.\u00a0Journal of educational psychology,\u00a076(5), 707. Millet, I. (2018). The relationship between grading leniency and grading reliability.\u00a0Studies in Higher Education,\u00a043(9), 1524-1535.