Iris Yocarini - Decision Accuracy in a Conjunctive versus Compensatory Grading System
Iris Yocarini researched the decision accuracy of the Binding Study Advice (BSA), a Dutch academic dismissal policy in which students either get a positive (continue to the second year) or a negative binding study advice (leave the bachelor program). Given the high stakes of the BSA, she believes it's important to evaluate the accuracy of this decision. To do so, one can compare the BSA decision based on true scores and observed scores.
The true score consists of different factors, such as interest and motivation. Ideally, academic institutions would make the BSA decision based solely on this score, because it would mean they would always make the right decision. Unfortunately, they can't, since this score isn't observable.
Tests provide us with an observed score. Here, we need to take in account a random measurement error, which may cause the observed score to be higher or lower compared to the true score.
Yocarini compared the accuracy of the conjunctive decision rule and the compensatory decision rule:
- The conjunctive decision rule is traditional in Dutch academic education and is based on each individual course.
- The compensatory decision rule is part of the 'Nominal = Normal' system at Erasmus University Rotterdam and allows (to some extent) for compensation of grades. The BSA is now based on the GPA, not on each individual course.
A common pro-compensation argument is that the 'average is more reliable'. But is this assumption actually true?
A simulation study led to the following results:
- The compensatory rule generally performs better and has a lower proportion of misclassifications.
- The more compensation you allow, the higher the sensitivity rate, which means a lower false negative rate (positive BSA based on the true score, but a negative BSA based on observed score) and decisions are more accurate.
- The more compensation you allow, the lower the specificity rate, which means a higher false positive rate (negative BSA based on true score, but a positive BSA based on observed score).
Which of the results is more important, depends on the perspective. From the perspective of the university, you might rather reduce the false positives. However, from the point of view of the student, you'd want to reduce the false negatives.
Gerard Baars - Impact of Insufficient Grades in a Compensatory Grading System on Student Progress
Within the past few years, the 'Nominal = Normal' system has been implemented at the Erasmus University Rotterdam. It aims to improve student success in the first year, so students can start their second year without any delay.
This is determined by four measures:
- 60 ECTS as a minimum credit requirement
- A limitation of the number of re-examinations
- The possibility to compensate for insufficient grades, for example with higher grades on other courses
- The introduction of active learning
In his research, Gerard Baars focused on the compensation regulation for insufficient grades. He longitudinally studied the number of compensations students used in their first year in relation to the number of ECTS they gained during the years afterwards.
Apart from two bachelor programs at the Erasmus School of Economics, 50 percent of students in other programs didn't use any compensation. The number of students who used two compensations was very small.
Baars also analysed the number of credits of the students after one and two years. He found that in some bachelor programs, students who didn't use any kind of compensation obtained more credits than students who did.
Based on this data, Baars suggests to limit the number of compensations in the first year to a number of 2. After all, students who used more than two compensations also had the weakest study progress in their second year.
Futhermore, Baars believes that if you raise the bar, a part of this group will most likely succeed.
This is an article by Jelena Barisic.