THE MORNING RANT: Brown University Professor Sets Trap for Students to Reveal they Cheated Using AI

When a professor at Brown University decided that mid-term and final exams would be take-home tests following the campus shooting last December, he inadvertently set a trap that revealed the seriousness of AI cheating on exams.

Professor Roberto Serrano wrote a first-hand account of the AI cheating story at The Free Press (“I’m a Brown University Professor; Half My Students Are Cheating” ) but it’s behind a paywall.

For an account of the situation without a paywall, Inside Higher Ed gave the story wide exposure last week in a piece titled “Brown Professor Suspects Majority of His Class Used AI to Cheat”

Here is a brief overview of the highlights (written by me, not by AI.)

• Roberto Serrano is an Economics professor at Brown, who teaches a course titled “Welfare Economics and Social Choice Theory.”

• This was the first time that Serrano was going to have the mid-term and final exam be in “take-home, closed-book” format rather than in-class as they had always been.

• The mid-term exam was accessible through an online portal, and students had an 11-hour window to complete it.

• Of the 86 students who took the mid-term, 40 had perfect scores. The average score was 96. By contrast, in prior semesters with in-class exams, the average scores ranged from 65 to 80.

• By running the test through ChatGPT, Serrano found that its AI answers closely mirrored what his students had written.

With this information, Professor Serrano announced to the class that he suspected many students had cheated using AI. He then sprung the trap by announcing:

1) The final exam was now going to be in-person.

2) The mid-term results would be considered void unless the final exam had a similar distribution of extraordinary grades.

In the words of Mr. Serrano “I am not declaring [the midterm] void for now. I am going to give the class a chance to prove me wrong. That is, if the distribution of the final exam is roughly similar to the distribution of the midterm, I will count the midterm. Otherwise, which is of course what I expect to happen, I will declare the midterm void and reweigh the final accordingly.”

Following that announcement, 27 of the 86 students either dropped the class entirely, or didn’t show up for the final exam. That is a whopping 31% who pretty much admitted to cheating on the mid-term.

But it gets even funnier for those who stayed in class:

• The remaining 59 students had an average score on the final exam of just 49%, a huge dropoff from the 96% average on the mid-term.

• Serrano therefore announced that the mid-term would not be counted in their grade. A score of 40 on the final was now required to pass. (This was lower than the 50 that was his previous pass threshold.)

• 19 of the remaining 59 students failed the class.

Below the fold is a chart showing the distribution of the 59 students’ final exam grades on the left, and their extraordinary, AI-assisted mid-term grades on the right.

A few students stand out. Especially noteworthy are the two students who clearly didn’t cheat:

• Student #1 scored a 95 on both exams.

• Student #22 scored a 55 on the midterm and then improved to a 59 on the final. S/he is the only student whose grade improved on the final.

• Two students scored a 100 on the mid-term and a 0 on the final.

Historically for employers, the importance of a college degree was that it served as an aptitude test, and it also documented that graduating students had endured a rigorous, four-year (or more) credentialing program. This scholastic tenacity was assumed to translate into a solid work ethic.

At this point, a mail-order diploma is as much a signifier of aptitude as a degree earned by ChatGPT, and the brazen use of AI to take an exam is no different than hiring a scholastic ringer to take the final for you. People in my day got expelled for doing that.

An actual education has almost nothing to do with why young adults are still going to college. There is still some residual prestige to having a degree from a name-brand university, and the power of networking is still strong. Plus, there is the pull of “the college experience.” But if the majority of students have no interest in actually getting an education, then the university model has run its course.

I almost feel sorry for the kids graduating college now, because there is an assumption that their degree is fraudulent due to AI. Even sadder, those who don’t cheat are almost certainly paying the price with a below-average GPA.

With all that said, if I were hiring right now, I’d try to find out who student number 22 is and hire him. He is smart enough to be a middle-of-the-pack student, and he’s ethical enough not to cheat, even when it’s an accepted practice by virtually all of his peers.

[buck.throckmorton at protonmail dot com]

posted by Buck Throckmorton at 10:52 AM

| Access Comments