Grading With AI

The Verge has a story that is at once hilarious, infuriating, and deeply disturbing. In this time of COVID-19 much education has moved online. Except for scale, this isn’t really new. It’s been possible for many years for children to get their education online. Some students—especially high school students—prefer that option to dealing with all the nonsense a traditional high school education entails.

Now, of course, we’ve had to scale up to include almost everyone and those providing that education are new to dealing with online learning. It’s hard for teachers used to the classroom but not impossible. I have a family member who is a middle school teacher and she has mostly tailored her classes to run through Zoom. Mike Zamansky, of Using Emacs fame, is a university teacher and he, too, has adapted to running his classes online. Neither he nor my family member thinks it’s as good as in-person teaching but they’ve adjusted their teaching to provide the best pedagogy they can.

Sadly, not everyone has adapted as well. The Los Angeles Unified School District, for one, has attempted to automate the teaching and grading of several courses. That’s not necessarily bad but, as the article relates, the “AI” grading of short answer tests involved merely scanning for keywords. Of course, the students immediately discovered this and started gaming the system by simply answering the question with a word salad of any keywords that might be applicable to the question. Sometimes, even just cutting and pasting the question into the answer would work.

The school district has, of course, tried to frame this behavior as cheating but I think it merely demonstrates that the students are way smarter than the school district. If you’re going to test students, then you’re morally obligated to carefully and rigorously evaluate the answers to those tests. When you outsource that to a simple minded algorithm that simply scans for keywords, it’s you who is cheating and you should shut up about students who found you out and responded appropriately.

As I say, it hilarious that the students discovered this and reacted suitably. But it’s also infuriating and disturbing that those charged with educating our children would do so with such an inadequate and immoral system. There are plenty of teachers who are proving daily that we can do much better.

This entry was posted in General and tagged . Bookmark the permalink.