posted in Technology
Universities drop AI detection tools over fears about accuracy: Some institutions are overhauling assessment and trying to move away from the emphasis on surveillance
www.ft.com/content/49304b1e-8a9d-4fb6-bc4d-37dd3430bb98posted in Technology
Universities drop AI detection tools over fears about accuracy: Some institutions are overhauling assessment and trying to move away from the emphasis on surveillance
www.ft.com/content/49304b1e-8a9d-4fb6-bc4d-37dd3430bb98Replying to @eicker@lemmy.world
The gold standard would be to make each student do an oral defense of their work. Hand-written exams are good too. Both of these approaches are time-consuming to mark, though.
Assessment design can require students to perform analysis of rare and obscure source material, so that there are no published materials for AI to draw on (which can make it struggle). Similarly, asking for a good bibliography that lists the sources used in the text and formats it correctly would show up the hallucinatory tendencies of AI, I think — though I haven’t tried the latest models.
Replying to @ooo_shiny@lemmy.nz
Agreed 100%.
Every exam should have some oral component. Doesn’t need to be the whole of one’s grade, but 10% sounds about right: significant enough to be impactful, but not so significant that people with severe anxiety can’t otherwise pass.
Replying to @FlashMobOfOne@lemmy.world
My biggest fear was this. I hated getting up information of a class
Replying to @Polisheocket@lemmy.zip
Same, but it motivated me to do the work, and it’s a skill that crosses over into work life significantly.
Replying to @ooo_shiny@lemmy.nz
Doesn’t scale.
It works great. That’s basically how it’s done in aviation, flight instruction is inherently 1 on 1, so I’d assign my students to do some reading and then it was pretty obvious if they’d done that the next flight based on who was asking questions. The ones that did the reading will come in asking me to clarify something, if they say “Yeah I read it” and I have to do the clarifying they didn’t read it. Oral and practical tests are the standard, show me you can fly the plane.
To make that work, most professional pilots have to be flight instructors at some point. It’s almost certainly the first flying job you’ll get after earning a commercial certificate, and thus spend at least some time doing that job. You’re not going to get that in most college classes, so where are you going to get the labor necessarily to give an oral exam to every student in every class?
Replying to @captain_aggravated@sh.itjust.works
Yes — I think if I were to teach a large undergraduate class tomorrow I would go with a combination of handwritten exams and, for homework assignments, critical analyses of the most obscure examples of whatever is being studied possible so that AI has no idea what you’re even on about (in my field, ask it to analyse music; it’s hopeless). Multiple choice is fine for stuff that can be rote learned.