Conversation

Notices

1. Embed this notice
   Paul Cantrell (inthehands@hachyderm.io)'s status on Thursday, 29-Jun-2023 06:04:26 JST Paul Cantrell

   • Carl T. Bergstrom

   Louder for the folks in the back!

   “Many (all?) current ChatGPT detectors have not been adequately assessed for issues of algorithmic bias and therefore should not be used to accuse students of misconduct in their written work.”

   All carefully stated, with appropriately judicious academic language, but these are some pretty stark “unsafe at any speed” results.

   Do not use putative AI detectors in your classroom. Just don’t. Please

   From @ct_bergstrom:
   https://fediscience.org/@ct_bergstrom/110623757885080991

   • Embed this notice
     Paul Cantrell (inthehands@hachyderm.io)'s status on Thursday, 29-Jun-2023 06:07:58 JST Paul Cantrell

     in reply to

     I’ll add an incautious, injudiciously stated Paul note to the above:

     The problem isn’t merely that the products don’t work. The problem is that these products are trying to solve — automating the detection of cheating — is a nonsense problem.

     If you set out looking to oil snakes, •you will end up buying snake oil•.

   • Embed this notice
     Paul Cantrell (inthehands@hachyderm.io)'s status on Thursday, 29-Jun-2023 06:11:26 JST Paul Cantrell

     in reply to

     improving pedagogy > policing pedagogy

   • Embed this notice
     Paul Cantrell (inthehands@hachyderm.io)'s status on Thursday, 29-Jun-2023 06:34:52 JST Paul Cantrell

     in reply to

     • Carl T. Bergstrom
     • Cleopatra

     @Cleopatra @ct_bergstrom
     Oh, the “real” exams have even more problems: they still allow cheating, they have accessibility problems up the wazoo, and they’re absolute •shit• for retention. It turns out that cramming for an exam is a terrible way to learn things, big shocker.

     Mass education is a tricky problem no matter how you slice it. Educating and evaluating at the same time multiplies the problems tenfold. And then when you try to increate the student-teacher ratio…

   • Embed this notice
     cleopatra@c.im's status on Thursday, 29-Jun-2023 06:34:53 JST Cleopatra

     in reply to

     • Carl T. Bergstrom

     @inthehands @ct_bergstrom
     That looks like it can get pretty nasty ☹️.
     Cheating-detection is the kind of stuff where you have to be *sure*, and if your algorithm just searches for people writing in simple and broken english then it won't be the rich white guys getting accused.

     imo there's a related issue that was already exposed by 'the shadow scholar' a few years ago: too many damn essays, too few real exams where you have to sit in a room and where cheating is pretty hard.