AI detectors as saviors of written exams?

AI detec­tors can be used to a lim­ited extent for the analy­sis of exam­i­na­tion papers. Courts have now also ruled on this and issued rec­om­men­da­tions.

In the next few blog posts, we will be look­ing specif­i­cally at the topic of AI in uni­ver­sity teach­ing. The first ques­tion will be to what extent tools for AI recog­ni­tion should be used in exam­i­na­tion papers. In the sec­ond arti­cle, we will look at the legal and didac­tic use of AI in the cre­ation of OER . We then dis­cuss prompt tips for high-qual­ity OER and con­clude the series with AI skills that will soon be manda­tory for uni­ver­sity employ­ees.

Examination papers with AI

The debate sur­round­ing the use of arti­fi­cial intel­li­gence (AI) in aca­d­e­mic exam­i­na­tions is becom­ing increas­ingly impor­tant. It is feared that unsu­per­vised writ­ten work is more sus­cep­ti­ble to cheat­ing and there­fore can­not be retained as a form of exam­i­na­tion. In par­tic­u­lar, the ques­tion of whether and how AI-sup­ported per­for­mance can be rec­og­nized or sanc­tioned is of con­cern to uni­ver­si­ties and courts alike.

The Admin­is­tra­tive Court of Munich, for exam­ple, ruled on two appli­ca­tions(appli­ca­tion 1, appli­ca­tion 2) from rejected appli­cants to the Mas­ter’s degree pro­gram for pro­vi­sional admis­sion to the pro­gram. They are said to have sub­mit­ted an essay for their appli­ca­tion that was allegedly gen­er­ated using AI. Both appli­ca­tions were rejected by the court on the basis of prima facie evi­dence of decep­tion.

Accord­ing to the prin­ci­ple of per­for­mance under higher edu­ca­tion law, exam­i­na­tions must be com­pleted inde­pen­dently and with­out unau­tho­rized aids. Can­di­dates usu­ally sub­mit a cor­re­spond­ing dec­la­ra­tion of inde­pen­dence. Accord­ing to the court, if this includes an explicit waiver of the use of AI, this con­sti­tutes a breach of reg­u­la­tions if the work was cre­ated in whole or in part using AI. So-called AI detec­tors can be used to deter­mine this. These are tools that ana­lyze con­tent to deter­mine whether it was gen­er­ated by an AI. How­ever, the results of AI detec­tors alone are not a deci­sive indi­ca­tion. An inde­pen­dent opin­ion from human review­ers is also required, the court con­tin­ued.

The reliability of AI detectors

The court appar­ently assumed that the cur­rently avail­able detec­tors are nei­ther accu­rate nor reli­able. In one study, 14 detec­tors were tested. It was found that the detec­tors tended to clas­sify the ana­lyzed texts as writ­ten by humans rather than rec­og­niz­ing them as gen­er­ated text. For texts that were cre­ated by an AI and then revised, the accu­racy is said to be only 50 per­cent. An AI-gen­er­ated text that has been slightly revised by an exam­i­nee may no longer be reli­ably rec­og­nized by the detec­tors.

Due to these uncer­tain­ties, proof of decep­tion should be based on a human analy­sis of typ­i­cal AI char­ac­ter­is­tics in addi­tion to the use of detec­tors. These include texts with per­fect form, with­out spelling, punc­tu­a­tion or gram­mat­i­cal errors, as well as typ­i­cal AI errors such as hal­lu­ci­na­tions, exag­ger­a­tions or inac­cu­rate ref­er­ences in quo­ta­tions. Fur­ther infor­ma­tion and tips on this can be found in the arti­cle “Rec­og­niz­ing AI text in exam­i­na­tion papers” by Matthis Kepser.

If you would like to learn more about the con­nec­tion between AI and OER, we would like to invite you to our next work­shop “AI and OER in use”. We will inform you about the legal aspects of cre­at­ing open edu­ca­tional mate­ri­als with the help of AI. We will also show you how AI can help to make OER more effec­tive and reduce the work­load. You are wel­come to reg­is­ter at support.twillo@tib.eu to reg­is­ter.

Download

Sam­ple tem­plate as PDF (Adobe Reader), DocX (Microsoft) or Open­Doc­u­ment (ODT) down­load.