Effectiveness of AI and traditional proctoring: A comparative study
What did the popular Bollywood character Munna Bhai do when he wanted to clear the medical entrance? He got Dr. Rustom Pawri, a qualified doctor, to impersonate him! While we cheered Munna on screen, in real life, cheating in exams is no laughing matter. Every day, we see reports of cheating across many academic and competitive exams. So, why do people cheat without fear, even at the highest levels?
Let us quickly look at how traditional proctoring works. A batch of students write exams with one or two people monitoring. It is humanly impossible to monitor every candidate continuously to ensure that there is no cheating. Asides and chits are commonplace; other forms of cheating, like impersonations, also occur.
So, will moving exams online help? Not if you go by available data. While online exams brought many benefits in terms of cost, scheduling, etc., cheating persisted. A HirePro report based on a sample size of nine lakh assessments found:
- Cheating grows by 80% with no proctoring.
- The most common form of cheating was someone sitting with the candidate to provide verbal help, followed by impersonation.
- 30–50% of people cheat at entry-level job assessments.
When you look at more data from this report to find a solution, you learn that proctoring via audio + video + image helps detect 98 per cent of cheating. Therefore, AI proctoring misses only a minimal 2 per cent of cheating instances. However, one needs to implement maximum caution as the report highlights that when the proctoring included only an audio and image combination, 58 per cent of instances of cheating missed detection.
Let us understand AI proctoring better to see how it helps.
What is AI proctoring?
AI proctoring is AI-powered invigilation during virtual exams to detect various forms of cheating. Unlike human proctoring, there is a constant focus on every candidate to monitor their activities and the environment, enhancing the integrity of the tests. Environment monitoring is crucial because candidates can search online, switch browser tabs and even impersonate people. With AI, it is like having a pair of laser eyes on every candidate to ensure that the test results are fair and square.
There are three types of AI proctoring, namely:
- Live proctoring:Human proctors monitor live sessions backed by AI algorithms to verify and take requisite actions as required.
- Recorded proctoring:You record complete sessions for audit. It helps verify the results with data pointers to back any anomalies detected.
- Automated proctoring:This option facilitates auto-detection of any suspicious behaviour. It raises an alarm when it detects any form of cheating, including impersonation and cross-conversations.
The significant advantage here is the technology, which has features such as facial recognition, two-factor authentication, environment surveillance, browser monitoring, reports, etc.
Here is a comparison of the effectiveness of both methods of proctoring.
Measuring the effectiveness of proctoring
You can measure effectiveness across four key parameters.
-
Deterrence
Traditional proctoring is reactive, whereas AI is proactive. Also, the AI algorithm keeps learning from various patterns to improve on the go. Knowing that AI can continuously watch and detect more and more issues, unlike human proctors, can deter people from cheating. In the long run, it can reduce instances of cheating, making AI proctoring highly effective in deterrence.
-
Scale
Human monitoring mostly errs with volumes, whether offline or online. A human cannot keep watching multiple screens with equal effectiveness. A simple example is the CCTV monitoring rooms in many complexes, where people are not highly attentive while peering at the screens. Also, humans need breaks! All these aspects lead to errors, more so with scale. AI, on the other hand, can ensure the same effectiveness even for large volumes.
-
Logistics
Exam logistics can be challenging, especially when done at scale. People allocation, travel, scheduling, etc., can be tedious and expensive. With AI, scheduling becomes hassle-free, with the added benefit of savings on personnel and expenses. Evaluation is instant and is backed by reports, lending transparency and accuracy and helping avoid any meddling by humans. AI proctoring, therefore, is heads above the traditional mode in this aspect.
-
User experiences
AI is more inclusive and accessible, especially for PwDs. It is less intimidating than human proctors and can accommodate options like adaptive assessments, allowing people to perform better. These aspects help gauge candidates holistically and make for better user experiences.
The effectiveness of AI proctoring across these four parameters seems to be much better when compared to traditional methods. But for best results and as a control measure, AI proctoring must always be backed by human assistance. While AI proctoring will evolve continuously, we cannot completely rule out human involvement in proctoring.