What Is Adaptive Testing? How to Create An Adaptive Assessment in OnlineExamMaker?¶
Adaptive testing (often called Computerized Adaptive Testing, CAT) is a type of assessment where the difficulty of questions automatically adjusts to the test taker’s ability level. Instead of giving everyone the same fixed set of questions, the system selects questions in real time based on how the examinee answered previous ones.
What is adaptive testing?¶
Adaptive testing is a method of assessment where the difficulty of questions is dynamically adjusted based on the test-taker’s performance. Unlike traditional fixed-form tests, where all examinees answer the same set of questions, adaptive testing tailors the test to the individual’s ability level in real-time. The goal is to efficiently and accurately estimate a person’s ability with fewer questions, making the process more precise and less time-consuming.
How adaptive testing works?¶
Adaptive testing relies on Item Response Theory (IRT), a psychometric framework that models the relationship between a person’s ability and their probability of answering a question correctly. The test dynamically selects questions from a pre-calibrated item bank, adjusting the difficulty based on the test-taker’s performance.
Algorithm of OnlineExamMaker adaptive testing system¶
Below is the alogrithm of OnlineExamMaker adaptive testing system:
1. Adaptive testing parameters
1) Ability Estimate (θ): Initial value is 0, after answer a question correctly/incorrectly, +0.2/–0.2 for Medium, +0.3/–0.3 for Hard, +0.1/–0.1 for Easy.
2) Standard error (SE): Initial value is 1, reduces SE by 0.04 for Medium, 0.05 for Hard, 0.02 for Easy (floor 0.1).
3) Lower 95% CI: Lower 95% CI = θ - 1.96 x SE (z - score for 95% confidence (assuming normality))
4) Upper 95% CI: Upper 95% CI =θ + 1.96 x SE (z - score for 95% confidence (assuming normality))
2. Exam organizers can configure the settings below:
Passing standard, for example: θ (Ability Estimate) = 0;
Minimum number of questions to answer, for example: 85;
Maximum number of questions to answer, for example: 150;
Exam taken time, for example: 300 minutes.
3. Exam results: Whether the candidate pass the exam
For example, if you set passing standard: θ (Ability Estimate) = 0.
When a candidate complete answering 85 questions:
1) Lower 95% CI > 0, candidate passes the exam
2) Upper 95% CI < 0，candidate fails
3) Lower 95% CI < 0 < Upper 95% CI, system continues to give the candidate questions. [**ignore the value of θ here]**
After 85th question, during 86th to 150th question:
4) Lower 95% CI > 0, the candidate passes the exam. [**ignore the value of θ here]**
5) Upper 95% CI < 0，candidate fails. [**ignore the value of θ here]**
6) If the candidate reaches 150 question: if θ ≥ 0, candidate pass, if θ < 0, candidate fails. [**ignore the value of **Lower 95% CI and Upper 95% CI here]
7) If the exam taken time (300 minutes) runs out, but the candidate doesn’t finish 85 questions, the candidate fails automatically.
How to create an adaptive assessment in OnlineExamMaker platform?¶
Step 1: Prepare questions for adaptive testing
You can create a large pool of questions with varied difficulty levels, and each question must be tagged for difficulty. In OnlineExamMaker Question Bank, we offer 3 difficulty tags, they are: easy, medium, and difficult (hard). For all the questions that you want to add to an adaptive assessment, a difficulty tag is necessary.
Step 2: Create a new exam
Go to "Exams" > "Adaptive Tests", then click "New exam" button to create a new adaptive test. You can fill exam title, select a category, and upload an exam cover image in this step.
Step 3: Select questions
Select the question categories that you want to add to the adaptive test, you can check the checkbox to select what question type you want to add to the adaptive assessment.
Step 4: Configure general settings of adaptive test
You can configure the general settings of the adaptive test, when to publish the exam, how many times a candidate can take the test, and exam language.
Step 5: Set adaptive testing answer rules
Min questions: Minimum number of questions to answer. If you set 85 here, then a candidate must answer at least 85 questions. After he answers 85 questions, the system will decide whether he passes, or fails, or has a chance to answer more questions.
Max questions: Maximum number of questions to answer. If you set 150 here, after a candidate answer 85 questions, Lower 95% CI < 0 < Upper 95% CI, system continues to give the candidate questions.
Passing standard: Whether a candidate pass the adaptive test, you can set θ (Estimate Ability) here, for example θ=0
Step 6: Share the exam link or QR code to participants
After you save the exam settings, and press "Publish" button to release the exam, you will get a direct link and QR code of the live exam. Then you can share the exam link or QR code to exam takers via email or mobile phone message.
Step 7: Analyze exam reports
After a candidate submit an adaptive test, our adaptive testing system generate the exam report immediately. The exam report shows candidate’s, pass/fail result, the number of questions answered (percentage correct), performance of each question category, and final estimate ability (θ).