An AI can draft forty test items in the time it takes to write four. That speed is real, and so are the errors. They are not random. AI-written items fail in the same few ways again and again, which means a teacher who knows the patterns can review a generated test in a fraction of the time it would take to write one, and catch nearly everything.
This guide lists those failure patterns with examples and gives a review routine to run before any generated test reaches learners. Under DepEd Order No. 15, s. 2026, AI may help draft test items, but summative tests and term exams remain teacher-made and based on the lessons taught, and the teacher is responsible for them.
The 10-second answer: Check every generated item for a wrong or double key, clues to the answer, level drift below the competency, content you did not teach, weak distractors, and unfamiliar context. Then run an item analysis after the test and fix what the data flags.
The eight failure patterns
1. The wrong key, or two right answers
The most serious error. The marked answer is wrong, or a second option is also defensible. It happens most with “best answer” items and with calculations. Check: answer every item yourself before looking at the key.
2. Clues that give the answer away
The correct option is noticeably longer or more precise; it repeats a word from the stem; the grammar of the stem fits only one option; or one item answers another. Check: read the options without the stem. If one stands out, rewrite.
3. Level drift
Asked for items on “analysing” a text, the AI writes “identify” items, because recall items are easier to generate. The test ends up measuring less than the competency demands. Check: compare each item’s verb with your table of specifications and the Bloom’s question stems for that level.
4. Content you did not teach
AI draws on everything it has seen, so it happily includes a term, an example, or a subtopic your class never met. DO 15 is clear that tests are based on the lessons actually taught. Check: for each item, point to the lesson where it was taught. If you cannot, cut it.
5. Distractors nobody would choose
Options that are obviously wrong, joke answers, or options from a different topic. An item with one plausible option is effectively a free point. Check: good distractors come from real mistakes learners make. Replace weak ones with the errors you see in class.
6. Unfamiliar or foreign context
Word problems priced in dollars, examples set in snow, names and situations your learners do not recognise. It adds reading load that has nothing to do with the competency. Check: localise the context, or strip it.
7. Vague or trick wording
Negatives in the stem (“Which is NOT…”) without emphasis, double negatives, “all of the above” used as a shortcut, and absolute words that make an option wrong on a technicality. Check: read each stem aloud. If you pause to parse it, rewrite it.
8. Confident factual errors
A wrong date, a misattributed quote, an outdated figure, or a policy detail that changed. AI states these as confidently as correct facts. Check: verify any fact you would not stake your name on, especially dates, names, figures, and anything about DepEd orders.
A ten-minute review routine
- Take the test yourself, without the key. Mark any item where you hesitated.
- Compare with the key. Every disagreement is an item to fix.
- Scan the options column by column for length and wording clues.
- Tick each item against the TOS for competency and level.
- Point to the lesson for each item. Cut what you did not teach.
- Localise names, prices, and settings.
For a high-stakes test, add a quick pilot with a few learners from another section.
After the test: let the data review it too
Some problems only show up in the results. An item analysis flags items that almost everyone missed, items the strongest learners got wrong, and distractors nobody chose. A negative discrimination index is often a wrong key that slipped through. Fix those items before you reuse them, and consider KR-20 and point-biserial for a reliability check.
Where Skoolari fits
Skoolari generates assessments from your own uploaded materials, which reduces pattern 4, and its TOS-Based mode builds the test against the competencies you list, which reduces pattern 3. The Auto-Checker’s item analysis after the test catches what review missed. None of that replaces your review: read every item before it reaches learners. See the plans, and see turning a PDF module into a quiz with AI for the full workflow.
Related reading: student data privacy when using AI tools, using item analysis as accreditation evidence, and item analysis, explained in Filipino.