How to Write Course Assessment Questions Using ChatGPT

Before opening ChatGPT, get clear on two things: which learning outcome this assessment targets, and what level of understanding you need to verify. A nutrition course module on macronutrients might have the outcome "calculate daily protein needs for different client profiles." That's an application-level outcome — so your assessment should require calculation, not just recognition. Tell ChatGPT both the outcome and the target Bloom's level. Without this specificity, you'll get generic recall questions regardless of what the module actually teaches.

Multiple choice works best for knowledge and comprehension checks — the foundational levels where students need to demonstrate they absorbed key concepts before moving on. Ask ChatGPT to generate questions with four options, one correct answer, and plausible distractors. Plausible distractors matter: if three of the four options are obviously wrong, the question tests process of elimination, not understanding. Tell ChatGPT to make each distractor represent a common misconception or a partially correct understanding, not a random wrong answer.

Short answer questions push students into application and analysis territory. Instead of selecting from options, they construct a response — which requires deeper processing. Ask ChatGPT to generate scenario-based short answer questions where students apply the concept to a specific situation. For reflection prompts, ask ChatGPT to write questions that connect the module content to the student's own practice or experience. "How would you adapt this approach for your specific client population?" is a reflection question. "List three benefits of the approach" is not — that's recall wearing a different format.

This is where your subject matter expertise is non-negotiable. Read every question and every answer option. Check that the correct answer is actually correct — ChatGPT occasionally generates questions where the "right" answer is debatable or outright wrong, especially in nuanced domains like coaching, therapy, or health. Check that question wording is unambiguous: if two answer options could both be correct depending on interpretation, the question needs rewriting. Remove any questions that test trivia rather than meaningful understanding.

An answer key that says "B is correct" teaches nothing. Ask ChatGPT to generate explanations for why each answer is correct and, crucially, why common wrong answers are wrong. This turns the assessment into a learning moment. When a student selects the wrong answer and reads an explanation that addresses their specific misconception, they learn more from getting it wrong than they would from getting it right. This is especially valuable in self-paced courses where you're not available to explain in real time.

Group your questions by module and sequence them intentionally. Start with one or two knowledge-level questions to build confidence, move into application questions that require deeper thinking, and close with a reflection prompt. This progression mirrors how understanding actually develops: you need the facts before you can apply them, and you need to apply them before you can reflect meaningfully. On Ruzuku, you can place these as exercises within each module — students work through the questions as part of the learning flow rather than facing a disconnected test at the end.

Assessments should test what matters, not what's easy to test. ChatGPT naturally gravitates toward factual recall because recall questions have clear right-and-wrong answers — they're the easiest to generate and to grade. But if your course teaches yoga teachers how to modify poses for students with injuries, the assessment that matters isn't "name three contraindications for shoulder stand." It's "your student tells you they had rotator cuff surgery six months ago — describe how you'd modify today's sequence."

Your role is to push every assessment toward application. When ChatGPT gives you a question that tests whether students memorized a list, ask yourself: do I actually care whether they memorized this? Or do I care whether they can use it? If it's the second — and it almost always is — rewrite the question to put students in a scenario where they have to act on what they know, not just recall it.

This is backwards design applied to assessment: decide what students need to demonstrate, then build the lesson that prepares them to demonstrate it. When you write assessments after the lesson, you tend to test whatever you happened to cover. When you write them first, you teach what students need to know to succeed. Use ChatGPT to draft assessment questions for each module before you finalize the lesson content — it clarifies your teaching priorities.

A module with nothing but multiple choice questions feels like a standardized test. A module with nothing but reflection questions feels like a journal. Combine two or three multiple choice questions for quick knowledge checks, one short answer question for application, and one reflection prompt. The variety keeps students engaged and gives you a fuller picture of their understanding.

When you build answer explanations, spend more time on why wrong answers are wrong than on why right answers are right. Students who got the question right don't need the explanation. Students who got it wrong need to understand the specific misconception that led them astray. ChatGPT generates decent first-draft explanations, but you'll want to refine them based on the actual mistakes you see your students make.

Ask ChatGPT for ten questions about any topic and seven or eight will test whether students can identify, define, or list something. These questions are easy to write and easy to grade, which is exactly why they're overrepresented. You need to specifically and repeatedly prompt for higher-order questions — and even then, review what comes back to ensure it's genuinely testing application rather than recall dressed up in a scenario wrapper.

ChatGPT tends to generate options that hinge on a single word difference, or questions where two answers are technically correct but one is "more correct." Trick questions don't test understanding; they test attention to fine print. If a student knows the material but gets tripped up by ambiguous wording, your assessment failed, not the student. Remove any question where the difficulty comes from the phrasing rather than the concept.

A question about coaching will feature a generic "client" with a generic "challenge." Your students are yoga teachers working with postpartum clients, or health coaches serving people with autoimmune conditions, or dog trainers dealing with leash reactivity. Replace ChatGPT's generic scenarios with ones drawn from your actual teaching context. The more specific the scenario, the more useful the assessment.