Quiz System¶

Overview¶

Quizzes are interactive assessments authored in edX CMS as LMS entries with _type: 'quiz'. They render inside the standard entry route (/learn/course/\[slug\]/entry/\[entrySlug\]) using a client QuizPlayer component, accept learner answers in a single submission, are graded server-side, and emit a completion signal consumed by the pagination/gating layer described in entry-pagination.md.

This document is the binding spec for Phase 8 of implementation.md and refines its rules where noted (see Divergences). Where this document and implementation.md disagree, this document wins.

Authoring via the structured-quiz widget¶

Graded quizzes are authored with the structured-quiz CMS widget (one per page). The wire shape is defined once in @open-learning-hub/widget-wire-schemas (structuredQuizContentSchema) and the LMS maps it into the server-only Quiz domain object via src/lib/cms/quiz-adapter.ts (adaptQuizFromPage). A page is classified as a quiz entry only when it carries a structured-quiz widget — QUIZ_WIDGET_TYPES in src/lib/course/entry-classifier.ts is {"structured-quiz"}. Legacy assessment widgets (form, multiple-choice, etc.) are now formative and classify their pages as lessons; they are not graded. There is no data migration — existing quiz_attempts / entry_progress rows remain valid.

Quiz Entry Shape (CMS)¶

Quiz entries are fetched from edX CMS through src/lib/cms/client.ts, adapted by src/lib/cms/quiz-adapter.ts, and validated with the Zod schemas in src/schemas/cms/quiz.ts. The CMS-authored shape is:

type Quiz = {
  _type: "quiz";
  _order: number;
  slug: string;
  title: string;
  instructions?: PortableText;
  passThreshold: number; // 0..1, fraction of points required to pass
  maxAttempts: number; // ≥1; 0 or null is rejected at validation
  displayMode?: "full_page" | "one_at_a_time"; // default "full_page"
  shuffleQuestions?: boolean; // default false
  showPerQuestionFeedback?: boolean; // default true
  questions: QuizQuestion[];
};

type QuizQuestion =
  MultipleChoiceQuestion | TrueFalseQuestion | ShortAnswerQuestion;

Question and option text is CMS-sourced and rendered as-is — i18n is the CMS's responsibility (see Internationalisation). The Zod schemas are the single source of truth for the wire shape and feed the OpenAPI registry per .cursor/rules/050 - APIs.mdc.

Question Types¶

Multiple choice¶

• multiSelect: false (default) renders radios; the learner picks exactly one option. Submission is invalid if zero options are selected.
• multiSelect: true renders checkboxes; the answer is the set of selected option ids and is correct only when it equals the set of options where correct === true. Partial credit is not awarded.

True/false¶

A degenerate two-option multiple choice. Modelled separately so authors can express boolean prompts without ceremony and so the UI can render a tighter affordance.

Short answer¶

• trim defaults to true; caseSensitive defaults to false.
• The learner's answer is matched against acceptedAnswers[] after applying the same normalisation. No fuzzy match, no regex, no Levenshtein — authors must enumerate accepted variants. This is intentional: learner-trust requires the grading rule be inspectable from the CMS payload alone.
• Empty answers are graded as wrong, not skipped.

Future question types (essay, file upload, drag-and-drop) are out of scope and are not aliased onto these types.

Scoring Logic¶

Scoring is per-question, equal weight, all-or-nothing.

• Each question is worth one point. Total points = questions.length.
• A question is correct iff its grading rule above returns true.
• score is correctCount / totalCount, in [0, 1].
• The attempt is passed iff score >= quiz.passThreshold.

perQuestionFeedback is computed when quiz.showPerQuestionFeedback === true (default) and is an array of { questionId, correct: boolean }. The expected/correct answer is not echoed back — this prevents trivial answer-mining across attempts. Authors who want to reveal the right answer post-quiz must do so through CMS-authored explanation copy rendered only in review mode.

Weighted scoring, partial credit, and negative marking are explicit non-goals; revisit only when a real curriculum need surfaces.

Retry & Attempt-Limit Policy¶

• quiz.maxAttempts is authoritative, fetched fresh from edX on every submission (the client value is never trusted — see Server Actions).
• The number of attempts a learner has used is count(*) FROM quiz_attempts WHERE user_id = ? AND tenant_id = ? AND course_slug = ? AND entry_slug = ? AND submitted_at IS NOT NULL.
• Attempts in progress (submitted_at IS NULL) do not count toward the limit; abandoned attempts are not preserved.
• A new attempt is allowed iff attemptsUsed < maxAttempts and the learner has not already passed. After passing, the Retry affordance is hidden — passing is terminal for the entry's gating signal even if attempts remain.
• When attemptsUsed === maxAttempts and the learner has not passed, the entry enters review-only mode (see Review Mode).

Progress & Gating Integration¶

Quizzes write two rows on submission:

1. A row in quiz_attempts recording the full attempt (always, on every submission).
2. An upsert into entry_progress setting completed_at = NOW() iff either the attempt passed or this submission exhausts maxAttempts. This matches the rule defined in entry-pagination.md § Quizzes.

The entry_progress upsert reuses the helper that backs markEntryComplete so persistence and revalidation behaviour is identical to lessons. quiz_attempts.passed is preserved separately so analytics surfaces (admin dashboards, learner score history) can distinguish "completed via pass" from "completed via attempts exhausted".

Gating only consumes entry_progress.completed_at; it never reads quiz_attempts.

Data Model¶

CREATE TABLE quiz_attempts (
  id BIGSERIAL PRIMARY KEY,
  user_id BIGINT NOT NULL REFERENCES users(id),
  tenant_id BIGINT NOT NULL REFERENCES tenants(id),
  course_slug TEXT NOT NULL,
  entry_slug TEXT NOT NULL,
  attempt_number INT NOT NULL,        -- 1-based, sequential per (user, tenant, course, entry)
  score NUMERIC(5,4) NOT NULL,        -- [0.0000 .. 1.0000]
  passed BOOLEAN NOT NULL,
  answers_json JSONB NOT NULL,        -- learner-submitted answers as graded
  feedback_json JSONB NOT NULL,       -- per-question correct/incorrect
  started_at TIMESTAMPTZ NOT NULL,
  submitted_at TIMESTAMPTZ NOT NULL
);

CREATE INDEX quiz_attempts_lookup_idx
  ON quiz_attempts (user_id, tenant_id, course_slug, entry_slug, attempt_number DESC);

• attempt_number is computed server-side at submission as coalesce(max(attempt_number), 0) + 1 for the same (user_id, tenant_id, course_slug, entry_slug) tuple, inside the same transaction that inserts the row, so two concurrent submissions are still consistent.
• answers_json stores the submitted answers in the same shape the client posted, post-normalisation. feedback_json stores the per-question correctness vector. Storing both makes review-mode rendering a pure read.
• Kysely query helpers live under src/db/queries/quiz-attempts/ (insert, latest, stats) and every query is scoped by both user_id and tenant_id, per .cursor/rules/080 - Data & Auth Integration.mdc. Server-side grading lives in src/lib/quiz/grade.ts.

The migration follows the numbered tsx convention in src/db/migrations/ and is run by scripts/db-migrate.ts.

Server Actions¶

All quiz mutations live in src/app/learn/course/\[slug\]/entry/\[entrySlug\]/actions.ts (alongside markEntryComplete), are marked 'use server', and validate every input with Zod.

submitQuizAttempt({ courseSlug, entrySlug, answers })¶

1. Resolve session via await auth(). Reject if unauthenticated.
2. Enforce course access via requireCourseAccess(courseSlug).
3. Enforce sequential gating server-side: load the course hierarchy and learner progress map, and reject when the target entry is still locked.
4. Re-fetch the quiz from edX CMS. Never grade against the client-supplied question shape. This is the single defence against tampered prompts/options/correct-flags.
5. Validate answers against the re-fetched quiz: every question id must be present, every payload must match its question type's schema.
6. Re-check attempt-limit: refuse submission with an attemptsExhausted error if the learner has already used maxAttempts or has already passed.
7. Grade per Scoring Logic.
8. In a single transaction:
   • Insert quiz_attempts row (computing attempt_number as described in Data Model).
   • If passed === true or the new attempt_number === maxAttempts, upsert entry_progress with completed_at = NOW() via the shared helper.
9. revalidateTag('course:' + courseSlug) and revalidateTag('progress:' + userId) so the sidebar, badges, and dashboard refresh.
10. Return { score, passed, correctCount, totalCount, attemptNumber, attemptsRemaining, perQuestionFeedback? }. perQuestionFeedback is omitted when quiz.showPerQuestionFeedback === false.

The action is idempotent against double-clicks via a client-side submitting flag; it is not idempotent at the database level (each call inserts a new attempt), and must never be retried automatically on network error — the client surfaces a retry affordance the learner activates explicitly.

startQuizAttempt(...) — explicitly out of scope¶

There is no "start" server action. quiz_attempts rows are written only on submission. started_at is captured by the client at mount and posted with the submission payload. Rationale: a separate start endpoint adds a write per quiz view with no user-visible benefit; we accept that started_at is client-clock and treat it as advisory.

Client Architecture¶

QuizPlayer and QuizResults are client components ('use client') at src/components/lms/quiz-player.tsx and src/components/lms/quiz-results.tsx. They are rendered by the entry page (src/app/\[locale\]/learn/course/\[slug\]/entry/\[entrySlug\]/page.tsx), which passes only toQuizPublic(quiz) plus the learner's prior attempt.

• Form state is held in local component state (useState) keyed by question id; submission runs inside useTransition so the Submit button reflects pending state.
• One submission per render — the Submit button is disabled while the submit transition is pending.
• displayMode controls learner flow:
  • full_page renders all questions in one form (existing behavior).
  • one_at_a_time renders one question per step with Back/Next navigation and a final review step before submit.
• When quiz.shuffleQuestions is true, QuizPlayer deterministically shuffles question order and multiple-choice option order (seeded by quiz.slug + current attemptCount) so SSR and client hydration stay in sync.
• Client-side validation enforces "every question answered" before allowing submit; server still re-validates.
• On a successful response the player swaps into QuizResults (see Results UI).
• The player never reads correct flags from the CMS payload — those fields are stripped from the typed CMS response on the public path; only the server submitQuizAttempt action sees the un-stripped CMS document. Achieved by exposing two Zod schemas: Quiz (server-only, includes correct) and QuizPublic (client/public, omits it). QuizPlayer does not implement timers or autosave.

Results UI¶

After a successful submission the QuizResults view renders inside the same entry page:

• Score line: correctCount / totalCount and the percentage. Translation-keyed.
• Pass/fail banner: colour, icon, and copy keyed on passed. WCAG AA contrast in both themes.
• Per-question feedback list (if enabled): one row per question echoing the learner's answer and a correct/incorrect indicator. The correct answer is not shown (see Scoring Logic).
• Attempt history: when prior attempts exist, QuizResults renders a per-attempt history list (attempt number, score percent, pass/fail, submission timestamp) for the current learner and entry.
• Affordances:
  • Retry — visible iff passed === false and attemptsRemaining > 0. Clicking remounts QuizPlayer with a cleared form.
  • Review — visible iff the entry is in review-only mode. Renders the latest attempt as a read-only QuizResults.
  • Next entry — visible iff entry_progress.completed_at IS NOT NULL for this entry (i.e., passed or attempts exhausted). Routes to the next entry via the same prev/next helper used elsewhere.

Focus management: on submission, focus moves to the pass/fail banner so screen-reader users hear the outcome immediately. On retry, focus moves to the first question's first input.

Review Mode¶

When attemptsUsed === maxAttempts and the learner has not passed, the entry is in review-only mode:

• QuizPlayer is not rendered. Instead, QuizResults is shown for the most recent attempt (highest attempt_number).
• The CMS-authored explanation field on each question (if present) is rendered alongside the learner's answer. This is the only context where explanation copy is ever shown.
• The Submit and Retry affordances are absent. The Next-entry affordance is present (the entry is complete via attempts-exhausted).

Review mode is also the surface a learner returns to after passing — same view, but with the highest-scoring attempt as the source row.

Components & Test IDs¶

Per the AGENTS.md data-testid convention, every custom component and interactive sub-element exposes a stable selector for Playwright. Primitives from @open-learning-hub/ui are exempt.

Each component above ships with a co-located *.test.tsx, per the 100% file-coverage rule in implementation.md Phase 11.

Implementation note. The shipped data-testids use a flatter quiz-* scheme: quiz-player-<entrySlug>, quiz-question-<questionId>, quiz-option-<questionId>-<optionId>, quiz-input-<questionId>, quiz-submit, quiz-results-<slug>, quiz-result-banner, quiz-score-line, quiz-review-<questionId>, quiz-review-status-<questionId>, quiz-retry, and quiz-exhausted. Treat this list as authoritative for Playwright/RTL selectors.

Internationalisation¶

UI chrome is translation-keyed; CMS-authored question/option/explanation copy is rendered as-is. Required keys (in all six locale files — en/es/fr/de/pt/zh.json — per AGENTS.md):

• quiz.submit, quiz.retry, quiz.review, quiz.next
• quiz.score.line (with {correct} / {total} / {percent} placeholders)
• quiz.banner.passed, quiz.banner.failed, quiz.banner.exhausted
• quiz.feedback.correct, quiz.feedback.incorrect
• quiz.attempts.remaining (with {count} placeholder), quiz.attempts.exhausted
• quiz.error.unanswered, quiz.error.previewReadOnly, quiz.error.network
• quiz.review.heading, quiz.review.explanationLabel

Accessibility¶

• Radio groups and checkbox groups use <fieldset> + <legend> for the question prompt.
• Question prompts are rendered as h3 inside an h2 quiz title so the heading order is flat and predictable.
• The pass/fail banner sets role="status" and receives focus on submission; it also announces via a polite live region for SR users who navigate away from focus.
• Form errors (e.g. unanswered question on submit) are associated with their inputs via aria-describedby and surfaced in a single role="alert" summary above the Submit button.
• Submit and Retry buttons disable via aria-disabled, not the HTML disabled attribute, when the disabled reason is policy (attempts exhausted) — so SR users can still discover them. They use the HTML disabled attribute when the reason is transient (isSubmitting).
• All colour pairings in the pass/fail banner and feedback rows pass WCAG AA contrast in both light and dark themes (verified by the @axe-core/playwright sweep — see implementation.md Phase 11).

Caching & Revalidation¶

• Quiz CMS payloads are fetched through src/lib/cms/client.ts with revalidate = 60 and tagged course:<slug> and entry:<courseSlug>:<entrySlug> so admin-driven invalidations scope cleanly.
• Per-learner state — attempt count, latest attempt for review, completion badge — is read from Kysely on every request and is never cached.
• submitQuizAttempt calls revalidateTag('course:' + courseSlug) and revalidateTag('progress:' + userId) so sidebar lock states, completion badges, and the dashboard refresh.

Testing Requirements¶

Unit (Vitest, happy-dom)¶

• Scoring: all-correct, all-wrong, mixed; per-question-type grading rules; multi-select set equality (correct subset, superset, missing-one); short-answer normalisation (trim, case).
• Retry-policy resolution: zero-used / mid-used / fully-used; "passed already" hides retry; pass on the final attempt does not double-write entry_progress.
• Quiz Zod schemas: reject maxAttempts < 1, reject passThreshold outside [0,1], reject MC questions with no correct: true option, reject SA with empty acceptedAnswers.

Component (Vitest)¶

Every component listed in Components & Test IDs has a co-located .test.tsx covering rendered states (idle, submitting, passed, failed, review-only) and interaction handlers.

E2E (Playwright)¶

• Pass on first attempt: submit → pass banner → next entry unlocked → quiz_attempts row written → entry_progress.completed_at set.
• Fail-then-pass: fail attempt 1, retry, pass attempt 2; both attempts in quiz_attempts; attempt_number is monotonically increasing; entry unlocks on attempt 2.
• Attempts exhausted without passing: entry unlocks anyway; QuizPlayer is replaced by review-only QuizResults; Retry button is absent.
• Tampered submission: a request mutating an option's correct flag client-side is graded against the server-fetched quiz and produces the correct score.
• Preview mode: a course_admin for the course can render the quiz under ?preview=1 but submitQuizAttempt returns previewReadOnly and writes nothing.
• Unanswered submit: clicking Submit with one question blank surfaces an inline error and does not call the server.
• Theme + locale sweep: pass/fail banner and feedback row render in light and dark, and at least one scenario renders in es to prove translation keys are wired.

Use DATA_DIR=./data/test and unique resource names per test (per AGENTS.md test convention).

Open Questions / Deferred¶

• Per-question time limits / overall quiz timer: out of scope.
• Partial credit / weighted scoring: out of scope; revisit with curriculum-team requirements.
• Essay / file-upload questions: out of scope; require a separate grading workflow.
• "Mark incomplete" undo for quizzes: not supported. A passed quiz cannot be retaken, by design.
• Cross-attempt analytics surfaces (item-difficulty, distractor analysis): dashboard-side concern, tracked under Phase 9.

Divergences from implementation.md¶

These requirements refine implementation.md and should be propagated when that file is next updated (Phase 12):

1. Preview-mode short-circuit. submitQuizAttempt is read-only when getEntryAccess(...) returns { mode: 'preview' }. No quiz_attempts row is written. Not mentioned in implementation.md Phase 8.
2. Two-schema split (Quiz vs QuizPublic). The correct flags on options and the acceptedAnswers array on short-answer questions are stripped before the CMS payload reaches the client bundle. implementation.md does not specify the split.
3. Always-on submit gating. Quiz submissions enforce prerequisite completion server-side even when the entries.sequentialGating feature flag is disabled for sidebar navigation. This is an intentional hardening choice and is now documented here.

Related Documents¶

• implementation.md — Phase 2 (data model), Phase 4 (Zod schemas), Phase 8 (binding spec).
• analytics.md — learner and admin reporting built on quiz_attempts (pass rates, per-question averages, attempts-to-pass).
• entry-pagination.md — completion handoff, gating, preview mode.
• hierarchy-structure.md — Course / Module / Entry shape.
• user-management.md — admin roles consumed by preview-mode short-circuit.
• .cursor/rules/030 - Coding Best Practices.mdc, .cursor/rules/080 - Data & Auth Integration.mdc — server-only enforcement and multi-tenancy scoping.
• .cursor/rules/050 - APIs.mdc — Zod-to-OpenAPI registry for the public quiz schema.
• .cursor/rules/060 - WCAG.mdc — accessibility baseline referenced from this document.