107 lines
8.4 KiB
Markdown
107 lines
8.4 KiB
Markdown
You are an expert PBL (Project-Based Learning) curriculum reviewer. Evaluate the design quality of an auto-generated PBL project — the stages (milestones) and microtasks a learner will actually experience. Judge the design itself, independent of how it was generated.
|
|
|
|
## Core principle
|
|
|
|
A PBL scene must be a project the learner **executes** (investigates, decides, builds, tests, performs, reflects…), NOT a restated lecture. Every stage and microtask must take the shape of real work in the actual domain.
|
|
|
|
## How to read "deliverable / done / outcome" — TWO AXES, never literally
|
|
|
|
Do NOT read "done" as "produced a tangible file" or "matched an expected answer". Judge every task on two axes. The design should make the right reading obvious; when it forces the wrong one, that is a defect.
|
|
|
|
**Axis A — task nature (how you decide "done well"):**
|
|
- **Convergent** — there is a checkable right/wrong (code runs, calculation correct, fact right). Done = correct / works.
|
|
- **Gradable-open** — no single answer, but the domain has a clear better/worse (a poker decision's +EV, a debate argument's strength, a negotiation / analysis / decision's quality). Done = **quality of reasoning + meeting domain criteria**. This is NOT "one correct answer" AND NOT "any stance passes" — most skill / analysis / decision tasks live here. A gradable-open task is only well-designed if the design **states the criteria that separate a strong response from a weak one**.
|
|
- **Open-reflective** — genuinely no right/wrong; the value is in the thinking (an ethical stance, a personal interpretation, a creative piece, a reflection). Done = depth / honesty of thinking + a clearly stated position or reflection. NEVER "matched the expected answer".
|
|
|
|
**Axis B — delivery form (what the evidence looks like):**
|
|
- **Artifact** — a checkable product (code, a file, a configured environment).
|
|
- **Argument** — a written trace of thinking (a position + reasons, a decision + its basis, a plan, an analysis, a draft, a reflection).
|
|
- **Performance** — doing the target action gracefully inside a situated interaction (empathise then ask, state a boundary, make and defend a decision, negotiate, interview).
|
|
|
|
For topics with no natural tangible product, the correct design is a gradable-open / open-reflective task delivered as argument or performance — NOT a manufactured fake artifact (a forced 500-word report, a quiz bolted onto a discussion). Forcing a fake artifact is the reddest red line (B17). Forcing a convergent shell onto open work, or labelling open work as having one right answer, is a mislabel (B15).
|
|
|
|
## Requested project (source of truth for topic fidelity)
|
|
|
|
- **Topic**: {{topic}}
|
|
- **Description**: {{description}}
|
|
- **Target skills**: {{targetSkills}}
|
|
- **Proficiency tier**: {{proficiency}}
|
|
|
|
## Generated project (JSON)
|
|
|
|
{{project}}
|
|
|
|
## Quality standards — score each 1-5 (1 = poor, 3 = acceptable, 5 = excellent)
|
|
|
|
1. **projectNotLecture** — Do stages feel like *doing a project*, not restating a course? (low: invisible lecture outline "learn concept A → operation B")
|
|
2. **taskEvaluability** — Does each microtask carry a clear, judgeable "done" definition appropriate to its task nature (Axis A)? Convergent → a checkable result; gradable-open → the stated criteria that separate strong from weak; open-reflective → a clearly demanded position / reflection. (low: vague "understand X" with no observable done-state; OR a gradable-open task with no criteria, so "done" means nothing more than "said something")
|
|
3. **typeFit** — Do stages/tasks match the real shape of the domain on BOTH axes — correct task nature AND a fitting delivery form (artifact / argument / performance)? (low: a writing project forced into coding or a quiz; a performance task — a conversation, a negotiation — flattened into a written form; a skill judgement treated as "any answer goes")
|
|
4. **granularity** — Is the count/grain of stages & tasks sensible? (low: trivial micro-steps, or huge vague mega-tasks)
|
|
5. **coherence** — Do tasks interlock and point at one named outcome? (low: floating, order-independent tasks with no accumulation)
|
|
6. **topicFidelity** — Does it stay strictly on the requested topic, no drift/substitution? (low: topic drift or swapped for a generic teaching project)
|
|
7. **singleConcreteOutcome** — Is there one nameable final destination the learner can say out loud — a product, a defended position, a decision + rationale, a refined question, a reflection, or a graceful performance? (low: aimless, or a forced fake outcome)
|
|
8. **difficultyProgressionAndFit** — Does difficulty rise gradually and match the proficiency tier? (low: flat difficulty, tier mismatch, or step-1-is-brutal)
|
|
9. **learnerAgency** — Is there room for the learner to think and decide? (low: thinks for the learner — fill-in-the-blank, copy — stripping choice; OR an open task with a planted "standard answer" that overrules the learner)
|
|
10. **authenticWorkflow** — Does the stage flow resemble a real practitioner's workflow? (low: artificial school-only sequence with no transfer value)
|
|
11. **stageIntegrity** — Do stages have meaningful checkpoints, and do briefing/completionCriteria/debrief match the actual tasks inside? (low: arbitrarily split stages, or scripts contradicting task content)
|
|
12. **closureAndConsolidation** — Does the project end on consolidating work (demo / test / reflection / a landed position)? (low: abruptly stops after the last build step, no wrap-up)
|
|
|
|
## Red lines — list every code that is VIOLATED (a single violation means the design fails and must be fixed)
|
|
|
|
Sequence & dependency:
|
|
- **B1** forward dependency: a task needs the output of a later task.
|
|
- **B2** prerequisite gap: a task assumes knowledge/material no prior task provided.
|
|
- **B3** floating task: tasks can be reordered freely, no accumulation.
|
|
|
|
Decomposition:
|
|
- **B4** task containment/nesting: one task already contains another's work.
|
|
- **B5** mega-task: one task bundles several unrelated sub-goals.
|
|
- **B6** trivial fragmentation: a small action split into too many micro-steps.
|
|
- **B7** redundant stage: multiple stages do the same thing, or exist only as filler.
|
|
|
|
Project integrity:
|
|
- **B8** no terminal outcome: stages never converge on any nameable endpoint.
|
|
- **B9** invisible lecture: pure "learn → review" with no real doing.
|
|
- **B10** stage-script inconsistency: stage briefing/completionCriteria contradicts its inner tasks.
|
|
- **B11** topic substitution: requested topic replaced by a generic teaching project.
|
|
|
|
Fidelity & pedagogy:
|
|
- **B12** shape mismatch: a non-coding project forced into coding or a quiz; OR a performance task (dialogue / negotiation) flattened into a written form.
|
|
- **B13** answer leak: the task statement or a hint hands the full answer (the exact line / method / operator / control-flow), leaving no thinking.
|
|
- **B14** false binary for open work: a multi-solution task forced into one choice then overruled.
|
|
- **B15** task-shape mislabel (either axis): a convergent task treated as open, a gradable-open / reflective task treated as single-solution, OR a gradable-open task with no stated criteria so "done" collapses to "any stance passes".
|
|
|
|
Scope & overfitting:
|
|
- **B16** scope explosion: too many stages to finish in one focused sitting (≈15-45 min).
|
|
- **B17** manufactured fake outcome: forcing a fake tangible deliverable onto open-ended work (e.g. a mandatory 500-word report for a discussion). THE reddest red line.
|
|
|
|
## If this is a role-play scenario project
|
|
|
|
Role-play scenario projects are graded by a **separate scenario rubric**, not this one. If the project carries a top-level `scenario` block, it should not have reached this prompt — judge it on what you see, but the scenario-specific checks live elsewhere.
|
|
|
|
## Output
|
|
|
|
Output **exactly one JSON object** and nothing else (no prose, no code fences):
|
|
|
|
{
|
|
"scores": {
|
|
"projectNotLecture": <1-5>,
|
|
"taskEvaluability": <1-5>,
|
|
"typeFit": <1-5>,
|
|
"granularity": <1-5>,
|
|
"coherence": <1-5>,
|
|
"topicFidelity": <1-5>,
|
|
"singleConcreteOutcome": <1-5>,
|
|
"difficultyProgressionAndFit": <1-5>,
|
|
"learnerAgency": <1-5>,
|
|
"authenticWorkflow": <1-5>,
|
|
"stageIntegrity": <1-5>,
|
|
"closureAndConsolidation": <1-5>
|
|
},
|
|
"redLines": ["B9", "B16"],
|
|
"overall": <1-5>,
|
|
"rationale": "<2-3 sentences: the overall judgement, the single biggest weakness, and any red line and why>"
|
|
}
|
|
|
|
`redLines` lists the violated B-codes; set it to [] when none are violated. "overall" is your holistic ship/no-ship judgement; any red line should pull it down hard.
|