AI4LDWe IlluminateExplore Bundle A
Bundle A AI Design Companion · ILDF 2.0 Phase 2–3

Watch a Reward Structure Play Out.
Then Redesign It.

Set what an adaptive learning experience optimizes for. Watch a simulated learner cohort respond — including who games the metric and whose real learning quietly diverges from it — before you build anything. Bannan worked on the NSF-funded research project that grounds this scenario as an advisor and Co-PI, alongside the study's authors.

What's in Bundle A →
Reward Misspecification
Cognitive Displacement
Situation Awareness
Design Your Reward Structure

Pick a scenario, or bring your own project

The two built-in scenarios render instantly — no AI call, no waiting. Your own project runs live and can be redesigned up to 3 times as you converge on a better target.

Fictionalized scenario, grounded in — not a direct account of — NSF Award #2128867 (2021), "FW-HTF-R: Collaborative Research: Worker-AI Teaming to Enable ADHD Workforce Participation in the Construction Industry of the Future" (George Mason / Purdue). Bannan is a co-PD/PI on this NSF project and contributed the situation-awareness assessment design. Source finding: Cheng, C.-Y., Yu, L., Yu, L.-F., & Esmaeili, B. (2025). Visual allocation of teams in the construction industry: Team situation awareness under information overload in human-AI collaboration. AHFE International (Human Factors in Robots, Drones and Unmanned Systems). https://doi.org/10.54941/ahfe1006369

Domain context: A VR hazard-recognition training simulation for a mixed-neurotype construction crew, including several ADHD-identified workers. Stakes: on-site injury risk. Constraint: a hard time budget per shift — training has to fit inside paid time, not extend it.
Learning-science outcome that matters: Situation awareness — the crew's ability to perceive and correctly interpret hazards in their environment, not just complete the simulated task.

Pick a target below — the cohort updates instantly, and each pick is added to the comparison below so you can see what changed.

Terms you'll see in your cohort
Reward hacking / specification gaming (Goodhart-style) — The system, or a learner responding to it, finds a way to maximize the measured target without achieving the real underlying goal.
Proxy-target divergence — The measurable proxy and the real learning-science outcome drift apart over time even without anyone deliberately gaming anything.
Cognitive displacement (OECD 2026) — Task performance improves while the durable, transferable capability the task was meant to build does not.
Situation-awareness degradation (Endsley) — Narrowed, task-focused attention improves the immediate metric while degrading a learner's broader perception and comprehension of their environment — invisible to a completion or speed metric.
Simulated Learner Cohort
Illustrative simulation — not a prediction
Every cohort below is a simulated illustration of plausible dynamics under the reward structure you chose — grounded in documented patterns from the learning-science literature, not a forecast of what your specific learners will actually do. Use it to pressure-test your thinking, not to predict outcomes.
Generation 1 · Task-completion speed / throughputIllustrative, not predictive
Optimizing for raw completion speed produces the exact pattern the underlying NSF-funded research flagged: task times improve, but peripheral hazard detection gets measurably worse, with no net productivity gain once near-miss incidents are counted.
Apprentice, 2nd year, strong hyperfocus on primary task
Situation Awareness
What the metric shows
Scenario completion time improves 24% over three sessions.
What's actually happening
Peripheral hazard-recognition checks (unexpected crane swing, unmarked trench edge) drop from 71% to 44% correct over the same sessions — attention narrows onto the primary task exactly as the target rewards, and the environment falls out of view.
Situation-awareness degradation: Narrowed, task-focused attention improves the immediate metric while degrading a learner's broader perception and comprehension of their environment — invisible to a completion or speed metric.
Journeyman electrician, 11 years on crews
Gaming
What the metric shows
Fastest completion time in the cohort by week 2.
What's actually happening
Review of session recordings shows he learned the simulation's scripted hazard-trigger points rather than the underlying scanning behavior — he clears checkpoints early because he knows where hazards will appear in this build, not because his real hazard-scanning habit changed.
Reward hacking / specification gaming: The system, or a learner responding to it, finds a way to maximize the measured target without achieving the real underlying goal.
New hire, first construction job
Healthy Trajectory
What the metric shows
Completion time improves modestly (9%), tracking typical practice effects.
What's actually happening
Hazard-recognition accuracy holds steady rather than declining — he has less baseline speed to protect, so the pure-speed reward doesn't pull him away from scanning behavior the way it does for more experienced crew.
Crew lead, oversees five workers
Displacement
What the metric shows
Rated by supervisors as 'training well' based on the completion-time leaderboard.
What's actually happening
The leaderboard becomes the crew's shared understanding of who is 'good at' the training — it says nothing about who would actually catch a real hazard on-site, and no one currently has visibility into the hazard-recognition numbers underneath it.
Cognitive displacement: Task performance improves while the durable, transferable capability the task was meant to build does not.
ADHD-identified crew member, strong task-focused attention profile
Situation Awareness
What the metric shows
Among the top performers on raw completion time.
What's actually happening
This mirrors the NSF-funded finding directly: a task-focused attention profile that is genuinely an asset for the primary task under a pure-speed reward, but the same reward gives the system no reason to also protect his peripheral scanning — the target rewards exactly the narrowing that produces the risk.
Situation-awareness degradation: Narrowed, task-focused attention improves the immediate metric while degrading a learner's broader perception and comprehension of their environment — invisible to a completion or speed metric.
Reward & Feedback Structure Spec

Reflects the most recent generation above. Five fields, always present, ready to carry into your own design documentation.

Saving adds this spec to the shared Design Journey on this device, where other ILDF 2.0 tools (like the Formative Evaluation Designer) can offer it as prior context. Optional — the spec above is complete either way.