Need help?
coding agent, world agent, world model, game, game agent

Human in the Loop: How to Judge What AI Makes

coding agent, world agent, world model, game, game agent

We bring the generator–evaluator pattern into the collaboration framework: a person sets intent and boundaries, a generator unfolds candidates, an evaluator checks them with rules, tools and models, and human judgement and authorisation of action are brought in as the task requires.

Model judges have biases that need calibration.[2] Review should therefore combine independent evidence, human judgement and feedback from real tasks, covering three levels:

LevelCore questionBasis for review
OutputIs the result correct, relevant and within constraints?Sources, tests and inspection of deliverables
TrajectoryCan key assumptions, operations and tool calls be verified?Observable records, data sources and test results
AlignmentDoes it answer the person's intent and real situation?Human review, authorisation and task feedback

Human participation runs through goals, exploration, review and revision. A system should preserve people's ability to step in and correct it, and keep responsibility in design, deployment and use clearly arranged.

Read the full LUMI research →

Featured Articles

How Dreams Happen: What Your Brain Does While You Sleep, coding agent, world agent, world model, game, game agent

dreams

How Dreams Happen: What Your Brain Does While You Sleep

Where Dreams Come From: Memory, Emotion and the Night Shift, coding agent, world agent, world model, game, game agent

dreams

Where Dreams Come From: Memory, Emotion and the Night Shift

Dreams and Inspiration: How to Catch the Ideas Your Sleep Makes, coding agent, world agent, world model, game, game agent

dreams

Dreams and Inspiration: How to Catch the Ideas Your Sleep Makes

Creation Begins with Connection, coding agent, world agent, world model, game, game agent

creativity

Creation Begins with Connection