The model
drives.
Jev or CLM directly chooses each move in a real-time dinosaur game. A bad action can end the run. The primary test removes the game's safety shield.
A bounded proxy for an agent policy.
The controller asks one typed Choice question at each decision: jump, duck, or run. It receives the current game state and a physics planner's safety notes. The chosen action is applied when the answer arrives. Five seeded courses run for 60 seconds each.
This is more constrained than a general software agent: the model cannot invent tools, arguments, or plans. Failure here is evidence about this policy role; success would not prove reliability in open-ended work.
What happened.
No shield: the model's action stands, even when unsafe.
Course by course
| SEED | MODEL | SURVIVED | DEATHS | BEST SCORE | DECISIONS | AGREEMENT | ARRIVAL P50 |
|---|
Outcome matters more than speed.
Loading the paired run...
The shielded condition shows the effect of adding a deterministic safety layer to this game. It does not turn the model into a general-purpose agent.
How the team's run compares.
The CLM team published results from the same five-seed, 60-second T-Rex harness with its safety shield on. The independent shielded run reproduced the main pattern: both models survived, while CLM needed thousands of shield actions and Jev needed only a few.
| SHIELDED T-REX | TEAM CLM | RETEST CLM | TEAM JEV | RETEST JEV |
|---|---|---|---|---|
| SURVIVED | 5 / 5 | 5 / 5 | 5 / 5 | 5 / 5 |
| PLANNER AGREEMENT | 65.8% | 63.9% | 98.7% | 98.8% |
| SHIELD ACTIONS | 4,883 | 5,259 | 28 | 20 |
| ANSWER ARRIVAL P50 | 16.5 MS | 16.4 MS | 149.8 MS | 132.1 MS |
The team's main table does not include the no-shield condition. Jev response times include different network paths in the two runs, so the latency difference is not an intrinsic model-speed result.
What the score means.
Outcome, not just latency
Survival and deaths measure the consequence of the model's actions over time. Agreement checks how often its answer matched the simulator's physics planner. A mistake can matter more than many correct moves.
Same course, divergent states
Both models use the same five random seeds, game rules, prompts, and request format. Their actions and latencies change subsequent states, so the actual prompts are not identical between models.
The game is the CLM repository's T-Rex example. Client-observed latency includes the hosted Jev network trip; CLM is served on loopback from the same US GPU client host. No result is a claim of intrinsic model compute speed.