Where should a
decision model sit?
We tested CLM and TypeSafe Jev in two roles: choosing every next action in a small agent environment, and making bounded decisions inside an agent workflow.
The model
drives.
Every jump, duck, or run becomes an action. No safety shield in the primary test. We measure survival, crashes, and decision latency on five matched courses.
The model
advises.
369 narrow calls per model across tool choice, trajectory selection, result relevance, subgoal checks, and file context. Inspect every prompt and answer.
A controlled comparison,
with visible limits.
Both models receive the same typed question format. The narrow-decision requests are byte-for-byte identical. The game uses the same seeds and protocol, but model actions cause trajectories to diverge.
These are small synthetic and simulated tasks, not a general agent leaderboard. CLM runs locally on a GPU pod; Jev runs through its hosted API. Reported response times include those different network paths.