Which tool
gets called?
CLM and Jev received the same 1,253 public BFCL v4 requests and each chose one function. The answer depends sharply on how the candidate functions are written.
A public test of one agent decision.
The Berkeley Function Calling Leaderboard includes requests where a model must select among several functions. This test used every case in its multiple and live_multiple sets, retained the official request and candidate order, and checked the chosen function name against the official answer.
The official BFCL leaderboard also checks generated arguments and other categories. This adaptation measures routing only, so its percentages are not BFCL leaderboard scores.
The paired run.
Loading results...
The description and raw JSON views were planned first. The prose view was added after CLM's raw JSON result was inspected; it is an exploratory check of formatting.
| VIEW | CLM CORRECT | CLM ACCURACY | CLM P50 | JEV CORRECT | JEV ACCURACY | JEV P50 |
|---|
How presentation changes selection
Bars show top-1 function-name accuracy. Each view uses the same cases and gold names; only candidate wording changes.
Tool choice is sensitive to the interface.
Loading the paired results...
This result applies to the CLM base head used here. It does not measure a separately trained BFCL head or a full agent that validates and executes calls.
How the team's chart compares.
The CLM team's published "BFCL v4 tool calling" chart reports 95.2% for CLM and 99.2% for Jev. The public release does not specify the case list, Choice prompt adapter, or scorer used for that chart. This public-case, function-name-only test is therefore a related check, not an exact reproduction.
| RESULT | CLM | JEV | WHAT IT MEASURES |
|---|---|---|---|
| TEAM CHART | 95.2% | 99.2% | "BFCL V4 TOOL CALLING"; DETAILS UNPUBLISHED |
| DESCRIPTION VIEW | - | - | FUNCTION NAME ONLY; NAME + DESCRIPTION OPTIONS |
Do not read the difference between these rows as a model regression: the test protocol and possibly the cases differ. The team's latency numbers also use an unspecified measurement path.
Inspect every decision.
What changed between views.
Same question, same answer key
Each view uses the same 1,253 requests and official gold function names. Both models receive identical typed Choice requests within a view. The options stay in the dataset's order.
Three ways to present options
Descriptions use the function name and purpose. Raw schemas include the full function object as JSON. The post-hoc prose view keeps parameter details but writes them as readable text.
CLM ran on a local GPU pod with its public v0.1-8B head; Jev used TypeSafe's hosted API. Client response time includes these different network paths and is not a pure compute-speed comparison. The results and runner are pinned to the public BFCL data revision shown below.