01 What happened
Cohere demonstrates that a Sol lead model with 16 Luna planning workers attains 61.8% reward on ProgramBench—close to the 63.2% of a full Sol team—while cutting inference cost by 70%, from $510 to $155.
02 Key details
- A Sol lead with 16 Luna planning workers achieves 61.8% reward on ProgramBench, versus 63.2% for an all-Sol team, at 70% less cost ($155 vs $510).
- Delegating only exploration and planning matches the performance of delegating all three stages, at half the cost.
- Planning is about twice as cost-efficient as implementation: 3.4 reward points per $100 versus 1.7.
- Scaling planning workers from 2 to 16 improved reward from 59.6% to 61.8%, while implementation and verification workers improved only up to 4 before declining.
03 Why it matters
This matters because Cohere's ProgramBench results show a Sol lead with 16 Luna planning workers reaches 61.8% reward (vs 63.2% for an all-Sol team) at 70% lower cost ($155 vs $510), though with a slightly lower point estimate. Organizations should evaluate the cost–quality tradeoff on their own tasks; the benefit is clearest when specification discovery is the bottleneck.
04 Who it matters to
Engineering teams and AI agent developers.
Original source