01 What happened
Cohere has introduced CALIBER, a method that trains reasoning models to estimate their confidence both before and after reasoning. This improves the reliability of confidence estimates but does not yet enable decision-making based on those updates.
02 Key details
- CALIBER trains a reasoning model to output two confidence estimates – before reasoning and after the answer – and reports both in a single response.
- On BigMathDigits, the 7B model reduces expected calibration error by 52.5% to 0.028, surpassing the best single-confidence baseline.
- Without further training, the 7B model cuts calibration error on GPQA (−26.8%), TriviaQA (−32.2%), and SimpleQA (−36.2%).
- Cohere has not tested whether the confidence updates help a model decide when to verify or abstain; CALIBER only provides the estimates.
03 Why it matters
Reliable confidence estimates help users decide whether to trust a model's output or verify it, which is critical for deploying AI in high-stakes settings.
04 Who it matters to
AI developers, researchers and machine learning engineers.
Original source