01 What happened
Meta describes an internal system that turns idle inference capacity into better ad recommendations, enabled by two elasticity levers: Rank More Ads (RMA) and Serve Better Models (SBM).
02 Key details
- An internal system expands ad recommendation work by using opportunistic capacity while protecting baseline latency and reliability.
- RMA increases the number of candidates evaluated by ranking stages when capacity is available.
- SBM routes more requests to a more powerful model, using accelerator surplus.
- SBM isolation is physical: baseline and powerful models use separate routing targets to protect baseline serving.
03 Why it matters
This addresses the capacity paradox: global constraint with local idle pockets caused by peak provisioning, timing gaps in model delivery, regional resource mix, and accelerator diversity.
04 Who it matters to
Machine learning engineers, infrastructure engineers, high-performance computing specialists and devOps specialists.
Original source