| |
|
| |
Join the Snorkel AI Reading Group, a recurring forum to explore the latest frontier developments in AI while building meaningful connections within the community.
In this afternoon's session, Nicholas Roberts will present his recent paper, Train-to-Test (T) Scaling Laws: Test-Time Scaling Makes Overtraining Compute-Optimal, work that will be featured at COLM 2026 (arXiv preprint available here).
Agenda:
4 pm - doors open
4:30 pm - talk begins
Boba tea & other refreshments will be provided !
Among other things, you'll learn:
Why pretraining scaling laws like Chinchilla don't account for test-time compute, & the trade-off that creates once inference cost scales with model size & sample count.
How Train-to-Test (T) scaling laws modernize pretraining scaling laws with pass@k modeling, jointly optimizing model size, training tokens, & inference samples under a fixed end-to-end budget.
Why the forecasts hold up across distinct modeling approaches, both the joint scaling effect on task loss & the impact on task accuracy.
Across eight downstream tasks, why optimal pre-training decisions shift radically into the overtraining regime, well outside the range of standard pre-training scaling suites.
How the team validated this by pre-training heavily overtrained models in the region T forecasts, confirming stronger performance, with the findings holding even after post-training.This work will be featured at COLM 2026, & was covered by VentureBeat: Train-to-test scaling explained.
This work will be featured at COLM 2026, & was covered by VentureBeat: Train-to-test scaling explained.
This work will be featured at COLM 2026, & was covered by VentureBeat: Train-to-test scaling explained.
|
|
|
|
|
|
|
|
