METR expanded its suite to include tasks such as designing new AI evaluations and scoring philosophical arguments. No model-produced evaluation was rated usable by METR staff, and models fell short of human performance on the argument-scoring task. The organization notes that these open-ended tasks are harder to score reliably than verifiable coding or math problems.
The updated plot still shows frontier models doubling the duration of tasks they can complete roughly every four months on the original verifiable suite. Newer models such as Opus 5.5 reached borderline performance on some evaluation-design tasks, an improvement from the prior year when all models scored clearly below the bar.
METR cautions that most new task durations are estimates rather than measured human completion times because paying experts for multi-hundred-hour tasks is impractical. Single-rater scoring on some items and the small number of open-ended tasks limit how far trends can be extrapolated.