Knowledge cutoff is a poor proxy for model capability

Summary

Microsoft researchers tested hundreds of product changes and found a model's knowledge cutoff date predicts its performance poorly, since models can fail on older features and succeed on newer ones.

Key facts

• Microsoft tested hundreds of product changes for the study.
• Models can fail on features released before their knowledge cutoff.
• Models can succeed on features released after their knowledge cutoff.
• The findings were published on 2026-09-21.

Why it matters

Teams choosing or evaluating models should not rely on knowledge cutoff dates as a capability signal, since the date reveals less about performance than the actual work does.