Why small language models fail at rare tasks

Small models forget rare tasks because frequent ones constantly overwrite learned skills during training

Published 9sem1 sourceNotable

≈ 20sSpeed

The fact

Researchers tested models from 4M to 4B parameters to identify this mechanism

Increasing target task frequency in training data may be sufficient—no need to scale up model size

Click the link to read an article on the topic:

Explore this topic

Auto-synthesis from 1 media source · identified on June 7, 2026