Gergely Orosz, The Pragmatic Engineer, posted about AI writing in docs, and the comments lit up. Turns out developers still read and rely on docs.
It made us wonder about overall docs quality of frontier labs, so we ran their docs through our knowledge health tool.
The findings: every lab we scored turned up contradictions, inaccuracies, and stale information.
Sampling ~10% of a lab's public docs, here's how they performed from best to worst:
- Anthropic: 4 issues
- DeepSeek: 9 issues
- Mistral: 10 issues
- XAI: 11 issues
- Thinking Machines Lab: 12 issues
- OpenAI: 13 issues
- Google DeepMind(Gemini): 19 issues
- Meta: N/A, unreadable docs for agents
Documentation is notoriously hard to maintain, but imagine a world with perfect docs that are always accurate and complete. Imagine how much time and tokens we can save.