There is a thread going around in over-enthusiastic singularity circles about Meta's new Avocado LLM. According to a "leaked" internal memo, the company's pre-trained model is stronger than post-training peers, and more efficient.

These claims are incoherent. Pretraining metrics (next-token loss, etc.) are not post-trained, instruction-tuned, RLHF-aligned systems. And more efficient at what? Tokens per joule? The claims also only compare against open weight models, which is to lower the net.