Meta Llama 4: From Benchmark Controversy to Superintelligence Labs – The Full Story

Meta's Llama 4 journey has been one of the most dramatic stories in AI this year – from a triumphant launch to benchmark scandal, leading to a complete restructuring of Meta's AI organization.

Llama 4 Model Family#

Scout (Available)#

• Parameters: 17B active / 109B total (16 MoE experts)
• Context: 10 million tokens – unprecedented for open models
• Best for: Long workflows and massive data analysis

Maverick (Available)#

• Parameters: 17B active (128 MoE experts)
• Best for: Coding, chatbots, and technical assistants
• Balance: Reasoning power vs response speed

Behemoth (Still Training)#

• Outperforms GPT-4.5 and Claude Sonnet 3.7 on STEM benchmarks
• Teacher model for distilling knowledge to smaller models
• Release date TBD

The Benchmark Controversy#

The most damaging revelation came from Meta's own chief AI scientist:

"We fudged a little bit" – Yann LeCun, admitting Meta used different model versions on different benchmarks to improve results.

Third-party researchers found their results didn't align with Meta's published benchmarks, causing:

• Loss of confidence among Meta leadership
• Internal frustration within the AI team
• CEO Mark Zuckerberg ordering an AI organization overhaul

Meta Superintelligence Labs (MSL)#

In response, Zuckerberg announced the establishment of Meta Superintelligence Labs – a new division focused on:

• Building artificial superintelligence (ASI)
• Separate governance from existing AI research
• Direct reporting to Zuckerberg
• Competing directly with OpenAI and DeepMind

LlamaCon Announcements#

At Meta's first-ever LlamaCon:

• Llama Guard 4: Safety filtering for AI outputs
• LlamaFirewall: Security layer for AI deployments
• Llama Prompt Guard 2: Protection against prompt injection
• Llama API: Limited preview combining open-source flexibility with closed-model features

Competitive Landscape#

With Qwen 3 and DeepSeek V4 as open-weight competitors, Meta is betting on:

• Democratization through open-source access
• Developer flexibility and cost advantages
• 25+ hosting partners (Nvidia, Databricks, Groq, Dell, Snowflake)