Welcome, humans.

Get ready to hear “AI Bowl” a lot this weekend, as this Super Bowl Sunday is set to include an onslaught of AI ads—with 30-second spots going for a cool $8M each.

Meta is showing off its AI-powered Ray-Bans, Salesforce and GoDaddy are promoting their AI tools, and Google had a major ad queued up for Workspace, but had to remake it because its cheese fact was wrong.

Apparently, ppl don’t love Gouda as much as Gemini (or blogs indexed by Gemini) thought.

Microsoft, Waymo, and even *~Gasp~* DeepSeek could air their own ads, too…

Most importantly, be on the lookout for one from OpenAI, who will be airing its first ever Super Bowl Ad.

Who’s behind this move? None other than Kate Rouch, the company’s new CMO and the ex-CMO of Coinbase back when it launched its own viral Super Bowl ad during The Crypto Bowl of 2022.

Sam’s already cryptically teasing the spot (x.com) with emoji riddles about a “Superb Owl.” Soo… should we expect a big announcement… or just another QR code?

As we’ve said before, Super Bowls (and other major sporting events) are great measures for where we are in the hype-cycle… let’s just hope 2025 doesn’t end the same way as 2022…

Here’s what you need to know about AI today:

• We break down the top 13 insights from Andrej Karpathy’s epic AI explainer.

• Amazon budgeted $100B for AI in 2025.

• Anthropic lost a key hire from OpenAI.

• Researchers built a $50 reasoning model in 26 minutes.

Here’s the top 13 insights from Andrej Karpathy’s epic, 3-hour AI explainer…

Yesterday, we shared Andrej Karpathy's mind-blowing 3+ hour video breaking down how ChatGPT and other AI models work—from digesting internet data to “reasoning.”

Why should you care? Well, Karpathy isn't just any AI researcher. He was employee #4 at OpenAI, ran Tesla's AI division, and is known for making complex AI concepts digestible for normal humans (including you, Mom).

Want to learn all about AI, but don’t have 3 hours to kill? Here are 10 key insights:

1. Pre-training Data (00:01:00): All AI requires training data, and they get this data from the internet; this internet data goes through extensive filtering—removing spam, adult content, and personal information—resulting in ~44 terabytes of high-quality text the models learn from.

2. Tokenization (00:07:47): Language models break text into “tokens” rather than characters. You can think of them like puzzle pieces for words. GPT-4 uses ~100,277 of these pieces to put together its responses.

3. Neural Network I/O (00:14:27): Models predict one token at a time, based on previous tokens, with a finite amount of computation possible for each prediction.

4. GPT-2 Training (00:31:09): Training costs plummeted from $40K (2019) to as low as ~$100 today thanks to optimization. Fun fact: Karpathy trains a GPT-2 model live during the video. Wild!

5. Base Model Inference (00:42:52): Base models come first, and act as “internet simulators,” generating text that matches patterns they've seen—but they aren't assistants.

6. Post-training Data (01:01:06): Models learn to be assistants through what’s called “supervised fine-tuning”, using conversation data that’s both human-created and synthetically generated.

7. Knowledge & Memory (01:20:32): Language models have two types of memory: parameters (like vague childhood memories) and the context window (like having a book open right in front of you). This is also why they hallucinate…it’s like when you’re pretty sure you remember something, but you’re actually mixing up different memories.

8. Model Thinking (01:46:56): Models need multiple tokens to process complex thoughts—they can't solve difficult problems in a single step—hence the effectiveness of “chain of thought” prompting (“go step by step”).

9. DeepSeek-R1 (02:27:47): Karpathy demonstrates how advanced models (like o3, Flash Thinking, and of course R1) do “explicit reasoning”, showing their work and checking answers from multiple angles.

10. RLHF (02:48:26): Reinforcement learning from human feedback (RLHF) helps improve model outputs, but has limitations since it relies on simulated, rather than real, human feedback.

Here's what Karpathy says we should expect next: 

• Language models are evolving toward true multimodality—seamlessly handling text, audio, and images together.

• They'll also become more independent, handling longer-running tasks (like Operator does) with human oversight.

• The challenge now lies in developing test-time training so models can learn from new experiences like humans do, especially when dealing with extremely long contexts in multimodal interactions.

Karpathy emphasizes treating AI as a powerful but imperfect tool—great for inspiration and first drafts, but still requires human verification and ownership of the final product.

FROM OUR PARTNERS

💼 Want to become an AI Consultant? 

This stat is WILD: The AI consulting market is projected to explode from $6.9 billion to $54.7 billion by 2032. Want to capture this opportunity?

So how do you, an AI enthusiast, become a handsomely compensated AI consultant?

Well, The AI Consultancy Project from Innovating with AI has already helped 300+ students build successful consulting businesses. 

Current students love:

• Tools and frameworks to find and serve clients.

• 6-month roadmap to building a 6-figure business.

• Methods to land your first client in as little as 3 days.

As a Neuron reader, you get early access to the next enrollment cycle!

Take advantage of the AI boom—request early access to The AI Consultancy Project ASAP!

Prompt Tip of the Day

Want to master o1 and o3-mini? Microsoft's latest research says instead of the usual GPT tricks (examples, step-by-step guides, chain-of-thought), these models need the opposite approach: crystal-clear context and minimal instructions.

Give them:

1. The essential facts/context they need (they don't know everything!).

2. A clear statement of what you want.

3. Your desired output format.

Then, step back and let their built-in reasoning engine work. These models crush complex tasks when you keep prompts clean and skip the usual GPT engineering. Simple = smarter!

Speaking of smarter, we had o1-pro synthesize all that advice and build a custom prompt template out of it (pastebin version)—enjoy!

Treats To Try.

1. Mistral, our favorite french AI, just launched a new app for Le Chat (iOS, Android) along with a new Pro Tier for $14.99/month for the highest performing model (there’s a free version too—read more here).

2. Screen Studio turns your screen recordings into polished videos with automatic zooms and easy sharing.

3. Ask Concierge lets you control all your work apps through chat, so you can say “find my last email with Sally” or “create a ticket for that bug” and it happens.

4. Future You from MIT connects you with your future self for chats that help you make better life decisions—watch this, then try it here.

5. Pickle creates a realistic digital double of you for video calls—this is literally what we joked about in our recent deep fake piece (demo).

6. Matle (pronounced mate-el) is like Wordle, but for chess—find the missing chess pieces to figure out who is check-mating the other player.

See our top 51 AI Tools for Business here!

Around the Horn.

• Amazon will spend $100B on capex in 2025 “to support demand for our AI services.”

• Google plans to release commercial quantum applications “within five years”, while NVIDIA thinks it’ll be more like 20.

• John Schulman and OpenAI cofounder who left for Anthropic, is now leaving Anthropic for Mira Murati (ex OpenAI CTO)’s new startup.

• Our fave AI Super Bowl ad that’s gone live already is this one for Gemini—raise your hand if that’ll be you this weekend. Same.

FROM OUR PARTNERS

The future of presentations, powered by AI

Gamma is a modern alternative to slides, powered by AI. Create beautiful and engaging presentations in minutes. Try it free today.

Intelligent Insights

• The Algorithmic Bridge argues as AI gets more powerful (like OpenAI's Deep Research), the gap between basic and expert users actually grows larger—meaning your ability to craft good prompts could be the difference between getting mediocre results and PhD-level analysis.

• Here’s a fantastic article from Ethan Mollick about what deep research means for the world.

• This visual guide to reasoning models is great for understanding how they work.

• Stanford + University of Washington researchers built an OpenAI competitor for just $50 and in ~26 minutes by creatively using “wait” commands to improve the model's reasoning (paper)—they distilled it from Gemini 2.0 Flash Thinking, btw. Nobody tell the market, you saw what they did after DeepSeek!

A Cat's Commentary.

In partnership with