đŸ˜șAI Search problems

PLUS: Our #1 prompt for editing your writing.

Welcome, humans.

OpenAI and Google released separate statements last week asking the Trump Administration to make training AI fair use (among other things) to support the “freedom to learn.”

And if this copyright issue isn’t solved? OpenAI says the AI race with China is over.

Here’s the thing, though: the AI race won’t actually be “over”—OpenAI’s ability to charge for its AI models will be.

If you can train an AI on the entire internet, that should be all good—as long as it’s free for the entire internet to benefit from.

But if you want to charge people for it? Then you gotta pay the licensing fees. And if you can’t afford to license the entire internet, then guess what? You don’t get to charge for the entire internet. We’ll see if that’s what the Supreme Court decides, anyway!

Here’s what you need to know about AI today:

  • We break down why AI citations are failing.

  • Google planned to replace Assistant with Gemini on mobile.

  • An AI angel investor made a $100K startup investment.

  • AI agent startups became hot acquisition targets.

AI search tools have a serious citation problem (and it's getting worse)

Imagine confidently citing a study that doesn't exist. That's basically what AI search tools are doing with news content rn—at alarming rates.

A Columbia Journalism Review study tested eight AI search engines to see how well they could identify and cite news articles. The results? Pretty abysmal.

Researchers fed news excerpts to tools like ChatGPT, Perplexity, Copilot, Gemini, and Grok, then asked for the original article, publisher, and URL.

Here's the shocking part: Collectively, these tools gave incorrect answers over 60% of the time. Grok 3 was wrong 94% of the time, while even Perplexity—the “best” performer—gave wrong answers to 37% of queries.

The search tools, ranked.

Even more concerning? These tools delivered errors with remarkable confidence:

  • ChatGPT incorrectly identified 134 articles, but expressed uncertainty only 15 times.

  • Most tools were more likely to provide wrong answers than admit ignorance

  • Premium models like Perplexity Pro ($20/month) and Grok 3 ($40/month) were actually more confident but less accurate than free versions.

The problem goes deeper than just getting facts wrong. Several alarming patterns emerged:

  • Bypassing website blocks: Tools ignored robots.txt files—the standard way publishers block crawlers. Perplexity identified National Geographic articles despite being explicitly blocked.

  • Fabricating links: Over half the URLs from Gemini and Grok 3 were completely made up. From 200 test queries, Grok 3 provided 154 broken links!

  • Citing syndicated content: Tools often linked to syndicated versions on sites like Yahoo News instead of original publishers—even when original publishers had licensing deals with the AI company (!!!).

What's particularly concerning is that even content licensing deals didn't guarantee accuracy. Time magazine has deals with OpenAI and Perplexity, yet neither correctly identified Time content 100% of the time.

Why this matters: When AI chatbots cite news sources, they borrow credibility from trusted publishers. As BBC News noted, “when AI assistants cite trusted brands like the BBC as a source, audiences are more likely to trust the answer—even if it's incorrect.” This harms both the publisher's reputation and misleads users.

These citations have consequences. Lawyers from Morgan & Morgan, the 42nd largest US law firm by headcount, were recently sanctioned $5K for citing eight nonexistent court cases in a legal motion.

When contacted, most AI companies didn't respond. OpenAI sent a generic statement about supporting publishers, while Microsoft claimed it respects robots.txt—contradicting the study's findings.

Our take: Search citation tools create a dangerous illusion of reliability. In our experience with Perplexity, the factual results will often be accurate, but the citations will be wrong (like overly featuring one link over the other results)

Take AI search results—especially news citations—with a massive grain of salt. Or better yet, do a traditional Google search to verify. 

Prompt Tip of the Day

Need an AI editor for your overly verbose writing? Same. But because of how language models work, they can’t actually “count words” like a Word / Google Doc can.

The TL;DR on why = because models process text as chunks (tokens) rather than individual words and make educated guesses based on patterns they've seen rather than actually counting one-by-one.

With that said, here’s the best word-count cutter prompt we’ve created for ourselves here at The Neuron—we use this with Claude, but you can try it on GPT as well.

Paste it along with the text to edit, fill in your ideal + current word counts, and try it out!

Treats To Try.

  1. OptimHire finds and screens tech talent for you, handling everything from candidate selection to interview scheduling (raised $5M).

  2. Willow Voice turns your speech into perfectly formatted text in any app, helping you write 3x faster—we tried it and found it v helpful.

  3. Freepik now has a video upscaler powered by the popular Topaz Labs.

  4. Touch Grass locks your apps until you prove you've actually gone outside and
 touched grass.

  5. RAIC Labs helps you instantly find and continuously track objects of interest across your visual data (video, image, geospatial).

  6. Simon Willison has become somewhat of a popular “AI coding influencer”—here’s his tips on how he uses AI to help him write code.

  7. Lovelace Studio is building Nyric, a tool to create and explore interconnected 3D worlds built by you and other players—read more.

  8. Typeface’s new marketing platform creates distinctive brand content for you with specialized agents, a living brand playbook, and collaborative workspaces

Around the Horn.

  • Google will use Gemini to replace Google Assistant on mobile devices, prioritizing features like free-flowing conversations and personal research assistance.

  • Agents were the hot topic at the HumanX conference this week, as agent startups are quickly becoming the next big acquisition trend after ServiceNow acquired Moveworks and UiPath acquired Peak.

    • Related: TechCrunch wrote a great deep dive explaining how everyone has a different definition of agent, and why that’s confusing (and hard to benchmark).

  • Law firm Hill Dickinson restricted AI tool access after discovering policy violations, including over 32K ChatGPT hits and 3K DeepSeek hits in a single week.

  • Someone created an AI angel investor, and “she” already invested $100K in a startup.

  • Listen to this interview with a student who used ChatGPT for homework and got caught—and what to do about the problem of using AI for school and work.

FROM OUR PARTNERS

ProfAI demo on March 18

Train your whole team on AI with ProfAI, a personalized, AI-powered coach.

ProfAI can:

  • Upskill teams on prompting, safe AI use, and effective use cases.

  • Personalize exercises, feedback, and lesson plans to the user.

  • Grant badges and certification in AI proficiency.

Sunday Special: Bots Behaving Badly

  1. A Russian disinformation network known as “Pravda” seemingly flooded the web with propaganda, and now, major 10 AI chatbots recycle pro-Kremlin arguments about a third of the time.

  2. A new study found that certain AI models fail to recognize 66% of critical health declines—and another study found that 65% of US hospitals use AI predictive models.

  3. Be careful what you download—this Disney worker accidentally downloaded malware in his AI and lots his job because of it.

  4. Academic publisher Elsevier defended a nonsensical term called “Vegetative electron microscopy” in nearly two dozen papers, likely stemming from AI misreading a 1959 paper where “vegetative” appeared in one column beside “electron microscopy” in another.

  5. Joyce Esser, who has motor neuron disease and uses an ElevenLabs AI voice clone, was temporarily banned for using words like “arse” in private conversations—raising questions about content moderation for assistive technology users.

A Cat's Commentary.

That’s all for today, for more AI treats, check out our website.

The best way to support us is by checking out our sponsors—today’s is Section.

What'd you think of today's email?

Login or Subscribe to participate in polls.