The week was about access. Cloudflare drew a hard line that could block Googlebot itself from millions of sites, Google made several publisher-friendly moves in AI Mode and Search, and new research showed that not every AI assistant even sees the same web.
Meanwhile, a poll of 1,212 votes settled the “who does GEO” debate: SEOs do. The throughline across all of it: who gets to read your content, and on what terms, is becoming the central question of search. Sources scanned: 300+
The week in numbers
- September 15 is Cloudflare’s deadline: after that, blocking AI training can also block Googlebot.
- AI agent requests on Cloudflare’s network grew more than 1,700% in a year.
- Over 50% of AI crawler traffic re-fetches pages that have not changed.
- 95% of 1,212 poll respondents say SEOs are leading or directly involved in AI search optimization.
Search engine updates
Cloudflare’s new AI crawler rules can block Googlebot
This is the story of the week. Cloudflare now sorts crawlers into three categories by behavior: Search, Agent, and Training. From September 15, multi-purpose crawlers that combine Search with Training, including Googlebot, Applebot, and Bingbot, will be governed by the most restrictive rule that applies.
In practice, any site that has blocked Training (including through the older “Block AI bots” toggle) will also block Googlebot on ad-monetized pages unless the owner opts out first. New domains and free-tier customers automatically get the new defaults.
Why it matters: if your sites or your clients’ sites run on Cloudflare, review the AI crawler settings before September 15, or you may unintentionally cut off Google Search crawling at the network level, where robots.txt cannot save you.
Cloudflare is also evolving Pay Per Crawl into Pay Per Use, compensating publishers when content shapes an AI answer rather than just when it is fetched. (Sources: Search Engine Journal, Cloudflare.)
Google makes recipes in AI Mode more publisher-friendly
Google updated recipe results in AI Mode to show creator names, recipe ratings, and ingredient counts for some recipes, making source attribution more visible. A small but welcome step for food publishers, and hopefully one that translates into clicks. (Source: Search Engine Land.)
Liz Reid: personalization can help small publishers
Google’s Head of Search argued that personalized search and preferred sources can help niche and small publishers surface, countering the fear that personalization buries small sites.
Her point: a trusted publisher with preferred-source status can appear more prominently than larger outlets that carry the same information.
Why it matters: earning “preferred source” status from your audience is quietly becoming a distribution lever, so it is worth prompting loyal readers to select you. (Source: Search Engine Journal.)
Google sends searchers to site-hosted AMP pages instead of cached ones
Google will now send Search users directly to site-hosted AMP pages rather than Google-cached versions, reducing the need to update the AMP cache or configure signed exchanges. If you still run AMP, this simplifies the setup and puts users on your own infrastructure. (Source: Search Engine Roundtable.)
Google Trends adds comparison-over-time charts
Google Trends now lets you compare keyword interest against a previous time period with a dedicated comparison chart, finally making historical trend analysis easier without exporting data. (Source: Search Engine Roundtable.)
Two studies worth your time
Eye tracking: which SERP elements get viewed but not clicked
MJ Cachón and Diego Criado of Laika measured the attention-to-action gap across SERP elements and found that users often view search features without clicking them: elements with similar visibility showed widely different click-through rates.
Why it matters: AI Overviews and other features can capture attention without driving clicks, so visibility and traffic metrics need to be read separately. This is real eye-tracking data on the question everyone has been guessing about. (Source: Laika.)
Not every AI assistant sees the same web
Andre Alpar compared how AI systems use live web sources versus training data, showing that crawler access, freshness, and retrieval behavior differ meaningfully between assistants. Some read the live web constantly, others lean heavily on stale training data.
Why it matters: your GEO strategy depends on which systems can actually reach your content right now, which connects directly to the Cloudflare story above. Blocking the wrong crawler category can make you invisible to the systems that do read the live web. (Source: Search Engine World.)
AI search insights
The debate is over: SEOs are doing the AI search work
After 1,212+ votes in Aleyda Solis’s poll, 95% of responses indicated that SEOs are leading or directly involved in AI search optimization efforts.
Whatever the industry ends up calling it (GEO, AEO, AI SEO), companies have already answered the question that matters: the SEO team owns the work.
Dan Taylor’s companion piece argues the point from the other side: AI search is nothing without SEO, and it knows it. (Sources: Aleyda Solis on LinkedIn, Search Engine Journal.)
A five-step fan-out framework for SEO and AI visibility
Cyrus Shepard shared a practical framework for identifying the fan-out queries behind keywords you already rank for, prioritizing the relevant subtopics, and improving both classic rankings and AI visibility with the same work.
Fan-out queries are quickly becoming the practical bridge between traditional keyword research and AI answer optimization. (Source: Zyppy Signal.)
Lily Ray: do great GEO without destroying your SEO
Lily Ray published a deck on how SEO and GEO overlap, with tactics to optimize for AI search visibility while protecting the organic rankings and traffic you already have. A good framing for teams worried that chasing AI citations means cannibalizing what works. (Source: SlideShare.)
How companies are deploying AI to grow traffic and revenue
Glen Allsopp collected real examples of how startups and larger companies use AI workflows to support growth, with ideas that translate to businesses of any size. Useful as a reality check on what AI adoption in marketing actually looks like beyond the hype. (Source: Detailed.)
Internal links quietly decay, and so does the equity they pass
Sophie Brannon explains how internal link decay redistributes PageRank away from important pages over time as sites grow, redirect, and prune, and how to identify and reclaim the lost equity.
A fundamentals piece that pairs well with a week dominated by crawler access questions: the links AI and Google follow are the ones you maintain. (Source: Search Engine Journal.)
The theme of the week: access is the new battleground
Cloudflare’s September 15 deadline reframes the AI content fight from “should I block AI bots” to “which crawler behaviors do I allow, and at what price.”
At the same time, Andre Alpar’s research shows the AI assistants themselves differ in what they can read, and Google is nudging publishers with friendlier attribution and personalization levers.
The winners of the next year will be the teams that treat crawler access as a strategy: knowing exactly which bots reach their content, what those bots do with it, and what they get in return. Audit your Cloudflare settings, check your server logs, and make access a deliberate decision instead of a default.
On the radar
- Semrush AI Visibility Index: a new study built on 126 million US prompts maps which retailers dominate AI recommendations across ChatGPT, Google AI Mode, Gemini, and AI Overviews. Vendor data, but a large sample. (link)
- Reddit SEO: Despina Gavoyannis on using Reddit for brand foundations and audience insight without getting banned. (link)
- Stakeholder buy-in: Nick LeRoy on using Google’s own documentation as proof instead of saying “best practice.” (link)
- Digital PR: Gianluca Fiorelli and Britt Klontz on earned media strategies and human-first pitching in the AI search era. (link)
- Travel content: “The Rosalia Effect,” Fiorelli on content architecture and information gain for AI search visibility in travel. (link)
- Unconfirmed volatility: community reports of ranking movement around June 19 to 21 that hit spam and black-hat tactics hardest, while most trackers stayed calm. Google has not confirmed an update. (link)
Tools to bookmark
- SEO Gets: connects GSC and GA4 into the analytics and reporting layer they should have had all along.
- Google Traffic Simulator (Zyppy): simulate GSC traffic patterns by domain, volume, direction, and slope, handy for mockups, training, and demos.
On the calendar
- Open Mic Night by Take It Offline x Search ‘n Stuff (London, July 10): SEOs, PPC specialists, content marketers, and founders with a microphone.
- Belgrade SEO Conference (Belgrade, September 18): Aleyda Solis, Mark Williams-Cook, Judith Lewis, and more.
If you only cover three things this week
- Cloudflare’s September 15 crawler deadline: audit your settings before blocking Training also blocks Googlebot.
- The Laika eye-tracking study: attention and clicks are different metrics, measure them separately.
- The 95% poll result: SEOs own AI search optimization, whatever it ends up being called.
That is the week. The thread tying it together: crawler access is turning into leverage, and the teams that treat it as a deliberate strategy rather than a default setting will come out ahead. If you want this roundup in your inbox every week, subscribe below.
By Danny Donchev, FortuneLords. Follow on LinkedIn and X.