EP 277- Correcting the Machine: Citation Labs on the Six Layers of AI Visibility (Part 1)
Garrett French, James Wirth and Valerie Cecil of Citation Labs talk to us about how they measure AI visibility across six layers, build the prompts worth tracking, and replace the stale facts AI repeats about a brand.
Garrett French, James Wirth and Valerie Cecil of Citation Labs and its local-sponsorship sibling ZipSprout join Greg Sterling and Mike Blumenthal for Part 1 of a two-part Near Memo conversation, on what has changed in the market and how an enterprise link-building shop now diagnoses and repairs what AI systems say about a client. Part 2 will cover the agency side: skills, tooling and pricing.
AI visibility is a chain of dependencies — a brand has to be returned, read, cited, mentioned and recommended, and framed correctly at every step — so the work shifts from winning a ranking to finding the layer where the chain breaks and supplying the missing or corrected facts, mostly on pages the brand doesn't own.
James Wirth's Six Layers of AI Visibility
James lays out the model Citation Labs audits against (34:58–38:09). Each layer raises the odds of the next, and none guarantees it. The audit that maps a client onto these layers is what the team calls a visibility probe.
| Layer | What it answers | The primary lever |
|---|---|---|
| 1. Returned | Does the brand's page appear in the search results the model pulls when it runs its query fan-outs? | Traditional SEO and link building; new content for fan-out queries where the brand has no presence |
| 2. Read / retrieved | Does the model open the page, or answer from the search snippet alone? | Page structure the model can lift cleanly; watching page-retrieval patterns as an early signal |
| 3. Cited | Is the page used as a source in the answer? | Insertions on publishers that are already being cited; correcting cited sources that state the wrong facts |
| 4. Mentioned | Is the brand named in the answer? | "Narrative expansion": covering the part of the story the brand is absent from |
| 5. Recommended | Is the brand put forward as an option, at what rank, and with what qualifier? | Supplying the facts that remove a limiting qualifier and move the brand up the list |
| 6. Framing | Across all five: are the facts, pricing, sentiment and attributes right? | Off-domain content the client can keep current when third-party pages won't update |
Takeaways
-
Accountability for links became the AI-visibility method. Citation Labs won an enterprise client by being the only one of five agencies, in a six-month pilot with equal budgets, able to show that a link moved a ranking that mattered. James was on the client side of that test and then asked for a job. Garrett says the habit of proving "what the work did" is what the team carried into AI visibility. (01:33–04:33)
-
Every enterprise client has moved to AI visibility. Garrett is explicit that this is three clients and anecdotal, but all three have been hit by zero-click and all three pushed the agency "wholly into AI visibility." Links still matter; they are now one lever among several. (10:32–11:58)
-
Conference-floor conversations contradict the "business as usual" data. At BrightonSEO San Diego, James heard talks built on large aggregate data sets showing little change, then heard the opposite at the booth: organic traffic down sharply, direct and referral numbers that no longer reconcile, and conversions nobody can attribute. His own audits show declines "far more than what the large data sets are reporting." (12:36–15:17)
-
Local is where the damage is smallest and the AI answers are worst. Mike sees web traffic falling for local clients without matching drops in calls and conversions. James doesn't trust local AI answers, finds them often wrong, and ends up back in the map pack or a maps app. Greg's read: AI works the top and middle of the funnel, Google gets the last click, and the AI influence never shows up in referrals. (15:17–20:26)
-
Greg's roof repaint never touched Google. He asked ChatGPT for three body shops that would paint only a roof, got a ranked list built on Yelp data, had it pull in Google reviews for comparison, and clicked through to Yelp's quote request. He calls himself an outlier. The anecdote still shows a complete local purchase path with Google present only as review data inside another product. (20:46–22:03)
-
Prompts are built from the offering, not from keywords. The team starts at the client's transaction pages and works backward to the problem a buyer is solving, drawing on customer-service logs, Reddit complaints and interviews with subject-matter experts. The resulting prompts run 50 to 300 words and specify a role inside the ideal customer profile. Ten or more are tracked daily for each offering, not each client. (28:31–32:50)
-
The model's opinion of the brand replaces the searcher's. A searcher used to bring their own sense of a brand's reputation to the ten blue links. Now the model assembles that judgment from far more sources than any person would read and attaches its own qualifiers. That is why sentiment and framing are tracked alongside citations. (33:04–34:27)
-
Stale comparison pages are a repair job you can do off-site. Garrett's example is a client whose service packages change often. Affiliate-style comparison pages kept getting cited with outdated facts and wouldn't update. Citation Labs published accurate pages it controls and can revise, then tracked the result: the new pages were cited, the brand was mentioned, and the framing changed. Garrett on the work: "We get to correct." (39:41–41:41)
-
A months-old Bing snippet set a telecom's pricing in the answer. In one audit, a non-Google model ran its fan-outs through Bing, which was still serving a cached snippet from a page updated months earlier. The model never opened the page and answered from the snippet, producing wrong pricing, offer terms and expiration dates. (43:21–44:37)
-
Models discount what a brand says about itself. James reports seeing models pass over owned-domain content in favor of third-party pages, even to the brand's cost. Many high-value cited publishers are closed to placements, which is why the team builds off-domain microsites as well as pitching insertions. (38:49–39:22, 43:11–43:21)
-
Page-retrieval patterns are the earliest signal. Valerie watches which pages models pull and in what form: FAQs, comparisons, graphs, statistics. Retrieval shows up before citations or mentions do, so it tells the team whether a content bet is working. James's caveat: write concisely, the way you would argue before a judge, without rewriting everything for machines at the reader's expense. (41:46–43:11)
-
Fan-outs now skip discovery and query brand sites directly. The team saw models stop running non-brand searches and go straight to
site:queries against brands they already know, a shift that spiked in ChatGPT around April and that James ties to a model change. Valerie adds that .gov sources are being favored more in some industries. Garrett's hunch: a fan-out means the training data wasn't enough to answer. (45:58–47:12) [1] -
They "professionalize" prompts instead of personalizing them. Greg has told ChatGPT never to show him eBay results, and it complies. Citation Labs can't model that. It asks what a given role (a CFO, say) would need in a perfect-information setting and builds that context into the prompt as a proxy. Owning the tracking tool means no cap on prompt volume for testing. (48:19–51:39)
-
Google still gets the tie-break. They measure ChatGPT and Google's surfaces together and act on the aggregate, but James would not close a ChatGPT gap at the expense of AI Overviews, AI Mode or Gemini. Success is measured in sentiment shift, mention count, recommendation rank, and whether a limiting qualifier disappears. (51:42–54:10)
Related Reading
- Citation Labs and its team page — Garrett French (CEO), James Wirth (Sr. Director, Strategy and Growth Marketing) and Valerie Cecil (Operations Manager).
- Citation Labs BrightonSEO 2026 resources — the field guides (AI Visibility, Information Design, Business Impact) from the BrightonSEO San Diego appearance James describes.
- Xofu — the Citation Labs sister company's prompt tracker for AI answers. On air the team says only that it built its own AI visibility tools.
- How Consumers Navigate High-Stakes Purchases in AI Mode — Kevin Indig's write-up of the usability study with Citation Labs and Eric Van Buskirk's Clickstream Solutions, the "project we did with Eric" Garrett mentions at 27:07.
- How to Track and Fix BOFU Visibility in LLMs — the Citation Labs podcast on the same sales-page-first approach.
- EP 276 — GEO Is CRO: Ross Hudgens on Chasing Influence, Not Citations — last week's counterpoint: Ross argues against chasing today's cited sources; Citation Labs argues for repairing them. *
- EP 275 — Rankings Up, Calls Down: What AI Visibility Is Really Worth to Local Businesses (Part 2) — the attribution problem Mike raises at 15:17.
- EP 274 — Why "Position One" Doesn't Exist in Local Anymore (BrightonSEO Recap) — Near Media's own recap of the same conference.
- EP 270 — The Hidden Advantage: How Brand Awareness Decides the Click in Legal Search — the legal user-behavior research Greg describes at 25:41.
Practitioner Notes
- The six layers give local audits a missing vocabulary. Most local AI-visibility reporting stops at "mentioned or not." Splitting returned / read / cited / mentioned / recommended / framing tells you which fix applies. It maps directly onto the fan-out and retrieval-hierarchy work on Ask Maps versus Gemini: if a GBP-grounded surface never issues a web fan-out, layers 1–3 are a profile and review problem, not a content problem.
- James's distrust of local AI answers matches the benchmark data. His "often wrong" experience lines up with LocalBench, where LLM accuracy on local numerical queries came in below 15.5%. That is the case for treating framing audits (hours, pricing, service area, offer terms) as a recurring local deliverable, and for checking Bing's index freshness after any change to a client's key pages.
- Garrett's critique of simulated purchase studies is the design brief for the survey work. His objection at 27:07, that participants have no skin in the game, is the gap the Choosing a Lawyer research (EP 270) acknowledges and the conversational consumer-journey survey is built to narrow, by interviewing people about a decision they actually made. Worth stating that limitation up front in the brand white paper, since a peer raised it unprompted.
- "Models discount owned content" raises the value of what local businesses already have off-site. For a local firm the third-party corpus is reviews, GBP, directories and sponsorship pages. ZipSprout-style community sponsorships produce exactly the independent, locally relevant pages the returned and cited layers reward, which makes them worth testing as an AI-visibility input and not only a link source.
Concepts
- Six layers of AI visibility — returned, read/retrieved, cited, mentioned, recommended, and framing (which runs across the other five). The diagnostic model for locating where a brand drops out of an AI answer.
- Visibility probe — Citation Labs' initial audit: simulate the prompts, run the fan-outs, and record where the brand appears or is missing at each layer.
- Query fan-out (QFO) — the set of searches a model issues behind a single prompt to ground its answer. Being absent from a fan-out query is treated as a fixable gap.
- Citation optimization — getting an already-cited third-party source to state what is factually true about a client. Presented as the successor to link building.
- Tracking-worthy prompts — long, role-specific prompts (50–300 words) that carry the buyer's situation, built from the offering backward and tracked daily.
- Sales-page link building — the firm's earlier discipline of building links to transactional pages and proving the ranking effect, which set the standard for proving impact now.
- Narrative expansion — adding the missing part of a brand's story so it gets mentioned for things it currently isn't.
- Page-retrieval patterns — which pages and formats a model actually fetches; used as a leading indicator ahead of citations.
- Chunkifying — structuring content into concise, self-contained units a model can lift without losing meaning.
- Professionalize, don't personalize — modeling the information needs of a role instead of an individual's history and preferences.
- Framing — how the model characterizes the brand: accuracy of facts, sentiment, and the qualifiers attached to a recommendation.
Pick your starting point:
00:00 Welcome and introductions
01:33 The five-agency test that started it all
06:23 What Citation Labs and ZipSprout do
09:09 Is AI search overhyped?
12:36 Traffic is down more than the data says
15:17 Local: traffic falls, conversions don't
20:46 Greg's ChatGPT hunt for a body shop
24:29 Attribution and the last-click habit
28:31 Building prompts from the offering backward
33:04 When AI decides a brand's reputation
34:27 The six layers of AI visibility
39:41 Fixing stale comparison pages
43:21 The telecom and the stale Bing snippet
44:37 Training data, caches and fan-outs
48:19 Personalization and the eBay problem
51:42 Weighting Google against ChatGPT
Full Transcript -->
Near Memo EP 277 — Correcting the Machine: Citation Labs on the Six Layers of AI Visibility (Part 1)
Editor's note: This transcript has been edited for reading. Stutters, repeated words, false starts and clear transcription errors were cleaned up, and brief backchannel ("yeah," "right," "mm") was folded into the main speaker's turn or dropped. Wording is otherwise faithful to what was said. Figures and claims are left as spoken.
Hosts: Greg Sterling, Mike Blumenthal
Guests: Garrett French, James Wirth and Valerie Cecil, Citation Labs / ZipSprout
Greg Sterling (00:10)
Hey everybody, welcome back to the Near Memo, the Near Media podcast, with me, Greg Sterling, and as always the lovely and talented Mike Blumenthal. Today we are joined by very special guests: Valerie Cecil, James Wirth, and of course Garrett French, from Citation Labs and ZipSprout. Welcome, everybody.
Garrett French (00:27)
Thank you.
James Wirth (00:28)
Thank you, Greg.
Greg Sterling (00:30)
And Garrett is awaiting parole, it looks like, but he'll still be talking to us from his cell today, which is very generous.
Garrett French (00:34)
Yeah. I got special permission for this. Well, they knew who Greg was.
Greg Sterling (00:40)
All right. Thank you all for joining us. We're going to do a special episode, broken into two parts. This one is about the changes in the market: what these guys are seeing from clients, how they're responding, how their tactics are changing. Then we're also going to talk about agency operations and management, and how agencies as entities need to adapt to what's going on. But before we get into all of that, why don't you introduce yourselves a little and give us some context? Many people are going to know who you are, but there will be some who do not.
Garrett French (01:12)
We should introduce each other, so I don't get to talk about myself.
Greg Sterling (01:16)
Sure. Go for it.
Garrett French (01:18)
He said sure. Are you guys going to edit this podcast? I just have to ask that.
James Wirth (01:19)
That's a great idea. Garrett introduces me, I introduce Valerie, Valerie introduces Garrett.
Garrett French (01:26)
Done.
Valerie Cecil (01:27)
Love it.
Garrett French (01:27)
James Wirth is amazing. He came on board six years ago.
James Wirth (01:32)
Six, going on.
Garrett French (01:33)
He came in from a client we were working for, where we had to prove that links did something after we built them. That really started the journey we're going to be discussing today. When you have to prove it to people who understand statistics, who understand ROI, where the money goes and what it produces, you have to go in with an accountability that link building really hadn't had before that. We really hadn't seen a lot of that before. So we had an emphasis and a focus on sales pages at that time. Greg, you said I couldn't pitch, but I just launched into it.
Greg Sterling (02:18)
Description is different than pitching, right? It's a subtle but meaningful distinction.
Garrett French (02:21)
Okay, fair. That's fair. But he came in, and they had pitted, what was it, six link building agencies against each other? Five?
James Wirth (02:29)
Five different agencies. Yeah.
Garrett French (02:32)
And they said, we're going to pick one of them. They ended up picking us because we were the only ones that could show impact, or that could work in a way where they were able to demonstrate internally the impact of a link on a ranking that was considered valuable.
So he came in to help us really expand that program. We had an emphasis that started on what we call sales page link building, really focusing on the transactional pages. As a result, we built out tooling and software so that we could demonstrate to our clients, which are primarily enterprise, that we've done something, that we've produced a result that's valuable. That has been hugely impactful for us as we've ventured into the AI visibility era, because that discipline around showing what the work did has forced us to go farther and farther down rabbit holes of what's actually happening. Have we done anything? Can we claim any kind of impact from our work?
So James has been with us for six years. He recently closed an enormous deal for us. He's contributing technology and strategy, he's vibe coding, he's closing deals. James is incredible. Your turn, James.
James Wirth (03:47)
Wow, that's way longer than my intro of me would have been. But very solid.
Valerie Cecil (03:48)
I was just going to say "Garrett builds links." I've got to work on my intro.
James Wirth (03:54)
I'd like to introduce Valerie Cecil. She is our head of ops. She is our glue, our engine, our sanity. She's behind everything we are able to claim as successes. If it was done well, then Valerie was behind it. And one of the things I like most about working here is that Valerie is Garrett's sister.
And that was after working with them for six months. Yes, Garrett's correct, I asked for a job. Link building is hard, and I've never seen it done so well. They beat out four other link building agencies after six months of pilot programs with the same budget across each. That's why I'm here today.
Mike Blumenthal (04:33)
Valerie, before your introduction, I just want to level-set people's expectation of honest communication. In the green room, Garrett said you were his oldest sister. Is that true?
Valerie Cecil (04:43)
Incorrect. Thank you for letting me clarify that. Way younger. Way younger.
Garrett French (04:45)
How dare — no.
Mike Blumenthal (04:49)
Okay. You don't have the beard, so I assumed you were younger.
Garrett French (04:51)
I didn't know there was going to be a fact-checking session. What is this garbage? I need to talk to my agent about this.
Valerie Cecil (04:58)
I love that question. And Garrett French: you may or may not have heard of him before today. If you hadn't, he founded Citation Labs. Lucky name, now that we're in a citation-building era. What was it, fifteen, sixteen years ago, Garrett? And then started ZipSprout about ten years ago.
Garrett French (05:15)
We're entering our sixteenth year.
Valerie Cecil (05:19)
Our sixteenth year. He is the most — well, intelligent —
Garrett French (05:22)
Boy.
Valerie Cecil (05:24)
— person that I've ever worked with, in terms of always wanting to do a better job of delivering to the market and making sure that we're adding value to the internet, to LLMs. I think that's what drives all of us. We're always experimenting, always learning more, always pushing, and he brings that directly to our clients. That's why we've been with some of our clients for ten years, which is pretty much unheard of in the link building slash citation building era. We're all so grateful that we're still around, and I think it's Garrett's constant push for survival that we're here.
Garrett French (06:00)
I'm blushing. She has never said so many nice things about me all at once. She accidentally complimented me one time and then just never did it after that.
Valerie Cecil (06:04)
I was so tired. It's been a hard week, Greg.
Greg Sterling (06:13)
This is really a self-esteem building session, not really a podcast.
Garrett French (06:16)
I know. Thank you. We need to stop now, because it's not going to get better than this for me.
Greg Sterling (06:23)
So give us one more thing, about your services. We're hearing about link building. What's the full range of things that you offer to the market? Not as individuals, but as an entity.
Garrett French (06:36)
Valerie can fill in the gaps, but we've been primarily off-site. In this area, we're either engaging with a cited source that doesn't say what is factual about a client, or doesn't line up factually with what we want an AI to know. So we might do a citation optimization approach there.
We might find a domain that's cited frequently across a prompt set, and we might say, this would be a great place to publish a QFO-oriented article — query fan-out oriented, or facing, or informing — because we know it's getting cited and it has visibility within this prompt set.
We still do link building. We still have people who come in with visibility problems or visibility challenges around target pages, a landing page or a sales page.
And then with ZipSprout, my darling, it's a local sponsorship and community engagement, community PR play, where we connect our clients to local communities through sponsorship and social engagement with that entity.
Greg Sterling (07:55)
Why did you start ZipSprout as a separate entity? I don't know what its legal organization is, but it has a separate identity in the market from Citation Labs. Why did you feel you needed to do that, versus just offering those services under the banner of Citation Labs?
Garrett French (08:10)
That's such a good question. We really had to establish to nonprofits and local community organizations that we were a legitimate, functioning marketplace: that we would be bringing them clients, and they would hopefully be adding information about those clients to their websites. We needed an identity that was meaningful, a brand that could make sense and be coherent for that specific type of community.
I did start it as a separate LLC initially. We folded it back in. And some of my thinking ten years ago — I don't have this thinking now — was, gosh, this is going to be a very sellable entity. By itself, as a standalone agency, it could be sellable. It's not for sale. But that was some of my thinking then. And I still believe it makes sense as a standalone brand. I don't know if I answered your question, though, Greg.
Greg Sterling (09:09)
Okay. We're on episode 277, so we've done a lot of episodes, and many of them over the last year and a half have been increasingly about the impact of AI: user behavior, tactical adaptations. AI is certainly a transformative technology, whether you hate it or love it. It's having a big impact in certain areas of society.
But in terms of SEO and search visibility, we see a debate playing out very often on LinkedIn and in other places, where some SEOs say this is all just wildly overhyped. It's still really all about Google. It's still Google's world. Sure, Google's transforming its SERP, but this AI stuff is being overblown, and it's only having a marginal or incremental impact, if any impact at all. What is your response to that position? Anybody can speak to it.
Garrett French (10:17)
I'm going to jump in, because I don't like it when I don't talk. It doesn't make me feel good. I would welcome an addendum from Valerie.
Mike Blumenthal (10:25)
Our job here is to make Garrett feel good, I think.
Valerie Cecil (10:27)
That's it. I'd appreciate a break from my normal job, Mike, so thank you.
Garrett French (10:32)
It's tough for her. So we have a small, anecdotal rejoinder to the narrative, which is that we have a handful of enterprise clients. I say handful; I really mean like three. Maybe three and a half. Every single one of them has pushed us wholly into AI visibility. Links are still a factor, but every single one of these clients has been impacted by the zero-click reality. The traffic is down. And we also see a lot of appetite for exploring and understanding what AI visibility is, what the mechanisms are, how we should measure it.
So we have shown up with our dev capacity, with our operational capacity, with James Wirth and his capacity for positioning impact and helping people understand what is or isn't actually happening. So my purview is that 100% of the market is moving towards AI visibility. Now, I know that's not the reality across — you know, I know that's not what's commonly discussed or said on LinkedIn. But that's my perspective, from my small and admittedly anecdotal —
Mike Blumenthal (11:58)
I would address this question maybe to James: the question of AI, and what does "AI" mean? Does that mean 80% Google AI, or does it mean 80% ChatGPT AI? First question. Second question: given that Google is an AI-first company and has been for a number of years, everything they do is AI-driven, including the organic SERP results. They just announced a new paper and a new ability to reorganize SERPs based on their machine learning and AI algorithms. So it's not even clear to me how you make these distinctions, and what it really means.
James Wirth (12:36)
Yep, that's a great call-out, and that's right on the nose. My soapbox usually is: wait a minute, we keep saying "Google versus AI," but what are we really talking about there? If I just go to Google.com, I'm presented, in Google's response to me, with ads of course, like we were talking about earlier, and that's been the game all along for Google, sixty-whatever percent of Alphabet's income or revenue. And then there may or may not be the AI Overview, and now we're getting into AI chat, which really is the differentiator for me. Am I having a conversation with AI around a search, or am I just looking at my traditional ten blue links? And if I'm having that conversation with a chat agent, is it Google's chat agent, or is it Perplexity or Claude or ChatGPT?
So the comparison of Google versus AI is really ambiguous, to the point where we don't really know what we're talking about from that perspective. But we do know a number of things, across a number of pretty large clients and all of the research I do: any site of substantive organic traffic historically has seen a dramatic decline in that traffic. Is that across the entire database of a very large analytics company and their index of the web? Maybe not, because we might be talking about fringe cases, even if we're analyzing a site with millions of visits per month of organic traffic historically.
When we were at the last conference, BrightonSEO in San Diego, where Garrett spoke and we were exhibiting, we were hearing talks about business as usual, maybe across all of the data that tools are measuring. That was in stark contrast to every single conversation we had, which was: our organic traffic has plummeted. We don't know where our traffic is coming from now. We don't have a good measure on conversions, because we're getting large disparities in our direct traffic and our referral traffic. Some of it, yes, is attributed to AI — ChatGPT, for example; that's the largest share. But outside of that, we don't know what's happening.
So that is the reality for us across every site I measure, whether it's a prospect or an existing client: traffic has tanked. It's down far more than what the large data sets are reporting, and they're struggling to understand that. That's where I'm spending the majority of my time.
Mike Blumenthal (15:17)
Can I add a follow-up question? Obviously, Greg and I are immersed in local, where the outcome goal is the call or the visit, not the website visit. We do see decline in overall web traffic, but we don't see equivalent, or as dramatic, declines in conversions. Now, tracking conversions is very difficult. With one client, David and I literally spent a year trying, and finally succeeding, to bring all of their inputs [from Google and their legal software] into a conversion dashboard, so we can now see where they're coming from, even though final attribution is clearly not the only story. It's very complicated.
But what we're seeing is decline in organic traffic, and not so much decline in local. Not so much decline in phone calls, not so much decline in conversions. Some. Have you thought about that, and do you have a different answer for that or not?
James Wirth (16:14)
Local is messy, in my own experience and in everything I'm seeing in the data I'm running. Local is really messy, and I personally am not trusting it. If I'm looking for something down the street or in a particular area, the response I get from ChatGPT versus Claude, and including if I just do that same search in Google and look at the AI chat response — I personally don't trust it, because it's often wrong. It sends me down rabbit holes that aren't relevant. It's got incorrect information. So I end up in traditional map pack results, going to Google Maps or Apple Maps or something like that and redoing that search, even though I'm typically starting it in an AI chat interface because I'm curious about what it's delivering.
Across all the measurements that I've seen so far, traffic is still squirrelly, the response is still very different, and it's not as refined as non-local AI conversations are. That's been my own personal experience, and all of the data that I've been seeing.
Greg Sterling (17:25)
I want to give Valerie a chance to jump in, because you guys have been monopolizing the conversation. Did you want to add anything, Valerie?
Valerie Cecil (17:34)
Thank you, Greg. Honestly, I don't think so, not yet. I'm just listening and taking it all in too. Thank you for the opportunity, though.
James Wirth (17:35)
Story of her life.
Greg Sterling (17:44)
I'm just being playful here. Okay, good. I'm relieved to hear that.
Garrett French (17:48)
She will interrupt you, don't worry. She's really good at that.
Greg Sterling (17:53)
So, in the research: we at Near Media do two kinds of research. We do user behavior testing that's similar to the stuff that Eric Van Buskirk is doing, except we do much larger sample sizes. We typically do them in legal these days, but we've done them in hospitality and restaurants and self-storage and healthcare and so on. We also do a lot of survey work, in different verticals, but legal is an area where we have a product, so that's why there's a lot of concentration there.
Garrett French (18:28)
Okay, who's pitching who now, Greg? Who's pitching who?
Greg Sterling (18:31)
I'm qualifying the statements I'm about to make.
Garrett French (18:33)
Greg, I would like an email after this podcast from you about your services. I didn't know that. That's really cool. I would like to do some of that stuff. Greg just made a sale on his podcast, all right? I'm just calling that out. Keep going.
Greg Sterling (18:48)
There's the ROI. There's the conversion. We know it's deterministic. Here it is.
No, that was not my intent. My intent was just to qualify what I was about to say, which is in local. When we talk about AI, we really have to talk about demographics. We have to talk about the differences between the different engines, or whatever we're calling them. We have to talk about the industries, because there are a lot of differences and a lot of nuances.
What happens in local — the crudest generalization I can make is that AI operates at the top and the middle of the funnel, where people are doing discovery and then comparisons. Typically, when people want to validate — here's the short list, I want to look at the reviews, or get the hours of operation, or get directions — they go into Google. Google is the last click. And then of course there's a call or a visit or whatever. And Google still shows up in the referrals. So the AI influence is kind of hidden in that context, because you're typically not getting a click-through from AI Mode or ChatGPT or Claude or Perplexity, or, God forbid, Grok.
So Google is still, in local, kind of the center of the universe, but its influence over people's decision-making is starting to change. I'll give you a quick case in point. Mike, you know, loves to listen to these anecdotal examples that I give.
Mike Blumenthal (20:26)
And I still wonder why Greg uses Google, or Yelp, for home services. But okay, go ahead. With that caveat.
Garrett French (20:35)
Anecdotes are welcome. I'm so excited.
James Wirth (20:39)
You opened a can of worms here, Greg.
Valerie Cecil (20:42)
No going back.
Greg Sterling (20:42)
I'm going to try and make it short, because we'll be on for three hours otherwise.
So I have an old Honda CR-V. It was my daughter's car in high school. The car's in good shape. It's got a lot of miles, but it's in good shape. It's got some very oxidized paint on the roof, which makes it look kind of ugly, unsightly. So I went —
Garrett French (20:58)
Wait, weren't you just talking shit about my basement, Greg? Come on, let's go take a tour of your car. No, stop everything. We're going to take a quick tour of Greg's CR-V. Come on.
Greg Sterling (21:08)
I'd now like to talk about insurance premiums. No. Anyway, I went to ChatGPT. I said, give me three auto body shops that can paint only a roof, do it affordably, and have great reviews. It gave me three. It gave me a hierarchy of them. And all the knowledge panels that ChatGPT shows were Yelp. It was all Yelp data. Then I said, show me the Google reviews. It did that, so it compared the Yelp and the Google. I identified one. I clicked through on the knowledge panel, and it sent me to the RFP thing that Yelp does — request for quote. I did that. And then of course they spam everybody under the sun, and I'm getting hundreds of people responding to me, which is just a heinous practice.
But nonetheless, later this morning I'm going to take it into a place I found on ChatGPT. I referenced Google to confirm —
Mike Blumenthal (21:57)
So come back next week for the outcome.
Greg Sterling (21:59)
No, it'll be fine.
James Wirth (22:00)
We'll post pictures.
Valerie Cecil (22:01)
Before and after pictures.
Greg Sterling (22:03)
But anyway, I was never on Google. I'm an outlier. I never visited Google.
You're exactly right, James, that there is inaccuracy. What we see in some of the surveys is that consumers want more information from AI than is available. They have a bunch of questions, and the information, the content, is not there. So they go to Google to get that content very often. It's not just reviews; it's particulars, very specific questions.
What I have seen — and then I'll stop my soliloquy — is that people really like the chat interface. If ChatGPT and Claude and all the others fell off the face of the earth, Google might freeze its SERP, but probably not. It would probably still keep migrating, for two reasons. One, because they know how much people like the interface, more than traditional search: the conversational aspect of it. And two, they're getting so much more data than they were getting in a traditional keyword search context.
One of the challenges of AI is a content problem. It's a data and content problem. If you can pump all of the relevant content into AI, and it can understand it and present it, you're in a much better position than with the thin-content approach people have historically taken. Okay, that was a really rambling thing.
Garrett French (23:27)
No, I agree with you. And I hope your car turns out okay, Greg. I really do.
Greg Sterling (23:33)
I'm not worried about it. It's a twenty-year-old car. It's going to be fine. I just want the top painted.
James Wirth (23:39)
Just want the top painted. Yeah. You know, some of the newer interfaces, Greg, are the ones I think are really helpful from that perspective. Like when I have AI Mode and it has its own column, a two-thirds, one-third split, and on the other part of the screen I've got the page that I'm viewing, so I can have the conversation and also view the information.
Another version of that is in ChatGPT, where I can now have the voice conversation. I'm having a voice discussion with the AI agent and I'm reviewing the conversation. I'm clicking on links. The conversation carries over. I'm switching apps on my phone to look at other things that it's talking to me about, and the audio conversation continues. So to your point, that's spot on in my experience and in what I'm seeing.
Garrett French (24:29)
Closing the attribution loop, I think, is some of what you're talking about too, Greg. How do we close the attribution loop? How do we know that, just because you have AI visibility on prompt X, that's going to produce a business result?
James Wirth (24:46)
Also, it's a multi-touch, multi-attribution environment. Maybe not me personally, or Greg or Mike, but one of our kids, if they're doing that research, might start on social media, or look there. We might switch over to a traditional Google search, then go back over to ChatGPT. Sometimes I run Claude against ChatGPT, and I'm seeing the differences in those answers.
Garrett French (25:11)
I do that too.
James Wirth (25:13)
So to your point, Garrett, who does get the attribution for that click? And does that really represent the total experience that I had, in that one environment? No, not at all.
Greg Sterling (25:22)
Is this really new, though? Are we now seeing what has always been essentially true, maybe in a slightly more — I don't want to say extreme, exactly —
Garrett French (25:36)
Accelerated. It's easier to do lots of research now.
Greg Sterling (25:41)
Everybody has been complacent about the last click. It's Google; it's all Google. Historically there have been multiple influences on purchase behavior, depending upon the context and the demographics and all this stuff. But I think the industry has just been lazy and sloppy. Years and years ago — I've talked about this before; this is probably 15 years ago now, maybe not quite — Microsoft tried to popularize a multi-touch attribution model, unsuccessfully. It's just been easy for people to look at the last click for the longest time. But the last click doesn't really show you what formed the short list.
For example, in our research we see the power of brand: brand awareness, prior awareness, and how people respond to that. When they're looking for lawyers and they hit a search results page, everything sort of looks the same. It goes ad, ad, ad. You've got a bunch of middle-aged white guys. Ad here, organic here, and people are scanning and scanning. Often they don't get past the LSAs. But we hear them say, "I've seen these guys, their TV ads," or "I know these guys." And that impacts their clicks, on paid and on organic.
Garrett French (26:58)
But these aren't people in live purchase scenarios, though, correct, Greg?
Greg Sterling (27:03)
They're in simulated purchase scenarios, yes.
Garrett French (27:07)
On the project we did with Eric, I think the people in the studies are just picking something without having any stake in the game, without any skin in the game, so to speak. Now, I'm not saying that invalidates it. But for me personally, that's where these kinds of studies — do we really know that someone who was actually in a car accident is going to choose who they've been seeing a lot? Or are they going to think about it more? Are they going to ask their family? Because it's a big decision, right?
Greg Sterling (27:42)
Clearly that's a valid critique. We're doing the best we can under constrained circumstances. We can't go out on the street and get car accident victims.
Garrett French (27:53)
Welcome to AI visibility, by the way. Keep going.
Mike Blumenthal (27:56)
Which is where I would like to transition, Greg. Garrett's been in the field building links with Citation Labs. He's been in the field building earned content with ZipSprout. And now he has this new exercise, building citations that either correct or enhance AI's understanding of a business. So he's testing it from the other end. I want to hear about those tests, and which of those you've found still have influence, have the most influence, are relevant.
Garrett French (28:31)
No, this is a great frame, because it still is attribution. We're not necessarily closing the loop to "someone converted on the website." But we have to be able to attribute the actions that we take to impact — not in the SERP, but in the answer space, in how AI is answering. That's really been where we've focused. So we're going to start with the prompt.
I'm talking about enterprise. I'm not talking about law firms, which I don't understand. I don't know that space. I'd have to dig in with you guys and we could talk all this through, and it's going to be a very different conversation than for a SaaS tool or something.
Greg Sterling (29:13)
Right. It's different by vertical.
Garrett French (29:15)
Yeah. Our work has been really trying to draw from customer service logs, draw from complaints on Reddit or wherever we're seeing complaints. And we also start from the actual transaction pages on the client's website. We're really trying to land in the space of the problem that someone might be solving with a purchase. What is the problem here? We think of that situationally, and we try to build out prompts that model or represent the situation for an item that's for sale, an offering.
From there, that's how we build out the universe of what we're going to track, what we care about. Now, do we attribute this? We're working on all of that. We're working on the attribution within the analytics that we're able to get access to on behalf of our clients. But to everyone's point on this call, that doesn't necessarily serve. We might never know.
So what we're trying to say is: in a perfect environment, what facts and information does an ICP, a role-specific person within this ICP, need to make a great purchase decision? What information do they need to achieve the best possible outcome for themselves and their organization, and solve their problem? And we work backwards from there.
Greg Sterling (30:43)
And you're inferring that from the prompts.
Garrett French (30:49)
We would infer that from customer service logs, interviews with subject matter experts, conversations with subject matter experts.
Valerie Cecil (30:55)
To generate the prompts.
Garrett French (30:57)
We absolutely generate prompts, and we use AI to create — well, we've been calling them tracking-worthy prompts, or just prompts, but they're really at the core of our practice right now.
Valerie Cecil (31:06)
Or final user prompts.
Garrett French (31:09)
But it has to embody the situation. It has to bring context. I'm talking like 300-word prompts — some maybe not that long, but fifty-, hundred-word prompts — that really get us situated in a role within the industry, within a problem they're having, for which the offering of the client is a possible solution.
Mike Blumenthal (31:32)
And do you look at that across all of the chatbots, or do you give special emphasis to the ones that generate the most — well, how do you do this in practice?
Garrett French (31:38)
With APIs. ChatGPT, Google. We are building tools constantly and adding new ones in. Where do we stand, guys? I don't know the answer. What are we drawing from, James?
James Wirth (31:58)
Google Gemini, AI Mode, AI Overviews, ChatGPT calls. We're doing some scraping to see what the difference is between API responses and what we get off the front end. We're experimenting constantly, but that's primarily what we're focused on.
Mike Blumenthal (32:13)
And how many queries do you need to do to get a sample that indicates a direction on any given service?
James Wirth (32:23)
We're level-setting on that as well. It's a little bit all over the place right now. For the ones we're most actively searching for, we're measuring a cohort of prompts, generally ten or more, and we're running that daily over time. That's most commonly what we're looking at.
Greg Sterling (32:38)
For each client?
Garrett French (32:41)
No, for a given offering from a client. They might have fifty or a hundred things for sale. We're really trying to land in the offering itself, what's for sale.
Greg Sterling (32:50)
So it's product, it's buyer persona, and then all the different inputs that you previously described.
Garrett French (33:00)
Yeah. Customer service logs. How do we model the problem? Sorry, go ahead, James.
James Wirth (33:04)
And a really big one too — this is the other type of inference that we're really honing in on — is how we're considering a brand. Greg, to your point earlier, we were talking about brand and the shifting importance of that. Before, it was my own subjective understanding of a brand, its reputation as it applied to me, as I'm clicking through maybe one to four of the ten blue links and exploring the results. Now it's AI's understanding of that brand. Before, I would rely on my understanding. Now we're relying on AI's understanding. Does it have reputation measured accurately? What does the sentiment look like? What's the framing accuracy of different attributes or features or claims?
So that's another really big shift with AI: AI is telling me the reputation of that brand, and whether it's applicable to me, with whatever qualifiers or context or subtext it's going to infuse into that conversation. It's much more important now, across all of this content that it can pull together — way more than I would ever do in any individual search I would have done in traditional searching. It's going to pull all this together and then infer some sort of value from that for me. So what Garrett was talking about, too, that's really important to all the research we're doing, is sentiment and framing.
Greg Sterling (34:27)
This is great information, but why don't we bring it down to a concrete client example, where you can talk about how you're implementing, or how you're making changes, based on this kind of methodology. Give us a case study.
Mike Blumenthal (34:40)
And how you're measuring outcome.
James Wirth (34:45)
Yeah, that's great. Okay, so Valerie, I want to tag-team this with you if I can, and just talk anonymously about —
Garrett French (34:51)
What, me? You don't want to tag-team with me?
Valerie Cecil (34:54)
No, we're good. You can take a break.
James Wirth (34:58)
No, your participation was inferred, just like we're talking about here.
So a given scenario is a client with mixed sentiment and gross inaccuracies in how they're represented. Clear bias being inserted into the conversations around them, and just incorrect information contributing to how an AI answer is built — or, beyond a single AI answer, how an AI conversation runs — based on outdated misinformation that AI is pulling from its various sources when it runs its query fan-outs and does the grounding based on what is returned through Bing and Google responses. Then it builds its initial answer. Then I qualify the conversation, and it continues to provide additional information. It refines, it infers, et cetera.
How we're measuring that is across six layers of AI visibility. It starts with the traditional SERP; we call that "returned." Then whether it's read, or retrieved. Then, does it get cited? Is it mentioned? Is it recommended? And then the final layer, which kind of goes across all of those, is: what is the framing? We've hinted at some of these as we've been having this conversation.
So we run this initial analysis. Right now we're calling it a visibility probe, across these six layers, and we're looking for where their visibility is and what the gaps are. We're simulating the prompts. We're running those query fan-outs. We're seeing who's returned. Is the AI reading those? Is it citing those results? Is it mentioning the brand? Et cetera.
Once we identify where those gaps are, that's where the work begins. We're looking at: are there specific query fan-outs being run that they have no presence in? Okay, that would be a gap. Let's fill that gap, so that they can be represented in the grounding layer and returned in those query fan-outs. Then they have a much better chance of being cited, although it's not exclusively pulled from there. If they're cited, they have a much better chance of being mentioned. If they're mentioned, they have a much better chance of being recommended. You get where it goes.
So the work starts after we have that view. Sometimes it's client-directed. Sometimes they're coming to us and saying, what do we do? We're losing visibility. How do we fix it? And we're doing the analysis. But that's the foundation for when we start to introduce services. Those services could be traditional link building, if they're not showing up in that initial result set. It might be that the framing is wrong; it's inaccurate information, so we need to get more information out there. That could be a mix of getting insertions on existing publishers that are currently being cited. Maybe it's spinning up our own web content, off-domain web content, in order to fill those gaps. And then we build the service offering around that. Is there a local component? Are they really weak in local markets? Is that a priority? Et cetera.
I give whatever I can in that analysis. The prospect or the client says, yes, that looks good. And then I just sort of vomit it all on Valerie, and she has to figure out what we're actually going to build for the client at that point. That's why this is a collaboration, with Garrett giving us the leadership and the direction.
Garrett French (38:15)
That's why Valerie's here. She knows what we really, actually do. She's the only one.
James Wirth (38:19)
Exactly. I'm doing the analysis and then tying it back in the impact reporting, and then it all goes to Valerie to make the magic.
Greg Sterling (38:27)
I'm going to push you a little further. Give us an example of a case where there was incomplete or inaccurate brand information or client information. What did you specifically do, and did it work? Just give a capsule summary of that.
James Wirth (38:43)
Do you want to talk about that, Valerie?
Valerie Cecil (38:44)
You could start with the case that you're describing, and I could take it from there, if you want to start with just one example.
James Wirth (38:49)
Sure. So it's a combination. Traditional link building. We're also doing community PR. We're doing a lot of off-domain content generation, so we're spinning up microsites, effectively, because we're getting all the placements we can on publishers that are getting cited. A lot of those are closed systems, so we have some level of success there, but it's limited. So we're creating our own content in order to get the descriptors, the attributes, the framing —
Garrett French (39:22)
It could be something like this — excuse me, James. Because Greg's a lawyer, man. He's about to jump in. I can just sense it. Look at him. He's about to talk now.
Greg Sterling (39:35)
No, no, no.
Garrett French (39:36)
I knew it. No, I'm just kidding.
Valerie Cecil (39:38)
To correct you, because you're wrong.
Garrett French (39:41)
Okay. Example: a client that changes their service — sorry, James.
James Wirth (39:47)
My example.
Valerie Cecil (39:47)
I know. He didn't let you finish.
Garrett French (39:50)
I didn't let him finish. A client that changes their services frequently, and what a particular service does. They're adjusting their packages constantly. These change a lot, and they want to make sure that that information stays accurate.
James Wirth (40:05)
And it's not, and we're measuring that it's not. That's another good example.
Valerie Cecil (40:08)
We can see where it falls off.
Garrett French (40:10)
So we say, all right: if somebody searches this, or if they're using ChatGPT or AI Mode, it's not going to be able to get the right information, or it's going to assume the wrong information. And this is in a space where we do see a lot of affiliate-style websites getting cited that are doing comparisons.
Valerie Cecil (40:30)
Affiliate-style websites. It's a comparison.
Garrett French (40:35)
Right. So it's old comparative matrices, but they're factually inaccurate comparisons. The facts are wrong on these pages, and they can't get these affiliate-type websites to update. So this is one approach: we go and say what's factually accurate now, and then we can update those pages as needed. Because when they change their packages, they want to have the right information out there. They're doing this for commercial impact. They're responding to the market in some way, or they've got a new addition or benefit to their particular service.
This has been a place where we have made changes that we can track. We can say, when we made this change, now the accurate information — our websites are being cited. The sites that we want to have cited are being cited. They're being mentioned, in some cases. And then yes, we can see that now the framing is different, or that the facts are accurate now. So it's sort of correcting. It's one of my favorite things in the world. We get to correct.
Valerie Cecil (41:41)
Page retrieval patterns. Sorry, Greg, just really quickly.
Greg Sterling (41:44)
No, go ahead.
Valerie Cecil (41:46)
James, I would love to mention how we're watching page retrieval patterns as well, as early signals that yes, we're on the right track with the narrative expansion, let's call it. We see a gap here: we're not talking about a very important part of the story that would help us get mentioned more, or mentioned in different ways, or mentioned about something that just isn't surfacing.
We're watching how pages are being considered, essentially, and we are finding those to be early signals of what James is talking about, all along those six layers. If we're not being considered at all at the beginning, how could we be expected to be mentioned in the end? I think that's a really important piece that we don't hear people talking a lot about: looking specifically at those page patterns. And we're trying to mimic the good and leave out the bad, essentially, in terms of FAQs, comparisons, graphs, the data, the statistics, all of those things. We're trying to pick all of — chunkifying.
James Wirth (42:45)
Chunkifying it in a way that makes it easy, yes. I'm not saying go rewrite all your content so that an LLM can absorb it at the expense of the human. But there are clear patterns. If I'm making a concise argument — presenting my legal case in front of a judge or a jury — and I'm being concise and framing that appropriately, that's beneficial to the LLMs.
Interestingly, we're also seeing them ignore owned-domain content and messaging, at the expense of the brand, in favor of third-party published content.
Garrett French (43:19)
Yeah. This was useful.
James Wirth (43:21)
In some cases — to one of the scenarios Garrett was talking about — we did an analysis in the telecommunications industry, on something that changes very frequently. The AI model, which was not Google, was running its query fan-outs through Bing. Bing had not updated its index. It had cached a page that had since been updated, months prior, and it had not reflected that.
Garrett French (43:54)
Yeah. This is wild.
James Wirth (43:56)
The LLM was not retrieving the page. It was just going on the snippet in the result. And it was inaccurate information. That resulted in the telecommunications company I was doing the analysis for having inaccurate framing around pricing, around their offering at that time, around expiration dates, et cetera.
So this is the scenario where it's inaccurate, outdated information. The LLMs are trying to retrieve as much information as quickly as they possibly can. A lot of times they're not vetting it. They might be inferring a lot from that information, and we end up with a very inaccurate, outdated result that may not represent or frame the brand correctly.
Mike Blumenthal (44:37)
So this process requires all of the traditional SEO skills: content, link building, microsites, all of these things. It's a very dynamic process. I guess the question is, one, does this work ever get integrated into the core knowledge of the AIs —
James Wirth (44:53)
In the training set.
Mike Blumenthal (44:54)
— in the training set, so this work can then become more stable? Or does it require this constant, dynamic approach?
James Wirth (45:04)
Or a hybrid, which is what we're seeing. Yes, they are updating their knowledge about a given brand or an offering, not necessarily because it's getting inserted into the training set, because they don't necessarily have to do that. It doesn't happen very quickly, and it's very expensive. So the hybrid model — and we're already seeing shifts in this — is the training set plus any sort of search query fan-out that they do. That's the more recent model. But then, currently, they're caching that information and relying on their own cached version of the web, or they're building their own index, and it's becoming a hybrid of all of these things. So yes, they pull from the training set, they maybe run some new query fan-outs, they use that to maybe validate what they already have, and then they build their answer. What were you saying, Garrett?
Garrett French (45:58)
If we see query fan-outs, that means the training data is insufficient to answer the question. That's my hunch.
Mike Blumenthal (46:05)
And then how does this vary between ChatGPT, Claude, Google Gemini and Google AI Mode?
Garrett French (46:10)
What a great question. James?
James Wirth (46:12)
We don't necessarily have the same view across all platforms, so we're having to do our own inference to some degree. But we have all kinds of signals. One thing that's very interesting: the AI model is skipping the non-brand search. They already understand the players in the market. They're running site-colon search operators in their query fan-outs and returning specific pages that they knew about, or specific things they're looking for on a given brand's website, in order to update the training set, update their own cache, look for any new information.
Garrett French (46:49)
We saw that spike in ChatGPT. Valerie, when was that? We were just talking to Luke about it.
Valerie Cecil (46:53)
It was early August? April.
Garrett French (46:56)
Was it April? It was out of the blue.
James Wirth (46:59)
It's been happening more and more. It was a model change.
Garrett French (47:01)
If you watch the query fan-outs, though, you get a sense of what's —
Valerie Cecil (47:04)
You get a lot. And also dot-govs being favored a lot more, we've seen, especially in certain industries.
Garrett French (47:12)
In the query fan-outs.
Mike Blumenthal (47:12)
And how dynamic is this? In other words, how often do you have to update your processes to catch up with these new approaches: which sites are not being looked at, which new sets of sites are being looked at, like the dot-gov you mentioned, Valerie?
Garrett French (47:17)
We don't know.
James Wirth (47:26)
Well, nothing is settled yet, Mike. Everything we're seeing is still the wild, wild west. It changes from one LLM to another, one model to another. As a paid user, I might be looking at one result set or answer. The free versions, which are the majority of the activity on a given platform, are looking at something different. So we're trying to measure the models within a given LLM that are being most widely seen and used. But then those models update pretty frequently, as we all see.
So that's another challenge: nothing is settled to the point where we could say, okay, yes, we figured this one out; now they've introduced a new one; let's see what's changed. Instead, across the multiple LLMs and models, it's a constant measure, refine, adjust, measure.
Greg Sterling (48:19)
How do you account for personalization, explicit personalization? For example, I've told ChatGPT to never show me a shopping result from eBay. Never. And it doesn't.
Garrett French (48:31)
What do you have against eBay?
Greg Sterling (48:32)
There are things that it learns over time about preferences and responds to, and things that are explicit instructions like that. How are you accounting for that in your model?
Garrett French (48:41)
I want to talk about this, and then James can fill in any gap that's left. We can't know how people are searching, or what their personal biases are about eBay, for example. But we can make strong assumptions about an ICP, about a role or a persona within the ICP that's trying to solve a business problem. And we can infer or learn from our clients, from their customer service logs and what people complain about, what a perfect-scenario informational environment would look like. We don't know. We can't know that. But we can build into that with prompts.
So we're not going to personalize. We can't. But we can professionalize. We can say, this type of role would need this type of information to make the best possible decision about X, to get the best possible outcome for themselves. Maybe they need a certain kind of help with diagnostics: is this really a problem? Because if the CFO is the one doing the prompting, they're going to want a different framework, and they're going to evaluate a circumstance with a different set of tools or a different approach. So we're really trying to ask: in a perfect-information setting, what does this person require?
We back into that literally from what's for sale. We say what's for sale, and we ask questions with the client or do our own research. We come back and say, hey, does this sound right? Great. Nobody knows. When you get down to the granular level we're talking about, most marketing managers, no offense, aren't necessarily familiar with a particular component of an industrial computer, let's say. They don't know what it does. Why is it valuable? Why is it useful? They don't necessarily know that information.
James Wirth (50:39)
There's a proxy, though, Garrett. And Garrett's amazing at prompt generation. One of our key differentiators is that we built our own AI visibility tools, so we're not limited to being really careful about maximum numbers of prompts that we can run, and that sort of thing. We have this amazing testing ground for us to do a lot of the research. And we're using proxies for that personalization. I think that's probably the way we're doing most of that personalization.
Garrett French (51:06)
Thank you. That was a much faster way to say it, James. Thank you so much.
James Wirth (51:10)
We can infuse that personalization based on what we see in the visibility probe. We can infuse that into the custom prompts that we're able to generate, and then how we track —
Garrett French (51:19)
We can't solve for Greg. But we can solve for what Greg might need to know in a perfect setting. Sorry, Greg, it's only available on eBay, buddy.
James Wirth (51:30)
Unless Greg's our ICP, and the client says we really need to solve for Greg in our simulated prompts. Then that's how we could do that.
Garrett French (51:32)
There you go. Greg only. It's going to be a tough sell, but let's try it.
Mike Blumenthal (51:42)
So in your modeling for success and outcome, how do you view the relationship, the percentage influence, of the various AI chats in this outcome? AI Mode, Gemini, AI Overview, ChatGPT, et cetera. How do you weight those in terms of importance? Because they're going to give you different answers. They're going to give you different input and require different input.
Greg Sterling (52:08)
And they rely on different citations, obviously.
Mike Blumenthal (52:13)
Or maybe they don't, but —
James Wirth (52:14)
Well, they're going to give us different answers. And they change those citations from one run of the simulation to the next as well. So we're measuring that in aggregate. We're leaning into where the visibility losses are, because that does change by platform, based on the ones that we can measure.
But I would say, just as traditional SEO always over-indexed on Google — we're not going to make a change at the expense of our Google visibility just because we have less visibility on Bing, or Ecosia, which is using Bing, or something like that — so too in this environment we're primarily focused on Google: on what is happening in AIO, AI Mode, Gemini. I wouldn't necessarily make a recommendation for a gap that we're seeing in ChatGPT at the expense of what we're seeing in Google. But we get different information from measuring ChatGPT than we do from measuring Google. So we're relying on the aggregate view across the LLMs to show us where those gaps are, help us understand what the model's looking for, and then build to fill those gaps in. That's where the magic comes in.
And then we're measuring visibility. For example, we're measuring sentiment. When content that we've built is getting retrieved or cited, is that influencing the AI answer in a meaningful way? Does that shift the sentiment, which we're also measuring with our AI visibility tool? Are we increasing the number of mentions that we're getting? Are we improving the recommendations? Do we go from "yes, they're recommended, with a qualifier" that's maybe a subset of the general population, to that qualifier going away, and now they're in the top three recommendations in terms of the recommendation rank?
That's how we're measuring the success of the changes we're attempting to influence, the narrative we're attempting to inform throughout that engagement.
Mike Blumenthal (54:10)
For me, Greg, let me just say that this all raises the question of how you productize it. What skills do you need in your organization? How do you price this? It raises all of the questions in your second segment. If somebody wants to get into this field of SEO, marketing, whatever, what skills do they need? How does an agency develop all these skills? Are there tools, or do they have to develop their own? And how do they price?
Greg Sterling (54:40)
That's an excellent place to stop, and it tees up our next session, where we'll talk about the operational side of all of this, how you implement it, et cetera, as Mike has indicated. We've barely scratched the surface, but you guys have provided a lot of great insight, and we will bring you back for the second half of this, to talk about the agency approach, on our next podcast. So thank you.
Garrett French (55:05)
We can't wait. We appreciate you guys so much. The focus, the scrutiny, is very welcome. Uncomfortable sometimes, almost too personal at times, with car anecdotes and stuff. But no, we love it. We're grateful for the opportunity to talk through what we're doing. It is a new era. It's a new frontier. But here we are, and we're really making a go at it. So we appreciate the opportunity.
Greg Sterling (55:40)
Great. And everybody, thanks for spending time with us. Thanks, Valerie, James and Garrett. We will be back with the second half of this episode, to talk about the agency side, next week. Thank you.
Valerie Cecil (55:45)
Thank you so much.
Garrett French (55:52)
Thank you.
Interested in sponsoring this podcast or our newsletters please reach out to mblumenthal@nearmedia.co