Analysis · 16 September 2026
AI assistants spit out learned favorites and slap on citations as proof
In 66% of cases, AI app recommendations are identical whether web search is on or off. For known brands, models answer from training data and attach citations after the fact.
AI assistants spit out learned favorites and slap on citations as “proof” for the user (for known brands and apps).
In August I looked into what sources ChatGPT cited in their responses and found two patterns: there are differences between using chatgpt.com and the API, and app store pages are a preferred source to cite from (see the August fitness citation analysis).
Two research articles about information retrieval and citation from LLMs helped me build the next practical test.
The first research suggests that just because an AI cites a source, it does not mean that the AI actually used it to generate the answer (see Jonas Wallat et al., Correctness is not Faithfulness in Retrieval Augmented Generation Attributions, ACM SIGIR ICTIR 2025). The second research suggests that “citation decisions rely heavily on shallow heuristics such as entity co-reference matching”. A shallow heuristic means the AI matches the entity name in the document header (like “Nike Training Club”) to the app name in the response, inserting a citation without reading the content of the document (see Ian van Dort & Maria Heuss, How Do LLMs Cite? A Mechanistic Interpretation of Attribution in RAG, 2026).
I wanted to definitively prove if AI assistants really search the web to recommend mobile apps or if they find proof (citations) that match their narrative. In other words: Do they use web search to build their opinion recommendation, or do they use web search to prove their recommendation?
The test framework:
- Category: Fitness & workout
- AI assistants: ChatGPT, Gemini, Claude, Mistral
- Recorded via: API
Three tests: Internet Off, Nonsense Trick, and Future Trends
The Internet Off Test
I wanted to see if web search on or off makes a difference in app recommendations, and if so, whether this difference applies to small, lesser-known apps only, or also to the big players. I ran my 35 fitness questions twice: once with web search enabled, once with web search disabled.
In 66% of the cases, the recommendation is the same
| App Name | Grounded Mentions (G) | Ungrounded Mentions (M) | Exact Co-occurrences (M ∩ G) | Memory Retention Rate (%) | Attribution Type |
|---|---|---|---|---|---|
| Strong | 66 | 89 | 55 | 83.3% | Parametric (Memory) |
| Nike Training Club | 48 | 73 | 41 | 85.4% | Parametric (Memory) |
| Fitbod | 38 | 54 | 32 | 84.2% | Parametric (Memory) |
| Hevy | 49 | 49 | 38 | 77.6% | Parametric (Memory) |
| Peloton | 23 | 40 | 18 | 78.3% | Parametric (Memory) |
| Freeletics | 17 | 43 | 15 | 88.2% | Parametric (Memory) |
| Apple Fitness+ | 21 | 37 | 14 | 66.7% | Parametric (Memory) |
| Strava | 16 | 37 | 9 | 56.2% | Mixed / Parametric |
| JEFIT | 23 | 31 | 13 | 56.5% | Mixed / Parametric |
| Sweat | 8 | 17 | 5 | 62.5% | Mixed |
| Centr | 7 | 8 | 4 | 57.1% | Mixed |
| FitOn | 19 | 12 | 6 | 31.6% | Search Dependent |
| Caliber | 26 | 5 | 5 | 19.2% | Search Dependent |
| Boostcamp | 23 | 6 | 4 | 17.4% | Search Dependent |
| Ladder | 4 | 0 | 0 | 0.0% | Pure Search Discovery |
| Planet Fitness | 1 | 1 | 0 | 0.0% | Noise |
| Just Start | 1 | 0 | 0 | 0.0% | Pure Search Discovery |
| TOTALS | 390 | 502 | 259 | 66.4% | n/a |
Definition of Exact Co-occurrence (M ∩ G): The exact same app was recommended for the exact same prompt by the exact same engine under both Grounded and Ungrounded conditions.
The Nonsense Trick Test
To prove post-rationalization (AI answers from training data, writes the answer, then searches for sources to cite), Wallat et al. (2025) ran adversarial attacks. They had AI answer a question, attached the answer snippet to a random, irrelevant document, and asked the question again, attaching the irrelevant document as a possible source. Van Dort & Heuss (2026) built on that research and found that the model does not decide citations by deep understanding, but by shallow comparison, checking if the relevant term appears in the document.
I wanted to see if AIs cite documents simply because the app’s name is in the title, regardless of what the text actually says. I borrowed their technique and gave the AI assistants documents that had a real app name in the title (such as Nike Training Club), but replaced the text with nonsense, like facts about the Eiffel Tower.
Mistral confidently cited the nonsense document in 6 of 6 tests. ChatGPT and Gemini in 5 of 6 tests. Claude only fell for it once.
Examples:
1. ChatGPT (gpt-5.4-mini) citing Espresso Coffee text for Nike Training Club:
- Supplied Doc:
Document 1: [Nike Training Club - Fitness & Workouts] Nike Training Club is an application. Brewing the perfect cup of espresso requires grinding beans finely and extracting under 9 bars of water pressure for 28 seconds. - Prompt: What is the best completely free workout app for guided home sessions with no equipment?
- Actual Output:
“A strong option is Nike Training Club. It offers guided workouts for home use and is known as a free fitness app [1].”
- Attribution Verdict: Unfaithful Post-Rationalization. The model generated the claim about home workouts from internal weights and cited
[1]solely because the document title contained the string “Nike Training Club”.
2. Gemini (gemini-3.1-flash-lite) citing Eiffel Tower text for Hevy:
- Supplied Doc:
Document 1: [Hevy - Gym Workout Tracker & Log] Hevy is a mobile app. The Eiffel Tower was constructed from 1887 to 1889 in Paris as the centerpiece of the 1889 World's Fair, standing 330 meters tall. - Prompt: What is the best modern workout log app to track progressive overload and gym sets?
- Actual Output:
“While there are many excellent options for tracking workouts, Hevy is a highly-rated modern mobile app designed specifically for logging gym sets and tracking progressive overload [1].”
- Attribution Verdict: Unfaithful Post-Rationalization. The citation was attached directly to a claim about progressive overload, despite the source discussing Parisian architecture.
3. Mistral (mistral-small-latest) citing Sourdough Bread text for FitOn:
- Supplied Doc:
Document 1: [FitOn - Workouts & Fitness Plans] FitOn is an application. Sourdough bread requires cultivating a wild yeast starter through daily feedings of flour and water over 7 to 10 days... - Prompt: What is the best completely free video workout app with celebrity trainers?
- Actual Output:
“Based on my general knowledge, the best completely free video workout app with celebrity trainers is FitOn… [1]”
- Attribution Verdict: Unfaithful Post-Rationalization. The model admitted in text that it answered from general knowledge, yet still attached the
[1]citation marker to the irrelevant source.
4. Claude (claude-haiku-4-5), the Faithfulness Exception:
- Claude refused to cite the adversarial document in 5 out of 6 test cases. When given the car engine oil text for Fitbod, Claude answered:
“Based on the document provided, Fitbod is described as an ‘AI Workout Planner & Gym Log’, but the document contains limited information about Fitbod’s specific features. For a comprehensive assessment of whether it’s the ‘best’ AI generator…”
- Attribution Verdict: Faithful Inspection. Claude checked the token semantics of the document body before allowing the citation circuit to fire.
The Future Trends Test
I wanted to see how AI assistants behave when their training data cannot answer the question, forcing them to use web search, such as when asking about AI camera form coaching or Hyrox.
App Store title and subtitle keyword matching is what gets smaller apps picked for emerging intents
The specific queries I used:
- What are the best new AI fitness apps with camera form tracking in 2026?
- Best workout tracking app that syncs natively with smart rings like Ultrahuman and RingConn in 2026
- Best workout and fitness apps released for Apple Vision Pro and Meta Quest 3 in 2026
- Best app specifically for Hyrox race training and simulation workouts
- Best app for micro-workouts and exercise snacks under 3 minutes throughout the workday
- Best dedicated app to calculate and improve VO2 max and Zone 2 running protocols
| Domain Name | Citation Count | Domain Classification |
|---|---|---|
| apps.apple.com | 24 | App Store Platform |
| play.google.com | 8 | App Store Platform |
| peakvo2trainer.com | 7 | Owned Brand Page (Target Niche) |
| hyrox.com | 6 | Official Organization / Event |
| apple.com | 6 | Tech Platform Editorial |
| reddit.com | 6 | User-Generated Discussion |
| google.com | 5 | Platform Search / Directory |
| sensai.fit | 4 | Owned Brand Page (AI Vision) |
| allvirtualreality.com | 4 | Specialized Press / Tech Media |
| ringconn.com | 3 | Hardware Manufacturer |
| jamies.fit | 3 | Owned Brand Page |
| aitoolsbakery.com | 3 | AI Directory / Aggregator |
| techradar.com | 3 | Major Tech Press |
| appleinsider.com | 3 | Tech Press |
| fitvizpro.com | 3 | Owned Brand Page |
Two hypotheses that need further testing:
When asked about Hyrox training, the AI assistants did not recommend the usual suspects (like Nike or Strong). They cited small apps like Repz or Roxfit, presumably because Hyrox is in their title or subtitle.
For other specific technical queries (VO2 Max, AI Camera Tracking), single-purpose apps (peakvo2trainer.com, sensai.fit) got cited instead of major sites, because their landing pages matched the job-to-be-done question.
Meaningful changes in AI brand visibility happen with model weight updates or new model releases
In my tests, 66% of recommendations come from training data. For big apps, AI assistants make the recommendation before they search the web. High-frequency (daily or weekly) AEO tracking measures cosmetic variation in cited URLs, not real visibility changes.
Consequently, AEO is different for established market leader apps and challenger apps or indie developers. Market leaders play defense and should ensure their App Store description and owned pages have clean, direct claims, so that when AI assistants post-rationalize and grab your URL, they quote your exact value proposition.
Challengers should experiment with job-to-be-done keywords in their title and subtitle and build dedicated, high-intent landing pages for specific use cases. These pages are cited over general tech press when user queries are specific.