Analysis · 4 November 2026
Different models from the same AI (ChatGPT) recommend different apps.
Comparing gpt-5.4-mini to gpt-5.6-luna shows that Luna decides differently and cites different sites.
Data and method: 172 answers across the categories, plus a second run of the 35 fitness prompts. Plus the same fitness prompts with search off. All with gpt-5.4-mini which was validcube’s default ChatGPT model until October and with gpt-5.6-luna which is the new default from November onwards. Luna is also ChatGPT’s default model for free users on chatgpt.com and their app (as of October 2nd).
The big question in the analysis articles so far is about how AI Assistants search the web, how that influences the apps they mention and recommend (can it be influenced?), or if they answer from memory and only slap on citations as proof of their answer.
| All five categories, 3 November 2026 | Luna | Mini |
|---|---|---|
| Answers that searched the web | 145 of 172 | 113 of 172 |
| Search queries per answer that searched | 4.8 | 2.3 |
| Answers with more than one round of searching | 58 | 5 |
| First-round search queries naming an app the question didn't name | 246 of 510 (48%) | 43 of 244 (18%) |
| Recommended apps that were already named in a first-round query | 241 of 336 (72%) | 75 of 250 (30%) |
| Queries limited to one website ("site:") | 355, in 94 answers | 21, in 9 answers |
| ... of those, on the app maker's own website | 279 | 17 |
| Queries containing the word "official" | 498, in 131 answers | 166, in 90 answers |
This table shows that the search behavior changed quite drastically between the two models from OpenAI. Luna uses search about 1.3 times as much as Mini, and searches more often per answer. In the November run, Luna had 58 answers with more than one search run, Mini had 5. One signal to find out if a model answers from memory, is to check if it uses the specific app name in its search query that wasn’t named in the prompt. Luna did this in 48% of its first search queries. Mini did this as well, but only in 18%.
This is different to Claude and Mistral who never named a brand in their first search query in this sample:
| All five categories, 3 November 2026 | Answers that searched the web | Search queries per answer that searched | First-round queries naming an app the question didn't name |
|---|---|---|---|
| ChatGPT (Luna) | 145 of 172 | 4.8 | 246 of 510 (48%) |
| Claude (Haiku 4.5) | 123 of 172 | 1.1 | 0 of 123 (0%) |
| Gemini (3.1 Flash-Lite) | 138 of 172 | 3.3 | not measurable (no query order) |
| Mistral (Small) | 132 of 172 | 1.1 | 0 of 132 (0%) |
Owned pages and listicles up, press pages down
| Share of cited links, all five categories | Luna, 3 Nov | Mini, 3 Nov | Mini, 2 Oct |
|---|---|---|---|
| App maker’s own website | 56% | 46% | 52% |
| Press and test sites | 11% | 26% | 28% |
| Listicles (affiliate rankings, vendor “best apps” pages) | 13% | 9% | 4% |
| App stores | 8% | 15% | 10% |
| Government, research | 8% | 1% | 1% |
| Reddit, YouTube, forums | 0% | 0% | 0% |
| Other | 5% | 3% | 6% |
| Cited links (each link counted once per answer) | 461 | 337 | 338 |
With Luna, the share of citations from press pages drops from 26% to 11% while owned pages (brand/company websites) increased slightly from 46% to 56%. Overall, Luna retrieves 3x as many pages compared to Mini, but cites relatively less. In absolute terms it’s still 1.4x more. An interesting finding is that Luna retrieves a lot of results from Reddit, but never cites it in its answers.
Websites that gained and lost the most citations
| Website | Type | Mini | Luna | Luna minus Mini |
|---|---|---|---|---|
| help.openai.com | own site (OpenAI) | 12 | 25 | |
| consumer.ftc.gov | official | 0 | 12 | |
| support.google.com | platform help | 3 | 10 | |
| helpx.adobe.com | own site (Adobe) | 5 | 12 | |
| help.fitbod.me | own site (Fitbod) | 1 | 7 | |
| mullvad.net | own site (Mullvad) | 2 | 8 | |
| itechguides.com | listicle | 2 | 7 | |
| apple.com | platform | 3 | 8 | |
| pimsleur.com | own site (Pimsleur) | 0 | 4 | |
| duolingo.com | own site (Duolingo) | 5 | 0 | |
| healthline.com | press | 5 | 0 | |
| techcrunch.com | press | 5 | 0 | |
| androidauthority.com | press | 6 | 0 | |
| tomsguide.com | press | 23 | 11 | |
| apps.apple.com | app store | 47 | 28 | |
| techradar.com | press | 33 | 13 |
In August, the analysis piece titled “ChatGPT loves to cite from press sites, the App Store and owned pages” and named the App Store, tomsguide.com and healthline.com the top 3 sites. healthline wasn’t cited once this time by Luna, tomsguide lost 12 and the App Store lost 19 citations compared to the Mini run on the same day. However, this decline can’t be attributed to the model switch in November alone, because both healthline and tomsguide already fell in September (on Mini, in fitness). On Luna, the App Store is the most cited URL in the run, but the distance is shrinking across all five categories. In the fitness category, where Fitbod is the most mentioned app, Fitbod’s own pages now beat the App Store listings (11 vs. 9).
Memory still dominates app mentions
Switching search off and on made no clear difference in what apps got mentioned (for Luna, in fitness). The variation is about the same when just asking the same question multiple times. 85% of the apps that Luna mentioned with search on also appeared when search was off.
If you want to dive deeper into Luna-Land, read these 20 Findings written up by AI.