Guide
How to track if ChatGPT recommends your product
In this article
In an afternoon you can build a small script that asks the AI engines the questions your customers ask, on a schedule. For every answer it records whether your product is named, which pages are cited as sources and who is named instead. It keeps every run, so you can see the history week by week.
Typing one question into the ChatGPT app tells you little. The same question can get a different answer an hour later, another engine can answer it differently, and the app's answer depends on your account and your past chats. One check is a single sample. A tracker gives you a rate you can compare over time.
- Use 10 to 30 questions written in your customers' words, and keep your brand name out of them.
- Ask each engine through its API with web search switched on, and save the answer text and the cited sources.
- Ask every question 3 to 5 times per engine and report a rate, such as named in 3 of 5 runs.
- Record a mention and a citation separately, keep every run in one table, and compare week on week.
Pick the questions your customers ask
Write 10 to 30 questions the way a customer would type them before they have heard of you. Use category phrasing ("best invoicing tool for freelancers") and problem phrasing ("how do I stop chasing late payments"). Leave your brand name out. A question that names you will usually get an answer that names you, and it tells you nothing about whether you are found.
The wording is already written down somewhere. Good places to look:
- Sales and onboarding calls: the sentence a customer used to describe the problem before they found you.
- Support tickets and emails, especially the first message from a new customer.
- Reddit and community threads where people ask for a recommendation in your category.
- Search Console queries that already bring people to your site.
Keep the list fixed once it starts running. If you change the questions every week, you can no longer compare one week with the next. Add a new question when your product changes, and note the date you added it.
How AI visibility is measured, explained →Ask each engine through its API
Each engine has an API that can search the web before it answers and return the sources it used. The examples below are TypeScript for Node 22, using plain fetch, so there is nothing to install apart from a way to run TypeScript, such as tsx. Each function takes a question and your API key. It returns the answer text and the cited URLs, or ok: false when the call fails, so a failed call never looks like an answer that leaves you out.
type Answer = { ok: true; text: string; cited: string[] } | { ok: false };ChatGPT: the OpenAI Responses API with web search
Call the Responses API with the web search tool, and set tool_choice so the model has to search. Without it the model may answer from memory and return no sources. The cited sources come back as url_citation annotations on the message content. The model name here is an example, so check OpenAI's current model list.
async function askChatGPT(question: string, apiKey: string): Promise<Answer> {
const res = await fetch("https://api.openai.com/v1/responses", {
method: "POST",
headers: {
"Content-Type": "application/json",
Authorization: `Bearer ${apiKey}`,
},
body: JSON.stringify({
model: "gpt-4o",
input: question,
tools: [{ type: "web_search_preview" }],
tool_choice: { type: "web_search_preview" },
}),
});
if (!res.ok) return { ok: false };
const data = await res.json();
if (!Array.isArray(data.output)) return { ok: false };
const parts = data.output
.filter((item) => item.type === "message")
.flatMap((item) => item.content);
const text = parts.map((part) => part.text ?? "").join("\n");
const cited = parts
.flatMap((part) => part.annotations ?? [])
.filter((a) => a.type === "url_citation")
.map((a) => a.url);
return { ok: true, text, cited };
}Gemini: generateContent with Google Search grounding
Call generateContent with the googleSearch tool. The sources come back in groundingMetadata.groundingChunks on the first candidate. Each source's uri is a Google redirect link, and its title usually holds the site name. The resolve helper below follows each link to the page it lands on, with a short timeout, and keeps the original link if that fails. Without it your own pages will never show as cited. Check Google's current model list for the model name.
async function resolve(url: string): Promise<string> {
try {
const res = await fetch(url, { redirect: "follow", signal: AbortSignal.timeout(5000) });
await res.body?.cancel();
return res.url || url;
} catch {
return url;
}
}
async function askGemini(question: string, apiKey: string): Promise<Answer> {
const model = "gemini-2.5-flash";
const url = `https://generativelanguage.googleapis.com/v1beta/models/${model}:generateContent`;
const res = await fetch(url, {
method: "POST",
headers: { "Content-Type": "application/json", "x-goog-api-key": apiKey },
body: JSON.stringify({
contents: [{ role: "user", parts: [{ text: question }] }],
tools: [{ googleSearch: {} }],
}),
});
if (!res.ok) return { ok: false };
const data = await res.json();
const answer = data.candidates?.[0];
const text = (answer?.content?.parts ?? []).map((p) => p.text ?? "").join("\n");
const links: string[] = (answer?.groundingMetadata?.groundingChunks ?? [])
.map((chunk) => chunk.web?.uri)
.filter(Boolean);
const cited = await Promise.all(links.map(resolve));
return { ok: true, text, cited };
}Perplexity: the chat completions API
Perplexity's Sonar models search on every request, so there is no tool to switch on. The sources come back in search_results, each with a url and a title. Older responses carried a plain citations array of URLs instead, so read that if search_results is missing or empty. Check Perplexity's current model list for the model name.
async function askPerplexity(question: string, apiKey: string): Promise<Answer> {
const res = await fetch("https://api.perplexity.ai/chat/completions", {
method: "POST",
headers: {
"Content-Type": "application/json",
Authorization: `Bearer ${apiKey}`,
},
body: JSON.stringify({
model: "sonar",
messages: [{ role: "user", content: question }],
}),
});
if (!res.ok) return { ok: false };
const data = await res.json();
const text = data.choices?.[0]?.message?.content ?? "";
const cited = data.search_results?.length
? data.search_results.map((r) => r.url)
: (data.citations ?? []);
return { ok: true, text, cited };
}Google AI Overviews: a search results data provider
Google has no first-party API for AI Overviews. To read them you need a search results data provider: a paid API that runs the Google search for you and returns the results page as JSON. Pick one whose response includes the AI Overview block, with its text and its list of cited references, and that can load an Overview Google renders after the page loads. Some questions get no Overview at all. Record those runs as no answer, which is a different result from an answer that leaves you out.
Ask every question more than once
Answers change from run to run and from engine to engine. The search step can return different pages each time, and each engine reads different sources. Our own study of 10 well-known companies showed how wide the spread can be. "Atlassian was named in 33% of its ChatGPT answers and 7% of its Google AI Overviews answers." And: "Railway ranged from 53% to 100% across engines."
We tested ten well-known companies in AI search →Ask each question 3 to 5 times per engine on each check. Report the result as a rate for each engine: named in 3 of 5 runs is 60%. A single yes or no from one run will swing from week to week even when nothing has changed, and a rate over several runs moves less.
Record a mention and a citation separately
A mention is your product named in the answer text. A citation is one of your pages linked as a source. They are separate measurements and they often disagree: an engine can recommend you while citing a review site, or cite your docs without naming you in the answer. Record both for every answer, along with the rivals named in your place.
Your own site is often a small share of the sources. In our scan of 10 little-known companies: "Own-domain citations were 0.82% for the unknown cohort against 2.63% for the household names, about a third rather than more." If you only count citations, you can miss the runs where you are named and the source is someone else's page.
We scanned ten companies nobody has heard of →Keep one row per run in a single table. The 2 rows below are an example to show the shape. They are not data.
| date | engine | question | run | status | named | cited | cited_urls | others_named |
|---|---|---|---|---|---|---|---|---|
| 2026-10-05 (example) | chatgpt | best invoicing tool for freelancers | 1 | ok | 1 | 0 | Rival A; Rival B | |
| 2026-10-05 (example) | perplexity | best invoicing tool for freelancers | 1 | ok | 1 | 1 | https://example.com/pricing | Rival A |
The function below fills one row. Named checks the answer text for your product's names. Cited checks whether any cited URL is on your domain. Others named checks the text against a list of rivals you keep. A failed call tells you nothing about whether you are named, so the row gets the status error and the rate leaves it out. A list only finds the rivals you already know about, so read a few answers by hand each month and add any new names you see.
import { appendFileSync, existsSync } from "node:fs";
const MY_NAMES = ["Example App", "ExampleApp"];
const MY_DOMAIN = "example.com";
const RIVALS = ["Rival A", "Rival B", "Rival C"];
const LOG = "ai-answers.csv";
function isMine(url: string): boolean {
try {
const host = new URL(url).hostname;
return host === MY_DOMAIN || host.endsWith("." + MY_DOMAIN);
} catch {
return false;
}
}
function record(engine: string, question: string, run: number, answer: Answer) {
const date = new Date().toISOString().slice(0, 10);
let row: (string | number)[] = [date, engine, question, run, "error", "", "", "", ""];
if (answer.ok) {
const text = answer.text.toLowerCase();
const named = MY_NAMES.some((name) => text.includes(name.toLowerCase()));
const ours = answer.cited.filter(isMine);
const others = RIVALS.filter((name) => text.includes(name.toLowerCase()));
row = [date, engine, question, run, "ok", named ? 1 : 0, ours.length > 0 ? 1 : 0, ours.join(" "), others.join("; ")];
}
if (!existsSync(LOG)) appendFileSync(LOG, "date,engine,question,run,status,named,cited,cited_urls,others_named\n");
appendFileSync(LOG, row.map((v) => `"${String(v).replaceAll('"', '""')}"`).join(",") + "\n");
}Put the functions above and the loop below in one file called track.mts, and run it with npx tsx track.mts. Keep each key in an environment variable on the machine that runs the script.
const QUESTIONS = [
"best invoicing tool for freelancers",
"how do I stop chasing late payments",
];
const RUNS = 3;
for (const question of QUESTIONS) {
for (let run = 1; run <= RUNS; run++) {
record("chatgpt", question, run, await askChatGPT(question, process.env.OPENAI_KEY!));
record("gemini", question, run, await askGemini(question, process.env.GEMINI_KEY!));
record("perplexity", question, run, await askPerplexity(question, process.env.PERPLEXITY_KEY!));
}
}Run it every week and keep the history
Weekly is enough for most products. Any scheduler works. A cron job on a machine that stays on is the simplest. A GitHub Actions schedule needs no machine of your own and can commit the CSV back to the repository after each run. A scheduled Claude Code run can execute the script and then summarise what changed since last week.
name: AI answer tracker
on:
schedule:
- cron: "0 7 * * 1"
workflow_dispatch:
permissions:
contents: write
jobs:
track:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 22
- run: npx tsx track.mts
env:
OPENAI_KEY: ${{ secrets.OPENAI_KEY }}
GEMINI_KEY: ${{ secrets.GEMINI_KEY }}
PERPLEXITY_KEY: ${{ secrets.PERPLEXITY_KEY }}
- run: |
git config user.name "AI answer tracker"
git config user.email "tracker@users.noreply.github.com"
git add ai-answers.csv
git commit -m "Weekly AI answer check" && git pushAppend every run to the same CSV, or load it into a SQLite table. One query then gives you the rate for each engine on each check date, leaving out the failed calls, and you can compare week on week:
SELECT date, engine,
ROUND(100.0 * AVG(named)) AS named_pct,
ROUND(100.0 * AVG(cited)) AS cited_pct
FROM answers
WHERE status = 'ok'
GROUP BY date, engine
ORDER BY date, engine;The cost scales with questions × engines × runs. Every question on every run is one call per engine, so 20 questions asked 3 times make 60 calls a week for each engine you track. Each provider charges for tokens, and the search step can carry its own charge on top. Check each provider's pricing page before you set the numbers, and start small.
What the tracker will not tell you
- Why you are absent. It shows that you are missing from an answer, and it does not show the reason.
- What to change. The answer is built from the sources behind it, so the next step is reading those sources and seeing who they name.
- What the consumer apps say. The apps can answer differently from the APIs, and nothing in this script measures them.
- Anything about Google AI Overviews unless you add a search results data provider, which is a separate source with its own cost.
Where AfterLaunch fits
AfterLaunch runs this check every week across ChatGPT, Gemini, Google AI Overviews and Perplexity and keeps the record. It reads the sources behind each answer alongside search, communities and directories, ranks the next move and drafts it. Its MCP server lets your own coding agent read the same record. It starts with a free scan, then a 14-day trial. You or your AI finish the work, and you approve what ships.
Connect AfterLaunch to your coding agent →Can I check this in the ChatGPT app instead?
You can, and it is worth doing now and then. A single check in the app is one sample, shaped by your account and your past chats, and it leaves no record. The API with web search gives you repeatable runs you can count. The 2 can differ, so treat the app as a spot check.
How many questions do I need?
10 to 30 is enough to start. Choose the questions your customers really ask, in their words, and cover the main problems your product solves. Fewer questions asked 3 to 5 times each tell you more than many questions asked once.
Does it cost much to run?
It scales with questions × engines × runs, and each provider prices tokens and search differently. Work out your weekly call count, check each provider's pricing page, and start with a small set of questions before you widen it.
Why do answers change between runs?
The search step can return different pages each time, and the model writes a new answer from them on every run. Engines also read different sources from each other. That is why the tracker asks each question several times and reports a rate.
Is Google AI Overviews included?
Not through a first-party API, because Google does not offer one. You need a search results data provider whose response includes the AI Overview text and its cited references, and some questions get no Overview at all.
The free scan asks the AI engines your customers' questions and shows where you are named, who is named instead and which sources they cite.
Scan my site