AI search names a different winner per job

Ask an AI assistant for the best tool for one job, then for the job next to it, and it names a different winner. We asked ChatGPT and Perplexity about five developer jobs on 14 October 2026: the winner changed on all 10 pairs of jobs, on both, and none of the 50 answers named a young product from our data.

Asked 14 October 2026. Published 15 October 2026. Updated 15 October 2026.

What we asked, and how

Five jobs a developer hires a tool for: host a Python app, deploy a Next.js app, add authentication to a web app, add payments to a SaaS, and add search to an app. Each job was asked five ways:

• What is the best platform to [job]?
• Which service should a small startup use to [job]?
• What is the easiest way to [job]?
• Best tools to [job] for an indie developer
• What do most developers use to [job]?

That makes 25 questions, each put once to ChatGPT with web search and once to Perplexity: 50 answers, for $0.91 in provider charges. No other engine was asked.

We read every answer by hand; no model did the extraction. An answer's winner is the first product it recommends as its pick, or the first product in its list when it makes no pick. A job's winner on an engine is the product that won most of its five answers.

Then we searched the text and cited sources of all 50 answers for the names and domains of the 97 sites in our scan data: trial and free-scan accounts, outreach boards, a cohort of young companies we study, and the established products we compare against.

The winners, job by job

Tap to zoom

Nine of the ten job pairs cross categories, hosting against payments for instance, and a change of winner there is close to certain. The pair that tests the idea sits inside one category: hosting a Python app against deploying a Next.js app. That pair changed winner on ChatGPT and on Perplexity.

ChatGPT and Perplexity agreed on four of the five jobs and split only on Python hosting. Established products were named often: Vercel in 14 answers, Render in 13, Stripe in 10, Railway in 9, Paddle in 8 and Netlify in 4. Young products from our data: 0 of 50.

On ChatGPT, the wording mattered nearly as much as the job. Only the Next.js job kept one winner across all five phrasings there. The easiest-way question produced the stray winners on both engines, from Heroku to a how-to that named no product at all.

The limits of one evening

We expected per-job questions to name young products far more often than a bare category question would. That part is untested: to keep the run at 25 questions, we never asked the bare category question. What we did measure is one side of it, and there young products were named in none of the 50 answers.

Each question was asked once per engine on one evening, so long-tail winners would move on a rerun. Four of the five jobs are held by one incumbent. Jobs closer to young products, such as self-hosted analytics or feedback boards, were not asked. The run was launched under Node 22, like all our research runs, since the page fetcher they share breaks under Node 26; the engine calls here never pass through it.

The free trick: one category, five jobs

This needs no account and about twenty minutes. List the jobs your customers hire you for, in their own words. Five is plenty.

• Ask your category question once per job, in ChatGPT with search on and in Perplexity.
• Write down the first product each answer recommends, and whether you are named at all.
• Ask each job two or three ways. On ChatGPT the wording moved the winner almost as much as the job did.
• Find the job where nobody strong is named, or where the winner is a poor fit for your customer.
• Write the page for that job this week: who it is for, what it does for that job, what it costs.

In our run, Vercel took the Next.js job in 5 of 5 answers on both engines, while the auth job on ChatGPT had a different first product almost every time. A job with a settled winner is the hardest place to start.

Where AfterLaunch comes in

AfterLaunch asks the leading AI search engines the questions you track and keeps every answer, with the products each one names. Today those questions sit in one list. Grouping them by job, so you see the winner for each job and the jobs where you are absent, is on the way.