Skip to content
Agent Shootout

Latest · Shootout #3

ChatGPT agent won the Ninja air fryer price hunt.

8/8 · 1 factual error in 1.3 min. 4 agents, one prompt, filmed and scored .

  1. 1ChatGPT agentPass · 8/8 · 1 factual error · 1.3 min
  2. 2GensparkPass · 8/8 · 1 factual error · ~5 min
  3. 3ManusPartial · 7/8 · 0 factual errors · 1.9 min
See the boardRead Shootout #3

Get the weekly results

One letter when the week’s films and scores are in.

The address is sent to Kit, which delivers the letter. The confirmation page says “You're in! Watch your inbox for the first shootout.” Unsubscribe in any edition.

On video

A short loop from Shootout #2. The film is the evidence.

  • Shootout #2, local research. ChatGPT agent, Manus, a map of Raleigh HVAC companies, and Lindy, on the same prompt.

Scored shootouts

  1. Shootout #3 ·

    Price comparison

    ChatGPT agent

    Pass · 8/8 · 1 factual error · 1.3 min

  2. Shootout #2 ·

    Local research

    Manus

    Pass · 8/8 · 0 factual errors · ~3.9 min

  3. Shootout #1 ·

    Trip plan

    Manus

    Pass · 8/8 · 0 factual errors · ~1.5 min

How it works

One public comparison, repeated. The prompt is written once. The session is fresh. The film is the score.

  1. 01

    Same prompt

    The task is written once and pasted word for word. No agent gets a rewritten brief.

  2. 02

    Fresh session

    Each run starts clean. No leftover chat, no memory, no instructions carried over.

  3. 03

    On video

    The screen is recorded. One labeled run is the score. That film is the evidence.

  4. 04

    Same week

    Every agent in the table runs inside one week. The letter goes out with the scores.

Scored tasks

Leaderboard

Rank on each task page is points, then items met, then fewest factual errors, then time.

Scored tasks for five agents: trip plan, local research, and price comparison. Row order and the time, interventions, and cost columns are the trip plan. Factual errors are the total across scored tasks. Each task page has its own ranking.
AgentTrip planLocal researchPrice comparisonFactual errorsInterventionsTimeCost
ManusPass · 2Pass · 2Partial · 10 factual errors0~1.5 min$0 · Free
ChatGPT agentPass · 2Partial · 1Pass · 22 factual errors112–16 min$0 · ChatGPT Plus
GensparkPartial · 1Partial · 1Pass · 21 factual error0~9 min$0 · Free
LindyPartial · 1Partial · 1Did not run: free trial expired, needs paid plan0 factual errors0~5 min$0 · Free trial
GrokPartial · 1Partial · 1Partial · 15 factual errors0~2 min$0 · SuperGrok

Time, interventions, and cost on this table are the trip plan. Each task page has its own time and rank.

Scoring rulesLeaderboardAgents

Factual errors are the total across scored tasks. A checklist miss is something left out. A factual error is a claim we checked and found wrong.How we score.

Scroll sideways for every task and metric.