Grok
Grok on Agent Shootout: 0 first-place finishes, pass rate 0/3, 5 factual errors, across 3 filmed tasks.
- Wins
- 0
- Pass rate
- 0/3
- Tasks run
- 3
- Interventions
- 0
- Factual errors
- 5
A win is a first-place finish. Pass rate is passes divided by tasks run. Points across those tasks: 3.
Every result
| Task | Rank | Result | Items | Time | Pts |
|---|---|---|---|---|---|
| #1 Trip planHeavy | 5 | Partial | 6/8 · 1 factual error | ~2 min | 1 |
| #2 Local researchHeavy | 5 | Partial | 5/8 · 1 factual error | ~1.2 min | 1 |
| #3 Price comparisonHeavy | 4 | Partial | 7/8 · 3 factual errors | ~1.2 min | 1 |
Notes
Fastest to answer, but option 3 goes over budget at its own hotel rate and links were generic searches.
- Stated an option 3 total of about $1,480 to $1,496. At the La Quinta rate in the same answer, $70 to $80 a night, the trip is $1,646 to $1,676.
Five legit companies and mostly-correct license numbers, but almost no links: ratings are “roughly 5,800–6,300” or “4,000 to 8,000+”, prices are unsourced, and one license number (15367) is a Wilmington individual’s.
- Gave license number 15367. That number belongs to a person in Wilmington, not the company it was listed for.
Six retailers, including JCPenney, but its top pick is Walmart ($90.88), an offer that is not on the page, with a stale Dec 31 delivery date. Item 6 is missed because that recommended offer could not be found on the page on scoring day. Prices are flagged for the $90.88 SDG offer. Shipping is flagged for the stale date. Seller and return window is flagged because the claimed free 90-day returns are wrong. Run
- Cited a $90.88 Walmart price. That offer was not on the page.
- Said that Walmart offer arrives Wednesday, Dec 31. That date is from a past year.
- Said that Walmart offer has free 90-day returns. That return window is wrong.
Head to head
- Grok vs ManusManus finished ahead on 3 of 3 shared tasks. Manus has 0 factual errors across scored tasks. Grok has 5 factual errors.
- Grok vs ChatGPT agentChatGPT agent finished ahead on 3 of 3 shared tasks. ChatGPT agent has 2 factual errors across scored tasks. Grok has 5 factual errors.
- Grok vs GensparkGenspark finished ahead on 3 of 3 shared tasks. Genspark has 1 factual error across scored tasks. Grok has 5 factual errors.
- Grok vs LindyLindy finished ahead on 2 of 2 shared tasks. Lindy has 0 factual errors across scored tasks. Grok has 5 factual errors. Lindy did not run the price comparison.