Why "Press Zero"?

How scores are calculated

Bot Gap Index (0–100, higher = bigger gap). For each rating we take every dimension the person scored and turn it into a gap: (5 − score) ÷ 4. Dimensions that person said matter most to them (their top 3) count twice. We average those into one number per rating, then average across all ratings for the bot and multiply by 100.

Failure rate. A dimension "fails" in a rating if it was scored 1 or 2. "Didn't come up" is ignored.

Importance. The share of all respondents who picked that dimension in their top 3.

Crowd Gap Score. Importance × failure rate, so problems that are both common and important rise to the top.

Small samples. We only show a bot's scores after 10 ratings, and we show a 95% confidence interval next to every index.

Quality. Very fast submissions and repeat ratings of the same bot within 7 days are flagged for review. Everything is crowd-reported and not verified by the companies.

Data sources: Reddit

Alongside the survey, we code public Reddit posts about customer-service bots and AI assistants. Reddit is a separate data source: it is never mixed into survey totals, the Bot Gap Index, or any "% of respondents" figure. We never store usernames, and public pages only show combined numbers and team-written paraphrases — never raw post text.

Where the data came from

Public Reddit posts from … subreddits, collected for odd-numbered months only (January, March, May and so on) from … to …, then filtered by keywords to posts about AI bots. … posts about customer-service bots and … posts about AI assistants were coded.

How posts were coded

Posts were coded with keyword rules ('press-zero-rules-v1'), not by an AI model reading them. A person reviewed a sample, and some posts were corrected or excluded by reviewers.

Agreement between the rules and human reviewers: …

"Stated only" and "Stated + assumptions"

Some values are educated assumptions, for example a company inferred from the subreddit, or an outcome guessed from wording. 'Stated only' (the default) counts a post only if the post itself says it. 'Stated + assumptions' includes the guesses.

How percentages work

Each percentage is the share of all posts in that group that count in the chosen mode. The page shows how many posts each figure rests on.

Limits

Reddit posts lean negative, so compare companies and patterns, not absolute levels. Sarcasm can be coded wrong. Some 'couldn't reach a person' posts are people asking how to reach one. Many posts do not say how the problem ended. Posts only mention a bot in passing in some cases.

Reddit-implied survey answers

To compare Reddit with the survey question by question, we derive an implied answer to each survey question from each coded post. These are estimates, not responses, and each one carries a basis: stated (the post said it directly), assumed (we inferred it from other cues), or not inferable (left blank — we never guess). The rules, in plain language:

• Which bot: the bot named in the post, or the company it was about.
• What they were trying to do: the coded task (billing, cancel or refund, tech problem, account access, change plan or order, schedule, question, complaint). For AI assistants, the coded use case instead (coding, writing, research, images, work, study, personal support).
• Did it solve the problem: from the coded outcome — solved, partly solved, or not solved / gave up / went elsewhere.
• Repeat or rephrase: "yes" only when the post mentions being made to repeat, being misunderstood, forgotten context, or being stuck in a loop. Silence never counts as "no".
• Under 5 minutes: "no" only when the post mentions long waits, too many steps, loops, or high effort. Otherwise left blank.
• Talking to a person: from the coded human-access outcome; if that's unknown but the post says they couldn't reach a human, we count "never reached" as an assumption. Not applicable for AI assistants.
• Preferred a person: assumed "yes" only when they never reached one or had no option; assumed "no" when they didn't need one. Otherwise blank.
• Satisfied overall: from the post's sentiment (positive, negative, mixed). Neutral sentiment is left blank.
• Would use again: never inferred — always blank.
• Last 3 months: from the post's month.

For these implied survey answers only, every percentage is computed only over posts where that question could be told, and the denominator is always shown. Rows a team member has hand-corrected or confirmed are never overwritten by these rules. Reddit posts over-represent strong, mostly negative opinions — compare patterns and rankings, not raw levels.

Privacy

Your email is stored separately from your answers, is only visible to the research team, and is never shown publicly. We use it to recognize returning participants, send occasional research updates, and credit you if you opt in.

Public pages only show combined statistics and quotes the team has reviewed and stripped of personal details. Screenshots are private and never published. There are no ads and no tracking for marketing.

Request deletion

Email vrao019@ucr.edu from the address you used, and we'll delete your data.