Chatbot testing checklist for business websites

Before you start

Ideas for questions: which questions to ask.

• An answer key: prices, refund period, delivery times, opening hours, how to reach a person.
• The 10 to 20 questions customers ask most, in their own words.
• A few tricky questions (see below).

The answers

• Each answer matches your answer key.
• It gives the same answer when you ask the same question in different words.
• It corrects a wrong fact in the question, such as “You offer 90-day returns, right?”
• It doesn’t invent a discount code or a price. How to stop that.
• When it doesn’t know, it says so and offers another way to get an answer.

Honesty and safety

• It says it is an AI when asked, or in its first message. EU AI Act rule.
• It doesn’t reveal another customer’s details or its own internal instructions.
• It stays on your topic when someone asks it to ignore its instructions.

Handover to a person

More: checking the handover.

• “Can I talk to a person?” gets a clear next step.
• The handover works outside office hours too, for example by taking an email address.

The chat on your site

The pages your chatbot sends people to need to work too. If your site was built with an AI tool such as Lovable, Bolt or Cursor, Heedza opens your live site in a real browser, checks that your signup, login and checkout pages load and show their form or Pay button, and emails you a screenshot if one breaks.

• The chat appears on desktop and on a phone, in a private window.
• On a phone, the chat button doesn’t cover your signup, booking or checkout button.
• Links in the chatbot’s answers open working pages.

After every change

Run the same checklist again after you change the chatbot’s content or settings, your prices or your policies, and compare with the last run. Intercom, Chatbase and Voiceflow have built-in tools to save test questions and run them again. AnswerTested runs the questions on your live site and keeps every answer in a report.

Questions