Automatic replies
Automatic replies are the point of the product and the setting most worth being careful with. A wrong answer sent automatically is worse than no answer at all, because the customer now believes something untrue and has stopped asking.
The good news is that the defaults are safe and the path to widening is mechanical.
The three levels
Automation is set per classification, not per workspace, so different kinds of question can run at different levels at the same time.
Manual. Neither automatic reply nor draft is on. The assistant still classifies, still summarizes, and still answers Draft reply and Find answer when someone asks. It just never writes unprompted.
Draft only. The assistant writes the answer and leaves it in the editor.
The conversation carries a Draft badge and appears in the AI Queue. Nothing
reaches the customer until a person presses Send.
Automatic reply. The assistant writes and sends. Optionally it also closes the conversation.
Roll it out in that order
For each category:
- Start manual. Let it classify for a week and check that the classification is right. Automation on top of wrong routing is worse than no automation.
- Switch to draft only. Read the drafts for a week or two. You are asking one question: would I have sent this?
- Turn on automatic reply once the answer has been yes consistently, and keep reading a sample.
The step people skip is the second one, and it is the one that tells you whether your knowledge base is good enough for this category. Drafts cost you nothing to be wrong.
Which categories first
Start where the answer is a fact you have written down, the cost of being wrong is a follow-up email, and the volume is real.
Good first candidates: delivery times, opening hours, return policy, product specifications, "where is my invoice".
Bad first candidates: anything about money leaving your account, anything involving a defect, a complaint, or a person who is already upset. Those categories are worth leaving on draft indefinitely.
The confidence threshold
Every classification carries a confidence. Below the workspace threshold, no automatic reply and no automatic close happen, whatever the category says.
This is a floor, not a preference. It means a message the assistant is unsure about never gets an automatic answer, even in a fully automated category. The conversation lands in the queue instead.
If you find automatic replies going out on messages that were misunderstood, raising the threshold is the first lever to reach for, before turning anything off.
Automatic close
Automatic close ends the conversation after the assistant answers. It is right when there is nothing to follow up: feedback, thanks, spam.
It is wrong wherever the customer might reasonably reply, because a closed conversation that reopens on their reply looks fine in the data and feels dismissive in the mailbox. A customer replying "that doesn't answer my question" to a closed conversation reopens it, so nothing is lost — but it did tell them you considered the matter finished.
Hand-over
When the knowledge base does not cover the question, the assistant does not answer. It escalates: the conversation goes to high priority, no reply is sent, and the reason is recorded.
This is the behaviour that makes automatic replies safe to run at all. It is worth checking that it works before you widen: ask your own widget something your knowledge base genuinely does not cover, and confirm the conversation lands in the queue rather than receiving a confident invention.
What to watch
After turning a category on, watch three things for a fortnight:
The corrections. Conversations where a person wrote again after an automatic reply are the honest measure. A handful is normal; a pattern means the underlying article is wrong or ambiguous.
The reopen rate. Conversations closed automatically and reopened by the customer mean the answer did not land.
Sentiment after an AI reply. A negative message immediately after an automatic answer is the clearest failure signal there is.
Reviewing quality covers where to find these.