Identifying content gaps easily with your AI Agent gap report | AI Agent

The AI Agent gap report highlights possible gaps in the content you've provided to your AI Agent, allowing you to quickly identify and fill them so that your AI Agent has everything it needs to handle customer queries.

This help file will explain the gap report for your AI Agent.

Before you start

You must have the AI Assist add-on. See here for information on pricing.
To access the Activity log, you’ll need an Admin or Team lead account.

01 Introduction

The AI Agent uses the content you've provided to generate replies to your customer's queries. If the generated reply is not sent because a human agent considers it inaccurate or lacking information, or if a human agent sends a follow up reply with more accurate information, then eDesk compares both replies and identifies the gaps.

If you then close the gap by providing the AI Agent with the piece of information that's missing or more accurate, the AI Agent will be able to reply to similar queries in future with a high level of confidence, taking the load off your team and delivering 24/7 customer care.

So how do you find out if there are gaps in the information you provide to your AI Agent? It's all in the AI Agent gap report.

02 Gaps explained

The AI identifies gaps in messages where an AI Agent has generated a reply but a human agent has chosen to send their own reply that differs from the AI Agent's reply.
eDesk will identify gaps in the following:

Previews that were not used by the human agent.
Drafts that were not used by the human agent.
Auto replies that a human agent has cancelled before they are sent or sent a follow up message after the auto reply.

Note that the AI Agent replies that are sent to the customer - either as an Auto reply or a generated Draft/Preview that is then sent by a human Agent - are not checked for gaps.

Gaps are either of type Correction or Missing:

A Correction is where some information in the human agent's reply differs from the information in the AI Agent's reply, raising the possibility that the AI Agent's information is wrong.
For example, the Training content stated that the return processing fee was $5, but the human agent said $6 in their reply.
Missing is where the human agent's reply contains information that is not in the AI Agent's reply, suggesting that the AI Agent does not have access to that information in its Training content, Instructions, or Policies.

03 What's in the AI Agent gap report?

The report presents:

A Summary that includes:

total number of gaps that were identified.
number of Corrections.
number of Missing.

A table listing the gaps, with the following columns:

Training gaps
A list of topics in which gaps have been identified. These topics are generated by the AI based on the customer's messages in tickets. An example of a topic is Shipping delivery timing and special requirements. For each topic, it also shows the number of tickets the AI Agent generated a reply that contained gaps.
A maximum of 100 topics will shown at any one time.
Type
The type of gap (Correction or Missing).
Details
Shows if the gap(s) have been filled or not.
Total
The number of gaps identified.
Actions
A green tick. Select this to remove the entry from the report once you've filled the gap.

Select a row in the table to expand the view. This lists the tickets in which an AI Agent generated a reply that contained a gap relating to the topic:

Expanded list to show tickets in the topic

For each ticket in the topic, the following columns are displayed:

Time
Ticket
Ticket ID that links to the ticket in the Mailbox.
Mode
Draft, Preview or Auto reply
Reply type
An AI-generated summary of the type of reply provided.
Gap filled
This will update once the gap has been filled, based on analysis of tickets created since the gap was identified.
Feedback
This column is populated if a human agent has reviewed and left feedback on a reply generated by an AI Agent. Feedback is left in the Reply analysis window.
Action
Select the Details button to open the Reply analysis (see next section of this help file). The Reply analysis shows the human agent's reply and the AI Agent's generated reply, and highlights the differences between the two, as well as providing other details.

04 Reply analysis

The Reply analysis is a powerful tool that enables you to see how successful the AI Agent was in generating satisfactory answers and handling tickets. It provides in-depth details about the AI Agent's activity for a given message, and also includes the human agent's feedback and actions in that message.

The Reply analysis is described in full in the Reply analysis section of this help file.

05 Accessing the AI Agent gap report

1. Go to Settings > AI > AI Agent gap report
2. Perform any of the following:

Inspect the summary and topic information.
Select the green tick in the Actions column in the row for a topic to remove the entry from the report once you've filled the gaps. See next section for more information.
Select a topic to expand the view to show the tickets in which an AI Agent generated a reply that contained a gap relating to the topic.
In the expanded view, select a ticket ID to open that ticket in the Mailbox.
In the expanded view, select the Details button in the Action column to open the Reply analysis.
In the Reply analysis, enter Feedback for the AI Agent's generated reply.
In the Reply analysis, select View ticket to open the ticket.

06 Resolving a gap

Once you have reviewed the gap report, you can decide whether you need to amend or add anything in your Training content, Instructions or Policies.

Once the gaps have been filled or you are satisfied that the Training content, Instructions or Policies don't need editing, you can select the green tick in the Actions column to remove the item from the gap report. A Mark Gap as Resolved modal window will be displayed. Select Confirm.

Mark as Resolved modal

If you fill the gap but don't Mark the Gap as Resolved as per the instructions above, the next time the AI Agent generates a reply for a ticket in this topic, the AI will check if the gap still exists. If it no longer exists, then it will be marked as filled in the Details column.