Understanding your AI Agent activity log | AI Agent

The AI Agent activity log provides insights that allow you to monitor and optimise your AI Agents' performance.

This help file will explain the activity log for your AI Agent.

Before you start

You must have the AI Assist add-on. See here for information on pricing.
To access the Activity log, you’ll need an Admin or Team lead account.

01 Introduction

The AI Agent delivers incredible benefits to your business, providing round-the-clock support to your customers and buying time for your busy team to focus on other critical tasks. To allow you to measure its value and also ensure it is performing at its very best, eDesk provides an AI Agent activity log that enables you to see exactly what your AI Agent has been doing during the selected time period.

02 What's in the AI Agent activity log

The AI Agent activity log is a list of tickets to which the AI Agent has contributed.
This includes:

Drafts created by the AI Agent
Previews created by the AI Agent
Auto replies created by the AI Agent.

The Log is displayed in table form, with the following columns:

Time
Ticket
Ticket ID
Mode
Draft, Preview or Auto reply
Reply type
An AI-generated summary of the type of reply provided.
Gaps
The number of information gaps identified for each message. These are gaps in the information provided to the AI Agent that would improve future replies if added. Learn more about what kind of information is provided to AI Agents here.
Feedback
This column is populated if a human agent has reviewed and left feedback on a message generated by an AI Agent in the ticket.

Note: If the AI Agent has contributed to multiple messages in a single ticket, then this ticket will appear multiple times in the listing, with one entry per message.

03 Accessing the AI Agent activity log

1. Go to Settings > AI > AI Agent activity log
2. If required, use the fields at the top to filter the list of tickets by:

Date range
Ticket ID
Mode
The type of contribution, which is Draft, Preview, or Auto reply.
Reply type
A summary of the message provided by the AI Agent.
Has Training gaps
If selected, tickets with gaps will be listed. Add this missing information to improve future replies.
Has agent feedback
If selected, tickets with human agent feedback will be listed. This feedback can be left on a specific message (via the message item or via the activity log).

3. Click the row to see the Reply analysis.

04 Reply analysis

The Reply analysis is a powerful tool that enables you to see how successful the AI Agent was in generating satisfactory answers and handling tickets. It provides in-depth details about the AI Agent's activity for a given message, and also includes the human agent's feedback and actions in that message.

In Settings > AI Agent activity log, select a row to open a modal window that contains the Reply analysis for that message.

Note:

The Reply analysis might identify more than one gap in the information for a single message.
There may be multiple Reply analysis entries for a single ticket depending on the Reply modes, information available from the order/ticket, the gaps found, if feedback has been left or not, etc

Examples of Reply analysis

In the example screenshots below, red text are annotations that explain the content on screen.

Example 1:

A draft was created by the AI Agent, and the human agent then customised it or wrote their own reply.
Based on this, gaps were identified based on the difference between the AI Agent's generated reply and the human agent’s reply.

Example 2

The AI Agent sent an Auto reply.
No gaps were identified.
The Training Content found and the content actually used is displayed.
A human agent left feedback on the AI Agent’s reply before it sent.

Example 3

The AI Agent sent an Auto reply.
It was a generic answer so no gaps were identified, no content was found or used and no feedback was left by a human agent.