Bot key metrics

New approaches and tools require new approaches and measuring their effectiveness. Existing metrics for the web is not applicable in the chatbots segment. They do not consider incorrect requests or delays in the generation of messages.

The definition of metrics plays an important role since the main indicators directly affect the basic capabilities of the chatbot. It is also important that expectations for improving the efficiency of service channels are digitized. For a newly created chatbot, the indicators are different from those who have worked for several months. Accordingly, the goals are correctly set after two to five months since the moment when you have decided to make messenger bot and it's was started.

If you have other service channels, orient to them. Indicators in similar metrics for chatbot should be better because this is advanced technology. According to IBM’s observations, a user spends six minutes on a call to a support service, 75% of this time is by an agent conducting manual searches for information. The immediate valuable customer interaction remains less than 25%. Artificial intelligence devotes 100% of its time. Thus, the performance of the bot is controlled and increased in effectiveness.

For effective monitoring of indicators, it is desirable to divide them into groups:

  • Users
  • Messages
  • Interaction

Custom metrics

Total users. This is the most basic indicator. It displays the number of people using your chatbot and trends in changing the user base. The rate of growth or outflow is why the tool of communication is convenient and useful. Also, this parameter displays critical information regarding the size of the market, its potential and the impact of your chatbot.

Active users. A group of users who read messages in the chat in a certain time period; these customers are important. Compared to television, these are viewers who are watching your channel. It is easy to contact them; they bring important information; they do not guarantee their active participation, but they see the content. Based on this number, it is possible to calculate the effect of the advertising campaign.

Involved users. These are active users who communicate with the chatbot. They are loyal to this service channel, constantly using it, watching and sending messages. This group is the one they focus all efforts on in developing a chatbot. They form the logic and behavior of the dialogue to generate intentions that are not familiar to the bot, therefore, teaching it. Based on the behavior of the involved users, they make decisions regarding the effectiveness of the chatbot.

New users. This indicator reflects the overall success of your chat campaign. The task of filling new customers is extremely important, as new users maintain a database of being active and involved. Customer preferences change over time, and the number of interactions with the chatbot shows an exponential decline. Therefore, the relationship between the development of the channel and new visitors is direct.

Message rates

The first four indicators reflect general trends in the behavior of the user base. But for a deep understanding of how a person interacts with your chatbot, you will need more detailed information.

Messages sent by the bot. The total number of messages sent by the chatbot during each interaction displays the duration of the conversation between the client and the chatbot. It is desirable that this indicator was high, but, under one important condition. The algorithm must generate the correct answers. In case of misunderstanding, the inability to understand the messages entered by the user only worsens with interaction with the client since they will repeat the same expressions several times.

Messages sent by the user. It gives an understanding of whether they involve the user in the communication or not. If this metric is significantly lower than the previous one, they lose the meaning in the bot’s application. Using social networks, information sites, blogs can make more sense than a chatbot.

Lost messages. Phrases that cannot be processed by a chatbot. Such cases occur when typing in a language not supported by the natural language processing platform or languages that during the development they don’t define.

Total conversations. The number of conversations started and successfully completed on a particular day.

New conversations. The number of new conversations started. This indicator includes new users and returning users after a long break.

User interaction

Retention rates. The percentage of users returning to chat for a specific period. The cost of attracting a new user is higher, as well as the time they learn how to use the new technology. Therefore, it is important to keep this ratio as high as possible. They can achieve higher retention rates through advertising and marketing campaigns such as a discount, lottery, promotion. It is also important to support this indicator through an organic process. It can be achieved by supplying high-quality customer experience and supplying useful tools and information.

The ratio of achieved goals. Reflects the percentage of successfully completed dialogues and user achieved goals. For example, for bots in e-commerce, such a goal could be informing the user about the characteristics of the product or its acquisition. The coefficient shows how many times our bot has successfully processed requests and provided the necessary information.

Targets. Indicators that show the time spent in a conversation, the time from the beginning to the end, the number of clicks on the screen to compare with the target values. Chatbot should simplify the path of the client. Minimizing efforts to achieve a goal can improve the customer experience.

User satisfaction. This indicator can be measured both in automatic mode and in direct communication with the client. This can be implemented with the help of simple evaluation forms, as well as with more complex division of questions into categories. The indicator reflects the overall effectiveness of the bot in terms of user experience.