Testing a Conversational AI (Chatbots) or How To test a Chatbot or ChatBot Testing 101

Chatbots are one of the most widely adopted AI/ML implementations in the business sector.

A chatbot is an intelligent machine used to imitate human conversation through text and voice commands. Bots are widely used as a personal assistant, customer service, HR, sales and marketing to name a few. Theyare everywhere and we rely on them to a certain extent, this makes it extremely important to assure the quality of the chatbots and test them thoroughly. They are built using NLU/NLP-Services (Natural language understanding and processing) and are subjected to constant training and improvement which has direct impact on tests.

Chatbots are a new kind of apps that require new kind of tools to perform effective and high-coverage testing —a goal that cannot be reached with slow, real-time, flaky end-2-end tests.

Last year was dominated by the smart devices and voice-based home assistants. These use the conversational interfaces unlike other application to interact with. They are built using advanced algorithms, ranging from pattern and expression matching engines to natural language processing and AI/Machine learning techniques. These systems are constantly learning by themselves improving the interactions with the user bringing up the challenge in the testing world of non-deterministic output. To such interfaces, natural language is the input and we, humans really love having alternatives and love our synonyms and our expressions using emoji’s gifs and pictures. Testing in this context moves to clouds of probabilities

Currently chatbots, artificial intelligence and machine learning are driving the industry. The hypegenerated around Amazon’s Alexa or Apple’s Siri or Google Home are only a few examples. They depend on speech recognition technologies. As the chatbots user do not have any barriers and due to the unpredictable user’s behaviour it becomes utmost difficult to verify the correctness on the output.Based on customer proximity and the enormous number of people a chatbot could reach, the quality assurancein this area is becoming very important.

Device fragmentation, many different NLP platforms and chatbot provider are the test challengeswe face while testing a chatbot. Botium is there to solve all these problems. It is the upcoming industry standard for chatbot test automation and quality assurance. Developers and operators get a fully integrated continuous chatbot test infrastructure that seamlessly fits into existing CD pipelines and processes.

Therefore As a part of this talk we would be covering the following

1. What is a chatbot? Types of chatbots
2. How ChatBots are different from the other Application?
3. Latest market response and customer feedback
4. Industries which use chatbots and its benefits
5. Comparison between industry benefits before and after using chatbots
6. What should be tested while testing chatbots?
7. Types of testing can be done on chatbot
8. Testing Strategies for ChatBots
9. Challenges
10. Testing Strategies for ChatBots
11. Demo

Chatbots are poised to exponentially grow in use and the technology to build them will rapidly evolve. Quality from a usability and functionality standpoint is a must and will be key for success.