Main image of article Building a Rule-Based Chatbot: What You Need to Know

Summary

Although we’ve seen an explosion of generative AI chatbots over the past few years, chatbots themselves have been around for much longer. However, those older, non-AI versions were rudimentary and predictable, usually based on a limited set of rules (“If the user asks about how to return a product, respond with exactly the following text…”). AI-based chatbots, on the other hand, can engage in remarkably human-like conversations, and pick up on context if they’re engineered correctly—but how can you build one?

Before we get into the nitty-gritty, it’s important to note the flip side of AI-powered chatbots: they’re also unpredictable at moments. Sometimes they say things the programmers didn’t expect. They can even go off the proverbial rails and say things that can get the company in trouble (or the developers at least; for example, they might start recommending a person go to a competitor for better products).

Because of those potential issues, many teams today are choosing to build hybrid chatbots: systems that combine the best of both worlds, from modern natural-language processing to the rules-based order of traditional chatbots. Let’s look at how you can build one in Python.

To build this type of chatbot, you’ll want to think in terms of layers, with each doing a specific job. Briefly, these are:

  • The Understanding Layer with Sentiment Deduction: This is where you use modern NLP (natural language processing) tools to understand not only what the user is asking, but their general sentiment. Are they happy? Frustrated? Confused? You’re also determining their intention (i.e., what they’re ultimately asking).

  • The Decision Layer: This is where the rules come in. We don’t want a chat to go off and come up with its own random ideas. Instead, we take the results of the previous layer and come up with a solid answer to their question or problem.

  • The Response Layer: Here we take the results of the decision, layer, and send it into an LLM along with the sentiment, and build a response that sounds natural. And the reason for including the sentiment is so that our chat can mimic a sense of empathy and compassion.

Let’s look at these in a bit more detail, along with the Python libraries and coding techniques needed.

In this layer, we gather up the meaning from the user’s input. Remember, users aren’t necessarily going to ask a question in perfect grammar, and they might not always say exactly what they mean. That’s where modern AI tools can really shine, pulling out the intended meaning and the sentiment.

Here we extract specific data points about the user’s question or comment, such as:

  • Intent: What do they hope to gain? Are they reporting an issue? Are they just sharing feedback?
  • Entities: This is the actual data they’re submitting, usually in key-value pairs. For example, they might be frustrated with a problem with their wifi and slow speed (if you’re building a chatbot for a telecom or modem-building firm, for instance). The key value pairs might be: service=wifi and issue=slow speed.
  • Sentiment: What is their mood? For example, they might be frustrated. Or they might be excited. Or just neutral.

To build this step, you’ll need different libraries for the different aspects, intent, entities, and sentiment.

For intent and entities, the following libraries can help:

  • spaCy: This is a modern Python library for natural language processing, great for tasks like part-of-speech tagging and entity recognition.
  • HuggingFace Transformers: This is a powerful Python library that provides access to pretrained transformer models.
  • Rasa NLU: This is an open-source natural language understanding library. It’s built specifically for chatbots and for determining intents and entities.

For sentiment analysis:

  • TextBlob: This is a beginner-friendly Python library for simple natural language processing tasks. It’s particularly good at sentiment analysis and part-of-speed tagging.
  • VADER (from NLTK): This is a specialized tool for sentiment analysis that’s tuned for social media and short text.
  • HuggingFace Transformers (again): This can also provide sentiment analysis as well as intent and entities.

For example, if the user enters something like, “I want to cancel my appointment for tomorrow.” you might come up with the following results:

  • Intent: Cancel appointment
  • Entities: Date=Tomorrow
  • Sentiment: Neutral

But if instead they enter, “Why is my internet down again! This is the third time this week!” You might get:

  • Intent: Report issue
  • Entities: Service=Internet and issue count=3
  • Sentiment: Frustrated

From there, you can then feed the three items into the rules engine. We’ll look at that next.

The decision layer takes the output of the understanding layer and feeds it into the rules to build a decision on how to proceed next. Ideally, the decision layer will generate a response, which is fed to the response layer, and the user will be satisfied.

Realistically, however, this may well be multiple steps, because a final decision might not be available yet. For example, if the user wants to cancel their appointment, you’ll likely need to ask them when they would like to reschedule.

Pro tip: This can result in frustration for the user, in which case you’ll want to provide them with an easy way to speak to a human directly, either through the chat app or through a phone call. For example, if you ask them when they want to reschedule, and they provide a date, but there are no appointments available that day, you don’t want to keep asking them over and over for a different date. Instead, you might need a human to intervene.

Regardless, the decision layer will apply the rules and the information it has so far, and build a response (even if the response is a question asking for more information). Here you apply business rules and what’s called “conversation logic” to come up with the next response. Although there are modern tools to help, this tends to be less AI and more logic-based.

Here are some approaches you can take:

  • Simple rules engines: These are libraries that help you construct rules and outcomes. A couple choices are:
    • Durable Rules: This is a lightweight Python library that lets you define your conditions and the actions to take.
    • business-rules: This is a Python library that lets you set up a list of rules and a list of actions.
  • State Machines: These are traditional Finite State Machine (FSM) libraries whereby you specify a fixed set of states, and the rules for transitioning from one state to a next. One popular such library is called transitions.
  • Workflow Libraries: These tend to be higher-level libraries built for specifying workflows. They can be useful here where you list a set of steps or processes to take, one followed by the next. One popular such library is SpiffWorkflow. Note that this library is very high level and what its developers consider “low code.” That can work either for or against you, depending on how much control you want over the specific steps. Use your best judgement.

After the decision layer performs an iteration, it feeds its data into the response layer. The output of the decision layer will likely be a sentence such as, “Try rebooting your router” or, “What date would you like to reschedule for?” Theoretically, these responses are perfectly fine. If you worded them well, you can simply print them out and ask for the user’s next input.

However, with modern AI tools, you can do much better. You can take the user’s sentiment and original question, then feed that into an LLM, along with the response. You might introduce the following into the LLM:

The user asked: “Why is my internet down again! This is the third time this week!” We detected their sentiment is “frustration”. Can you craft a compassionate response around “Try rebooting the router” while factoring in their original question and sentiment?

To try this out, we fed the preceding paragraph into ChatGPT. Here’s what it produced as a viable response to the user:

“I’m really sorry you’re dealing with this again — I can understand how frustrating it must be, especially after it’s happened multiple times this week. Let’s start by trying a quick router reboot, which often helps resolve connection issues. I’m here to help walk you through it if you need!”

Notice how it acknowledges their emotion, references the situation, delivers an action, and offers further support. All from ChatGPT! That’s a pretty good response to issue back to the user.

Of course, the user might respond with something like, “I tried that and it didn’t work!” But then we would repeat the process, analyzing their continued sentiment, and so on.

Notice that, to test out the response, we simply typed what we needed into ChatGPT, no coding necessary. From Python, you have some options.

The easiest is to simply use a library like OpenAI, which lets you send a prompt like the one we built, and feed it into ChatGPT and get back a response, all without having to do any AI on your end. This is an ideal and easy way to do it… except it costs money. If you have a busy shop with lots of people asking for online help, you can end up with a pretty hefty bill that you owe OpenAI.

Instead, you could run the AI yourself. However, you need a decent GPU to be able to do the LLM processing, which is the real work of AI. But if you do have access to such resources, there are several Python libraries you can put to work:

  • Transformers by HuggingFace: We’ve already mentioned this one, and it’s quickly becoming the de facto standard for doing LLM AI locally on your own machine. The library is high-level and quite easy to use; you just specify a model, request a generator, which is really a function, and then call the generator, passing in your prompt. It will then give you back a generated response similar to the way ChatGPT did.
  • llama.cpp: This is a C++ library with Python bindings, allowing you to call it right from your Python app, without having to write any C++ code. The process is nearly identical to what we just described with the Transformers library.
  • LangChain: This one is a bit more sophisticated, but powerful, as well. LangChain is a high-level Python framework that connects to models and adds conversation management on top. You can build entire workflows, chaining together prompts and responses. While powerful, it takes a little more work to get up and running. (Note that the company that makes LangChain offers both community open-source products, as well as premium products.)

As we mentioned earlier, most likely this won’t be a single response and the user is set. In most cases, you’ll need more information, or the user might want to provide further detail (such as “Rebooting didn’t solve the problem.”). From there, it’s a balance of sticking to your rules engine, all while not letting the LLM take off and start suggesting crazy stuff. You’ll get back a response from the user and then return to the first step, analyzing it again for sentiment, intent, and so on. Then it’s back to the rules.

But this begs the question: How do we know when we’re done?

Essentially, as you’re building the rules, you’ll provide certain “end” rules that detect that all required actions are complete. In the case of the wifi problem, these might be, for example:

  • The user states that the wifi is working again without problem.
  • The user is getting nowhere and asks to speak to a human.

Once the user reaches one of these states, the chatbot will complete it, either by providing a thank you message (which could be generated by the LLM and based on the combined sentiments of the user inputs) or by explaining that a live representative will join the conversation shortly (or “call this number to speak to a human”, etc.).

This kind of app can be fun to build, as it brings together many different technologies. Interestingly, the rules part (which is decidedly not AI) is really the most difficult part. You need to come up with as many different scenarios as possible and work them all in, typically in a sequence. This is where early testing matters, because you want to predict as many scenarios as possible, and not leave the system stuck, unsure of how to proceed.

The result will be a great tool that the users will hopefully be satisfied with, as it will provide empathy and a natural human-like fluency, all while staying within the boundaries of the rules. Ready to try this out yourself? We recommend sticking to something simple like an appointment scheduler. Start small, focusing on just a couple of clear rules. Soon you’ll have it up and running!