Our goal at DeepPavlov is to enable anyone to build their very own assistant. We offer a range of components that can be easily utilized as they are or customized according to your needs to build a personalized assistant. In this tutorial, we will guide you through the process of creating a simple and light-weight bot that specializes in discussing movies and answering factoid questions. Our approach will only utilize the existing Dream components, without introducing any new ones.
Let’s take a closer look at the parts that make up Dream before we start developing our bot. As you can see in the figure below, our pipeline architecture consists of the following components :
- Annotators perform NLU preprocessing on an utterance;
- Skill Selector chooses the relevant skills that can provide the next bot’s response;
- Skills are in charge of generating candidate responses;
- Candidate Annotators perform NLU postprocessing on the candidate responses;
- Response Selector selects the most suitable response from the generated candidates;
- Response Annotators carry out post-processing of the bot’s response;
- Dialogue State stores all the information about the current dialog including all annotations and meta-data.
What do we need?
Now that we are familiar with the Dream architecture, let’s choose the components we need for our future bot. The complete list of available components can be found here. For our bot, we will need the following components:
- Sentseg – allows us to handle long and complex user’s utterances by splitting them into sentences and recovering punctuation if it is missing;
- Combined Classification light-weight — annotator for topic classification, dialog acts classification, sentiment, toxicity, emotion, factoid classification;
- Entity Detection — extracts entities and their types from utterances;
- Entity Linking — finds Wikidata Knowledge Graph’s entity identifiers for the entities detected with Entity Detection;
- Wiki Parser — extracts Wikidata triplets for the entities detected with Entity Linking.
- Movie Skill — conducts scripted conversations related to movies;
- Factoid QA — answers factoid questions using Knowledge Graph’s annotations;
- Program Y Skill — an AIML-based skill that responds to some common questions and requests using pre-written answers;
- Dummy Skill — a skill with multiple non-toxic candidate fallback responses in case other skills fail to provide a response.
- Combined Classification light-weight — annotator for topic classification, dialog acts classification, sentiment, toxicity, emotion, factoid classification.
Let’s create our bot!
Install DeepPavlov Dreamtools:
pip install git+https://github.com/deeppavlov/deeppavlov_dreamtools.git
Clone Dream repository:
git clone https://github.com/deeppavlov/dream.git
Go to cloned repository:
Create your own distribution with above discussed components: dreamtools new dist movies_and_qa \
dreamtools new dist movies_and_qa --display-name "New Movies and QA Assistant" --author email@example.com --description "Distribution for discussing movies and answering questions" --overwrite --annotators components/1Q9QXih1U2zhCpVm9zxdsA.yml --annotators components/dflkgjdlkh342r9ndf.yml --annotators components/tgzaSQggV7wgMprOmF1Ww.yml --annotators components/M1sE6hOm20EGBWBdr0vIOw.yml --annotators components/O4FVnkAwjay1mL1FbuRGWw.yml --skills components/4yA8wZWOEnafRfz6Po9nvA.yml --skills components/qx0j5QHAzog0b39nRnuA.yml --skills components/ckUclxqUplyzwmnYyixEw.yml --skills components/uYkoK0vRp4bbIg9akI1yw.yml --response-annotators components/dflkgjdlkh342r9ndf.yml --response-annotators components/05PqJXVd7gV7DqslN5z3A.yml --last-chance-service components/70NLr5qqOow5.yml --timeout-service components/x8rLTpIWct4P.yml --response-annotator-selectors components/LXrJDIf43gwNmPMNXG5Eg.yml --skill-selectors components/xSwFvtAUdvtQosvzpb7oMg.yml --response-selectors components/KX4drAocVa5APcivWHeBNQ.yml
Go to asisstant_dists/movies_and_qa and create proxy.yml, and paste the following code:
services: sentseg: command: ["nginx", "-g", "daemon off;"] build: context: dp/proxy/ dockerfile: Dockerfile environment: - PROXY_PASS=proxy.deeppavlov.ai:8011 - PORT=8011 entity-detection: command: [ "nginx", "-g", "daemon off;" ] build: context: dp/proxy/ dockerfile: Dockerfile environment: - PROXY_PASS=proxy.deeppavlov.ai:8103 - PORT=8103 entity-linking: command: [ "nginx", "-g", "daemon off;" ] build: context: dp/proxy/ dockerfile: Dockerfile environment: - PROXY_PASS=proxy.deeppavlov.ai:8075 - PORT=8075 wiki-parser: command: [ "nginx", "-g", "daemon off;" ] build: context: dp/proxy/ dockerfile: Dockerfile environment: - PROXY_PASS=proxy.deeppavlov.ai:8077 - PORT=8077 dff-movie-skill: command: ["nginx", "-g", "daemon off;"] build: context: dp/proxy/ dockerfile: Dockerfile environment: - PROXY_PASS=proxy.deeppavlov.ai:8023 - PORT=8023 factoid-qa: command: ["nginx", "-g", "daemon off;"] build: context: dp/proxy/ dockerfile: Dockerfile environment: - PROXY_PASS=proxy.deeppavlov.ai:8071 - PORT=8071 dff-program-y-skill: command: ["nginx", "-g", "daemon off;"] build: context: dp/proxy/ dockerfile: Dockerfile environment: - PROXY_PASS=proxy.deeppavlov.ai:8008 - PORT=8008 version: '3.7'
Build your distribution:
docker-compose -f docker-compose.yml -f assistant_dists/movies_and_qa/docker-compose.override.yml -f assistant_dists/movies_and_qa/proxy.yml up --build
*Please note that in the command, we also utilize the assistant_dists/movies_and_qa/proxy.yml configuration. This configuration enables you to conserve your local resources by employing proxied copies of certain services hosted by DeepPavlov.
In a separate terminal tab, run:
docker-compose exec agent python -m deeppavlov_agent.run agent.debug=false agent.channel=cmd agent.pipeline_config=assistant_dists/movies_and_qa/pipeline_conf.json
Enter your username and have a chat with Dream!
- Dream Website
- Dream on GitHub
- DeepPavlov Discord
- Dream FAQ
- Creating Assistants with DeepPavlov Dream. Part 2: Customizing Prompt-based Distribution
- Creating Assistants with DeepPavlov Dream. Part 3. Task-Performing Distribution with Reasoning and API Integration
- Creating Assistants with DeepPavlov Dream. Part 4: Document-based Question Answering with LLMs
- Building Generative AI Assistants with DeepPavlov Dream
- Prompting For Response Generation: Which LLM to Choose to Build Your Own Chatbot