DeepPavlov

An open source library for deep learning end-to-end dialog systems and chatbots.

License Apache 2.0 Python 3.6, 3.7 Downloads

DeepPavlov is an open-source conversational AI library built on TensorFlow and Keras.

DeepPavlov is designed for

Models

Named Entity Recognition Slot filling
Intent/Sentence Classification Question Answering over Text (SQuAD)
Sentence Similarity/Ranking TF-IDF Ranking
Morphological tagging Automatic Spelling Correction

ELMo training and fine-tuning

Skills

Goal(Task)-oriented Bot Seq2seq Goal-Oriented bot
Open Domain Questions Answering eCommerce Bot
Frequently Asked Questions Answering Pattern Matching

Embeddings

BERT embeddings for the Russian, Polish, Bulgarian, Czech, and informal English

ELMo embeddings for the Russian language

FastText embeddings for the Russian language

Auto ML

Tuning Models with Evolutionary Algorithm

Installation

  1. We support Linux and Windows platforms, Python 3.6 and Python 3.7
    • Python 3.5 is not supported!
    • installation for Windows requires Git(for example, git) and Visual Studio 2015/2017 with C++ build tools installed!
  2. Create and activate a virtual environment:
    • Linux
       python -m venv env
       source ./env/bin/activate
      
    • Windows
       python -m venv env
       .\env\Scripts\activate.bat
      
  3. Install the package inside the environment:
     pip install deeppavlov
    

QuickStart

There is a bunch of great pre-trained NLP models in DeepPavlov. Each model is determined by its config file.

List of models is available on the doc page in the deeppavlov.configs (Python):

    from deeppavlov import configs

When you’re decided on the model (+ config file), there are two ways to train, evaluate and infer it:

Before making choice of an interface, install model’s package requirements (CLI):

    python -m deeppavlov install <config_path>

Command line interface (CLI)

To get predictions from a model interactively through CLI, run

    python -m deeppavlov interact <config_path> [-d]

You can train it in the same simple way:

    python -m deeppavlov train <config_path> [-d]

Dataset will be downloaded regardless of whether there was -d flag or not.

To train on your own data you need to modify dataset reader path in the train config doc. The data format is specified in the corresponding model doc page.

There are even more actions you can perform with configs:

    python -m deeppavlov <action> <config_path> [-d]

Python

To get predictions from a model interactively through Python, run

    from deeppavlov import build_model

    model = build_model(<config_path>, download=True)

    # get predictions for 'input_text1', 'input_text2'
    model(['input_text1', 'input_text2'])

You can train it in the same simple way:

    from deeppavlov import train_model 

    model = train_model(<config_path>, download=True)

Dataset will be downloaded regardless of whether there was -d flag or not.

To train on your own data you need to modify dataset reader path in the train config doc. The data format is specified in the corresponding model doc page.

You can also calculate metrics on the dataset specified in your config file:

    from deeppavlov import evaluate_model 

    model = evaluate_model(<config_path>, download=True)

There are also available integrations with various messengers, see Telegram Bot doc page and others in the Integrations section for more info.

Breaking Changes

Breaking changes in version 0.5.0

Breaking changes in version 0.4.0!

Breaking changes in version 0.3.0!

Breaking changes in version 0.2.0!

Breaking changes in version 0.1.0!

License

DeepPavlov is Apache 2.0 - licensed.

The Team

DeepPavlov is built and maintained by Neural Networks and Deep Learning Lab at MIPT within iPavlov project (part of National Technology Initiative) and in partnership with Sberbank.