Features

Components

Based on neural Named Entity Recognition network. The NER component reproduces architecture from the paper Application of a Hybrid Bi-LSTM-CRF model to the task of Russian Named Entity Recognition which is inspired by Bi-LSTM+CRF architecture from https://arxiv.org/pdf/1603.01360.pdf.

Based on fuzzy Levenshtein search to extract normalized slot values from text. The components either rely on NER results or perform needle in haystack search.

Component for classification tasks (intents, sentiment, etc) on word-level. Shallow-and-wide CNN, Deep CNN, BiLSTM, BiLSTM with self-attention and other models are presented. The model also allows multilabel classification of texts. Several pre-trained models are available and presented in Table below.

Dataset Valid accuracy Test accuracy
DSTC 2 0.8744 0.8801
DSTC 2 0.9682 0.9684
SNIPS 0.8829 –
InsultsKaggle 0.8757 0.7503
AG News 0.8735 0.8859
Twitter.mokoron 0.8021 (with smiles), 0.8008 (no_smiles) 0.7949 (with smiles), 0.7943 (no_smiles)

Based on Hybrid Code Networks (HCNs) architecture from Jason D. Williams, Kavosh Asadi, Geoffrey Zweig, Hybrid Code Networks: practical and efficient end-to-end dialog control with supervised and reinforcement learning – 2017. It allows to predict responses in goal-oriented dialog. The model is customizable: embeddings, slot filler and intent classifier can switched on and off on demand.

Dialogue agent predicts responses in a goal-oriented dialog and is able to handle multiple domains (pretrained bot allows calendar scheduling, weather information retrieval, and point-of-interest navigation). The model is end-to-end differentiable and does not need to explicitly model dialogue state or belief trackers.

Pipelines that use candidates search in a static dictionary and an ARPA language model to correct spelling errors.

Based on LSTM-based deep learning models for non-factoid answer selection. The model performs ranking of responses or contexts from some database by their relevance for the given context.

Based on R-NET: Machine Reading Comprehension with Self-matching Networks. The model solves the task of looking for an answer on a question in a given context (SQuAD task format).

Based on character-based approach to morphological tagging Heigold et al., 2017. An extensive empirical evaluation of character-based morphological tagging for 14 languages. A state-of-the-art model for Russian and several other languages. Model assigns morphological tags in UD format to sequences of words.

Skills

An open domain question answering skill. The skill accepts free-form questions about the world and outputs an answer based on its Wikipedia knowledge.

Parameters evolution

Implementation of parameters evolution for DeepPavlov models that requires only some small changes in a config file.

Embeddings

Word vectors for the Russian language trained on joint Russian Wikipedia and Lenta.ru corpora.

Examples of some components

  • Run goal-oriented bot with Telegram interface:

    python -m deeppavlov interactbot deeppavlov/configs/go_bot/gobot_dstc2.json -d -t <TELEGRAM_TOKEN>

  • Run goal-oriented bot with console interface:

    python -m deeppavlov interact deeppavlov/configs/go_bot/gobot_dstc2.json -d

  • Run goal-oriented bot with REST API:

    python -m deeppavlov riseapi deeppavlov/configs/go_bot/gobot_dstc2.json -d

  • Run slot-filling model with Telegram interface:

    python -m deeppavlov interactbot deeppavlov/configs/ner/slotfill_dstc2.json -d -t <TELEGRAM_TOKEN>

  • Run slot-filling model with console interface:

    python -m deeppavlov interact deeppavlov/configs/ner/slotfill_dstc2.json -d

  • Run slot-filling model with REST API:

    python -m deeppavlov riseapi deeppavlov/configs/ner/slotfill_dstc2.json -d

  • Predict intents on every line in a file:

    python -m deeppavlov predict deeppavlov/configs/intents/intents_snips.json -d --batch-size 15 < /data/in.txt > /data/out.txt

View video demo of deployment of a goal-oriented bot and a slot-filling model with Telegram UI.