Context Question Answering

Open In Colab

Medium

1. Introduction to the task

Context Question Answering is a task of finding a fragment with an answer to a question in a given segment of context.

Context:

In meteorology, precipitation is any product of the condensation
of atmospheric water vapor that falls under gravity. The main forms
of precipitation include drizzle, rain, sleet, snow, graupel and hail…
Precipitation forms as smaller droplets coalesce via collision with
other rain drops or ice crystals within a cloud. Short, intense periods
of rain in scattered locations are called “showers”.

Question:

Where do water droplets collide with ice crystals to form precipitation?

Answer:

within a cloud

Datasets that follow this task format:

2. Get started with the model

First make sure you have the DeepPavlov Library installed. More info about the first installation.

[ ]:
!pip install -q deeppavlov

Then make sure that all the required packages for the model are installed.

[ ]:
!python -m deeppavlov install squad_bert

squad_bert is the name of the model’s config_file. What is a Config File?

Configuration file defines the model and describes its hyperparameters. To use another model, change the name of the config_file here and further. The full list of the models with their config names can be found in the table.

3. Models list

The table presents a list of all of the Context Question Answering models available in DeepPavlov Library.

Config name

Dataset

Language

Model Size

F1 score

EM

squad_bert

SQuAD v1.1

En

1.3 GB

88.86

81.49

qa_squad2_bert

SQuAD v2.0

En

1.3 GB

83.56

75.54

qa_multisberquad_bert

MultiSQuAD

Multi

2 GB

80.76

63.81

squad_ru_bert

SberQuAD

Ru

2.0 GB

84.71

66.21

squad_ru_convers_distilrubert_2L

SberQuAD

Ru

1.2 GB

65.20

44.52

squad_ru_convers_distilrubert_6L

SberQuAD

Ru

1.6 GB

80.57

61.54

4. Use the model for prediction

4.1 Predict using Python

After installing the model, build it from the config and predict.

[ ]:
from deeppavlov import build_model

model = build_model('squad_bert', download=True, install=True)

Input: List[context], List[question]

Output: List[answer, start_character, logit]

[ ]:
model(['DeepPavlov is a library for NLP and dialog systems.'], ['What is DeepPavlov?'])
[['a library for NLP and dialog systems'], [14], [200928.390625]]

4.2 Predict using CLI

You can also get predictions in an interactive mode through CLI (Command Line Interface).

[ ]:
!python -m deeppavlov interact squad_bert -d

-d is an optional download key (alternative to download=True in Python code). The key -d is used to download the pre-trained model along with embeddings and all other files needed to run the model.

Or make predictions for samples from stdin.

[ ]:
!python -m deeppavlov predict squad_bert -f <file-name>

5. Train the model on your data

5.1 Train your model from Python

Provide your data path

To train the model on your data, you need to change the path to the training data in the config_file.

Parse the config_file and change the path to your data from Python.

[ ]:
from deeppavlov import train_model
from deeppavlov.core.commands.utils import parse_config

model_config = parse_config('squad_bert')

#  dataset that the model was trained on
print(model_config['dataset_reader']['data_path'])
~/.deeppavlov/downloads/squad/

Provide a data_path to your own dataset.

[ ]:
# download and unzip a new example dataset
!wget http://files.deeppavlov.ai/datasets/squad-v1.1.tar.gz
!tar -xzvf "squad-v1.1.tar.gz"

Note that if you want to provide your own dataset, it should have the same format as the SQuAD dataset downloaded in this cell.

[ ]:
# provide a path to the train file
model_config['dataset_reader']['data_path'] = '/contents/train-v1.1.json'

SQuAD dataset info

There are two versions of the SQuAD dataset available for training at the moment:

  • SQuAD 1.1 contains 107,785 question-answer pairs on 536 articles. Dataset size: 33.52 MiB.

  • SQuAD 2.0 combines all of the questions from SQuAD 1.1 with over 50,000 un-answerable questions written adversarially by crowdworkers. Dataset size: 44.34 MiB.

Train the model using new config

[ ]:
model = train_model(model_config)

Use your model for prediction.

[ ]:
model(['DeepPavlov is a library for NLP and dialog systems.'], ['What is DeepPavlov?'])
[['a library for NLP and dialog systems'], [14], [200928.390625]]

5.2 Train your model from CLI

[ ]:
!python -m deeppavlov train squad_bert

6. Evaluate

6.1 Evaluate from Python

[ ]:
from deeppavlov import evaluate_model

model = evaluate_model('squad_bert', download=True)

6.2 Evaluate from CLI

[ ]:
! python -m deeppavlov evaluate squad_bert -d