deeppavlov.models.kbqa

class deeppavlov.models.kbqa.type_define.AnswerTypesExtractor(lang: str, types_filename: str, types_sets_filename: str, num_types_to_return: int = 15, **kwargs)[source]

Class which defines answer types for the question

__init__(lang: str, types_filename: str, types_sets_filename: str, num_types_to_return: int = 15, **kwargs)[source]
Parameters
  • lang – Russian or English

  • types_filename – filename with dictionary where keys are type ids and values are type labels

  • types_sets_filename – filename with dictionary where keys are NER tags and values are Wikidata types corresponding to tags

  • num_types_to_return – how many answer types to return for each question

  • **kwargs

__call__(questions_batch: List[str], entity_substr_batch: List[List[str]], tags_batch: List[List[str]], types_substr_batch: Optional[List[List[str]]] = None)[source]

Call self as a function.

class deeppavlov.models.kbqa.query_generator.QueryGenerator(wiki_parser: WikiParser, rel_ranker: RelRankerInfer, entities_to_leave: int = 5, types_to_leave: int = 2, rels_to_leave: int = 7, max_comb_num: int = 10000, gold_query_info: Optional[Dict[str, str]] = None, map_query_str_to_kb: Optional[List[Tuple[str, str]]] = None, return_answers: bool = True, *args, **kwargs)[source]

Class for query generation using Wikidata hdt file

__init__(wiki_parser: WikiParser, rel_ranker: RelRankerInfer, entities_to_leave: int = 5, types_to_leave: int = 2, rels_to_leave: int = 7, max_comb_num: int = 10000, gold_query_info: Optional[Dict[str, str]] = None, map_query_str_to_kb: Optional[List[Tuple[str, str]]] = None, return_answers: bool = True, *args, **kwargs) None[source]
Parameters
  • wiki_parser – component deeppavlov.models.kbqa.wiki_parser

  • rel_ranker – component deeppavlov.models.kbqa.rel_ranking_infer

  • entities_to_leave – how many entities to leave after entity linking

  • types_to_leave – how many types to leave after entity linking

  • rels_to_leave – how many relations to leave after relation ranking

  • max_comb_num – the maximum number of combinations of candidate entities and relations

  • gold_query_info – dict of variable names used for formatting output sparql queries

  • map_query_str_to_kb – mapping of knowledge base prefixes to full https

  • return_answers – whether to return answers or candidate relations and answers for further ranking

  • **kwargs

__call__(question_batch: List[str], question_san_batch: List[str], template_type_batch: Union[List[List[str]], List[str]], entities_from_ner_batch: List[List[str]], types_from_ner_batch: List[List[str]], entity_tags_batch: List[List[str]], probas_batch: List[List[float]], answer_types_batch: Optional[List[Set[str]]] = None, entities_to_link_batch: Optional[List[List[int]]] = None) Tuple[List[Any], List[Any]][source]

Call self as a function.

class deeppavlov.models.kbqa.query_generator_base.QueryGeneratorBase(template_matcher: TemplateMatcher, rel_ranker: RelRankerInfer, load_path: str, sparql_queries_filename: str, entity_linker: EntityLinker, rels_in_ranking_queries_fname: Optional[str] = None, wiki_parser=None, entities_to_leave: int = 5, rels_to_leave: int = 7, syntax_structure_known: bool = False, use_wp_api_requester: bool = False, use_el_api_requester: bool = False, use_alt_templates: bool = True, delete_rel_prefix: bool = True, kb_prefixes: Optional[Dict[str, str]] = None, *args, **kwargs)[source]

This class takes as input entity substrings, defines the template of the query and fills the slots of the template with candidate entities and relations.

__init__(template_matcher: TemplateMatcher, rel_ranker: RelRankerInfer, load_path: str, sparql_queries_filename: str, entity_linker: EntityLinker, rels_in_ranking_queries_fname: Optional[str] = None, wiki_parser=None, entities_to_leave: int = 5, rels_to_leave: int = 7, syntax_structure_known: bool = False, use_wp_api_requester: bool = False, use_el_api_requester: bool = False, use_alt_templates: bool = True, delete_rel_prefix: bool = True, kb_prefixes: Optional[Dict[str, str]] = None, *args, **kwargs) None[source]
Parameters
  • template_matcher – component deeppavlov.models.kbqa.template_matcher

  • rel_ranker – component deeppavlov.models.kbqa.rel_ranking_infer

  • load_path – path to folder with wikidata files

  • sparql_queries_filename – file with sparql query templates

  • entity_linker – component deeppavlov.models.entity_extraction.entity_linking for linking of entities

  • rels_in_ranking_queries_fname – file with list of rels in queries for questions with ranking

  • wiki_parser – component deeppavlov.models.kbqa.wiki_parser

  • entities_to_leave – how many entities to leave after entity linking

  • rels_to_leave – how many relations to leave after relation ranking

  • syntax_structure_known – if syntax tree parser was used to define query template type

  • use_wp_api_requester – whether deeppavlov.models.api_requester.api_requester component will be used for Wiki Parser

  • use_el_api_requester – whether deeppavlov.models.api_requester.api_requester component will be used for Entity Linking

  • use_alt_templates – whether to use alternative templates if no answer was found for default query template

  • delete_rel_prefix – whether to delete prefix in relations

  • kb_prefixes – prefixes for entities, relations and types in the knowledge base

abstract __call__(*args, **kwargs)

Call self as a function.

class deeppavlov.models.kbqa.rel_ranking_infer.RelRankerInfer(load_path: str, rel_q2name_filename: str, return_elements: Optional[List[str]] = None, ranker: Optional[Chainer] = None, wiki_parser: Optional[WikiParser] = None, batch_size: int = 32, softmax: bool = False, use_api_requester: bool = False, rank: bool = True, nll_rel_ranking: bool = False, nll_path_ranking: bool = False, top_possible_answers: int = - 1, top_n: int = 1, pos_class_num: int = 1, rel_thres: float = 0.0, type_rels: Optional[List[str]] = None, **kwargs)[source]

Class for ranking of paths in subgraph

__init__(load_path: str, rel_q2name_filename: str, return_elements: Optional[List[str]] = None, ranker: Optional[Chainer] = None, wiki_parser: Optional[WikiParser] = None, batch_size: int = 32, softmax: bool = False, use_api_requester: bool = False, rank: bool = True, nll_rel_ranking: bool = False, nll_path_ranking: bool = False, top_possible_answers: int = - 1, top_n: int = 1, pos_class_num: int = 1, rel_thres: float = 0.0, type_rels: Optional[List[str]] = None, **kwargs)[source]
Parameters
  • load_path – path to folder with wikidata files

  • rel_q2name_filename – name of file which maps relation id to name

  • return_elements – what elements return in output

  • ranker – component deeppavlov.models.ranking.rel_ranker

  • wiki_parser – component deeppavlov.models.wiki_parser

  • batch_size – infering batch size

  • softmax – whether to process relation scores with softmax function

  • use_api_requester – whether wiki parser will be used as external api

  • rank – whether to rank relations or simple copy input

  • nll_rel_ranking – whether use components trained with nll loss for relation ranking

  • nll_path_ranking – whether use components trained with nll loss for relation path ranking

  • top_possible_answers – number of answers returned for a question in each list of candidate answers

  • top_n – number of lists of candidate answers returned for a question

  • pos_class_num – index of positive class in the output of relation ranking model

  • rel_thres – threshold of relation confidence

  • type_rels – list of relations in the knowledge base which connect an entity and its type

  • **kwargs

__call__(questions_batch: List[str], template_type_batch: List[str], raw_answers_batch: List[List[Tuple[str]]], entity_substr_batch: List[List[str]], template_answers_batch: List[str]) List[str][source]

Call self as a function.

class deeppavlov.models.kbqa.template_matcher.TemplateMatcher(load_path: str, templates_filename: str, num_processors: Optional[int] = None, **kwargs)[source]

This class matches the question with one of the templates to extract entity substrings and define which relations corresponds to the question

__init__(load_path: str, templates_filename: str, num_processors: Optional[int] = None, **kwargs) None[source]
Parameters
  • load_path – path to folder with file with templates

  • templates_filename – file with templates

  • **kwargs

__call__(question: str, entities_from_ner: List[str]) Tuple[Union[List[str], list], list, Union[list, Any], Union[list, Any], Union[str, Any], Union[list, Any], Union[str, Any], Union[list, Any], Union[str, Any]][source]

Call self as a function.

class deeppavlov.models.kbqa.ru_adj_to_noun.RuAdjToNoun(freq_dict_filename: str, candidate_nouns: int = 10, freq_thres: float = 4.5, score_thres: float = 2.8, **kwargs)[source]

Class for converting an adjective in Russian to the corresponding noun, for example: “московский” -> “Москва”, “африканский” -> “Африка”

__init__(freq_dict_filename: str, candidate_nouns: int = 10, freq_thres: float = 4.5, score_thres: float = 2.8, **kwargs)[source]
Parameters
  • freq_dict_filename – file with the dictionary of Russian words with the corresponding frequencies

  • candidate_nouns – how many candidate nouns to leave after search

  • **kwargs

__call__(**kwargs)

Call self as a function.

class deeppavlov.models.kbqa.tree_to_sparql.TreeToSparql(sparql_queries_filename: str, syntax_parser: Component, kb_prefixes: Dict[str, str], adj_to_noun: Optional[RuAdjToNoun] = None, **kwargs)[source]

Class for building of sparql query template using syntax parser

__init__(sparql_queries_filename: str, syntax_parser: Component, kb_prefixes: Dict[str, str], adj_to_noun: Optional[RuAdjToNoun] = None, **kwargs)[source]
Parameters
  • sparql_queries_filename – file with sparql query templates

  • syntax_parser – component for syntactic parsing of the input question

  • kb_prefixes – prefixes for entities, relations and types in the knowledge base

  • adj_to_noun – component deeppavlov.models.kbqa.tree_to_sparql:RuAdjToNoun

  • **kwargs

__call__(questions_batch: List[str], substr_batch: List[List[str]], tags_batch: List[List[str]], offsets_batch: List[List[List[int]]], positions_batch: List[List[List[int]]], probas_batch: List[List[float]]) Tuple[List[Union[str, Any]], List[Union[List[str], List[Union[str, Any]]]], List[Union[List[str], Any]], List[Union[List[Union[str, Any]], Any]], List[Union[List[Union[float, Any]], Any]], List[List[int]], List[Union[List[Any], List[str]]]][source]

Call self as a function.

class deeppavlov.models.kbqa.wiki_parser.WikiParser(wiki_filename: str, file_format: str = 'hdt', prefixes: Optional[Dict[str, Union[str, Dict[str, str]]]] = None, rel_q2name_filename: Optional[str] = None, max_comb_num: int = 1000000.0, lang: str = '@en', **kwargs)[source]

This class extract relations, objects or triplets from Wikidata HDT file.

__init__(wiki_filename: str, file_format: str = 'hdt', prefixes: Optional[Dict[str, Union[str, Dict[str, str]]]] = None, rel_q2name_filename: Optional[str] = None, max_comb_num: int = 1000000.0, lang: str = '@en', **kwargs) None[source]
Parameters
  • wiki_filename – file with Wikidata

  • file_format – format of Wikidata file

  • lang – Russian or English language

  • **kwargs

__call__(parser_info_list: List[str], queries_list: List[Any]) List[Any][source]

Call self as a function.