deeppavlov.models.morpho_tagger¶
-
deeppavlov.models.morpho_tagger.common.predict_with_model(config_path: [<class 'pathlib.Path'>, <class 'str'>], infile: Optional[Union[pathlib.Path, str]] = None, input_format: str = 'ud', batch_size: [<class 'int'>] = 16, output_format: str = 'basic') → List[Optional[List[str]]][source]¶ Returns predictions of morphotagging model given in config :config_path:.
- Parameters
config_path – a path to config
- Returns
a list of morphological analyses for each sentence. Each analysis is either a list of tags or a list of full CONLL-U descriptions.
-
class
deeppavlov.models.morpho_tagger.lemmatizer.UDPymorphyLemmatizer(save_path: Optional[str] = None, load_path: Optional[str] = None, rare_grammeme_penalty: float = 1.0, long_lemma_penalty: float = 1.0, **kwargs)[source]¶ A class that returns a normal form of a Russian word given its morphological tag in UD format. Lemma is selected from one of PyMorphy parses, the parse whose tag resembles the most a known UD tag is chosen.
-
class
deeppavlov.models.morpho_tagger.common.TagOutputPrettifier(format_mode: str = 'basic', return_string: bool = True, begin: str = '', end: str = '', sep: str = '\n', **kwargs)[source]¶ Class which prettifies morphological tagger output to 4-column or 10-column (Universal Dependencies) format.
- Parameters
format_mode – output format, in basic mode output data contains 4 columns (id, word, pos, features), in conllu or ud mode it contains 10 columns: id, word, lemma, pos, xpos, feats, head, deprel, deps, misc (see http://universaldependencies.org/format.html for details) Only id, word, tag and pos values are present in current version, other columns are filled by _ value.
return_string – whether to return a list of strings or a single string
begin – a string to append in the beginning
end – a string to append in the end
sep – separator between word analyses
-
__call__(X: List[List[str]], Y: List[List[str]]) → List[Union[List[str], str]][source]¶ Calls the
prettify()function for each input sentence.- Parameters
X – a list of input sentences
Y – a list of list of tags for sentence words
- Returns
a list of prettified morphological analyses
-
prettify(tokens: List[str], tags: List[str]) → Union[List[str], str][source]¶ Prettifies output of morphological tagger.
- Parameters
tokens – tokenized source sentence
tags – list of tags, the output of a tagger
- Returns
the prettified output of the tagger.
Examples
>>> sent = "John really likes pizza .".split() >>> tags = ["PROPN,Number=Sing", "ADV", >>> "VERB,Mood=Ind|Number=Sing|Person=3|Tense=Pres|VerbForm=Fin", >>> "NOUN,Number=Sing", "PUNCT"] >>> prettifier = TagOutputPrettifier(mode='basic') >>> self.prettify(sent, tags) 1 John PROPN Number=Sing 2 really ADV _ 3 likes VERB Mood=Ind|Number=Sing|Person=3|Tense=Pres|VerbForm=Fin 4 pizza NOUN Number=Sing 5 . PUNCT _ >>> prettifier = TagOutputPrettifier(mode='ud') >>> self.prettify(sent, tags) 1 John _ PROPN _ Number=Sing _ _ _ _ 2 really _ ADV _ _ _ _ _ _ 3 likes _ VERB _ Mood=Ind|Number=Sing|Person=3|Tense=Pres|VerbForm=Fin _ _ _ _ 4 pizza _ NOUN _ Number=Sing _ _ _ _ 5 . _ PUNCT _ _ _ _ _ _
-
set_format_mode(format_mode: str = 'basic') → None[source]¶ A function that sets format for output and recalculates self.format_string.
- Parameters
format_mode – output format, in basic mode output data contains 4 columns (id, word, pos, features), in conllu or ud mode it contains 10 columns: id, word, lemma, pos, xpos, feats, head, deprel, deps, misc (see http://universaldependencies.org/format.html for details) Only id, word, tag and pos values are present in current version, other columns are filled by _ value.
Returns:
-
class
deeppavlov.models.morpho_tagger.common.LemmatizedOutputPrettifier(return_string: bool = True, begin: str = '', end: str = '', sep: str = '\n', **kwargs)[source]¶ Class which prettifies morphological tagger output to 4-column or 10-column (Universal Dependencies) format.
- Parameters
format_mode – output format, in basic mode output data contains 4 columns (id, word, pos, features), in conllu or ud mode it contains 10 columns: id, word, lemma, pos, xpos, feats, head, deprel, deps, misc (see http://universaldependencies.org/format.html for details) Only id, word, lemma, tag and pos columns are predicted in current version, other columns are filled by _ value.
return_string – whether to return a list of strings or a single string
begin – a string to append in the beginning
end – a string to append in the end
sep – separator between word analyses
-
__call__(X: List[List[str]], Y: List[List[str]], Z: List[List[str]]) → List[Union[List[str], str]][source]¶ Calls the
prettify()function for each input sentence.- Parameters
X – a list of input sentences
Y – a list of list of tags for sentence words
Z – a list of lemmatized sentences
- Returns
a list of prettified morphological analyses
-
prettify(tokens: List[str], tags: List[str], lemmas: List[str]) → Union[List[str], str][source]¶ Prettifies output of morphological tagger.
- Parameters
tokens – tokenized source sentence
tags – list of tags, the output of a tagger
lemmas – list of lemmas, the output of a lemmatizer
- Returns
the prettified output of the tagger.
Examples
>>> sent = "John really likes pizza .".split() >>> tags = ["PROPN,Number=Sing", "ADV", >>> "VERB,Mood=Ind|Number=Sing|Person=3|Tense=Pres|VerbForm=Fin", >>> "NOUN,Number=Sing", "PUNCT"] >>> lemmas = "John really like pizza .".split() >>> prettifier = LemmatizedOutputPrettifier() >>> self.prettify(sent, tags, lemmas) 1 John John PROPN _ Number=Sing _ _ _ _ 2 really really ADV _ _ _ _ _ _ 3 likes like VERB _ Mood=Ind|Number=Sing|Person=3|Tense=Pres|VerbForm=Fin _ _ _ _ 4 pizza pizza NOUN _ Number=Sing _ _ _ _ 5 . . PUNCT _ _ _ _ _ _