module documentation

Built-in span-level token classes.
Class ​Auto​Link Autolink tokens. ("<http://www.google.com>")
Class ​Core​Tokens Undocumented
Class ​Emphasis Emphasis tokens. ("some text")
Class ​Escape​Sequence Escape sequences. ("*")
Class ​HTMLSpan Span-level HTML tokens.
Class ​Image Image tokens. ("![alt](src "title")")
Class ​Inline​Code Inline code tokens. ("some code")
Class ​Line​Break Hard or soft line breaks.
Class ​Link Link tokens. ("[name](target)")
Class ​Raw​Text Raw text. A leaf node.
Class ​Span​Token Undocumented
Class ​Strikethrough Strikethrough tokens. ("~~some text~~")
Class ​Strong Strong tokens. ("some text")
Class ​XWiki​Block​Macro​End A "block" macro closing tag. ("<onlySpacesAllowed>{{/macroName}}")
Class ​XWiki​Block​Macro​Start A "block" macro opening tag. ("{{macroName<optionalParams>}}<newLine>")
Function add​_token Allows external manipulation of the parsing process. This function is called in BaseRenderer.__enter__.
Function remove​_token Allows external manipulation of the parsing process. This function is called in BaseRenderer.__exit__.
Function reset​_tokens Resets global _token_types to all token classes in __all__.
Function tokenize​_inner A wrapper around span_tokenizer.tokenize. Pass in all span-level token constructors as arguments to span_tokenizer.tokenize.
Variable ​_attrs Undocumented
Variable ​_cdata Undocumented
Variable ​_closing​_tag Undocumented
Variable ​_comment Undocumented
Variable ​_declaration Undocumented
Variable ​_instruction Undocumented
Variable ​_open​_tag Undocumented
Variable ​_root​_node Undocumented
Variable ​_tag Undocumented
Variable ​_tags Undocumented
Variable ​_token​_types Undocumented
def add_token(token_cls, position=1):
Allows external manipulation of the parsing process. This function is called in BaseRenderer.__enter__.
Parameters
token​_cls:SpanTokentoken to be included in the parsing process.
positionUndocumented
def remove_token(token_cls):
Allows external manipulation of the parsing process. This function is called in BaseRenderer.__exit__.
Parameters
token​_cls:SpanTokentoken to be removed from the parsing process.
def reset_tokens():
Resets global _token_types to all token classes in __all__.
def tokenize_inner(content):

A wrapper around span_tokenizer.tokenize. Pass in all span-level token constructors as arguments to span_tokenizer.tokenize.

Doing so (instead of importing span_token module in span_tokenizer) avoids cyclic dependency issues, and allows for future injections of custom token classes.

_token_types variable is at the bottom of this module.

See also: span_tokenizer.tokenize, block_token.tokenize.

_attrs: str =

Undocumented

_cdata: str =

Undocumented

_closing_tag =

Undocumented

_comment: str =

Undocumented

_declaration: str =

Undocumented

_instruction: str =

Undocumented

_open_tag =

Undocumented

_root_node =

Undocumented

_tag: str =

Undocumented

_tags: set[str] =

Undocumented

_token_types: list =

Undocumented