pygments.lexer.RegexLexer

class documentation

class RegexLexer(Lexer):

View In Hierarchy

Base for simple stateful regular expression-based lexers. Simplifies the lexing process so that you need only provide a list of states and regular expressions.

Method	`get_tokens_unprocessed`	Split `text` into (tokentype, text) pairs.
Class Variable	`tokens`	Undocumented

Inherited from Lexer:

Method	`analyse_text`	No summary
Method	`get_tokens`	Return an iterable of (tokentype, value) pairs generated from `text`. If `unfiltered` is set to `True`, the filtering mechanism is bypassed even if filters are defined.
Class Variable	`alias_filenames`	Undocumented
Class Variable	`aliases`	Undocumented
Class Variable	`filenames`	Undocumented
Class Variable	`mimetypes`	Undocumented
Class Variable	`name`	Undocumented
Method	`__init__`	Undocumented
Method	`__repr__`	Undocumented
Method	`add_filter`	Add a new stream filter to this lexer.
Class Variable	`priority`	Undocumented
Instance Variable	`encoding`	Undocumented
Instance Variable	`ensurenl`	Undocumented
Instance Variable	`filters`	Undocumented
Instance Variable	`options`	Undocumented
Instance Variable	`stripall`	Undocumented
Instance Variable	`stripnl`	Undocumented
Instance Variable	`tabsize`	Undocumented

def get_tokens_unprocessed(self, text, stack=('root')):

overrides pygments.lexer.Lexer.get_tokens_unprocessed

overridden in pygments.lexers.dylan.DylanLexer, pygments.lexers.erlang.ElixirLexer, pygments.lexers.graphics.AsymptoteLexer, pygments.lexers.haskell.CryptolLexer, pygments.lexers.html.XsltLexer, pygments.lexers.int_fiction.Inform6Lexer, pygments.lexers.int_fiction.Inform6TemplateLexer, pygments.lexers.int_fiction.Tads3Lexer, pygments.lexers.javascript.LassoLexer, pygments.lexers.jvm.AspectJLexer, pygments.lexers.lilypond.LilyPondLexer, pygments.lexers.lisp.CommonLispLexer, pygments.lexers.lisp.EmacsLispLexer, pygments.lexers.lisp.ShenLexer, pygments.lexers.modula2.Modula2Lexer, pygments.lexers.objective.SwiftLexer, pygments.lexers.pawn.SourcePawnLexer, pygments.lexers.php.PhpLexer, pygments.lexers.python.NumPyLexer, pygments.lexers.scripting.LuaLexer, pygments.lexers.shell.SlurmBashLexer, pygments.lexers.textedit.VimLexer, pygments.lexers.textfmts.HttpLexer, pygments.lexer.ExtendedRegexLexer, pygments.lexer.ProfilingRegexLexer, pygments.lexers.c_cpp.CFamilyLexer

Split text into (tokentype, text) pairs.

stack is the inital stack (default: ['root'])

tokens: dict =

overridden in pygments.lexers.actionscript.ActionScript3Lexer, pygments.lexers.actionscript.ActionScriptLexer, pygments.lexers.actionscript.MxmlLexer, pygments.lexers.algebra.BCLexer, pygments.lexers.algebra.GAPLexer, pygments.lexers.algebra.MathematicaLexer, pygments.lexers.algebra.MuPADLexer, pygments.lexers.ambient.AmbientTalkLexer, pygments.lexers.amdgpu.AMDGPULexer, pygments.lexers.ampl.AmplLexer, pygments.lexers.apdlexer.apdlexer, pygments.lexers.apl.APLLexer, pygments.lexers.arrow.ArrowLexer, pygments.lexers.asc.AscLexer, pygments.lexers.asm.Ca65Lexer, pygments.lexers.asm.Dasm16Lexer, pygments.lexers.asm.GasLexer, pygments.lexers.asm.HsailLexer, pygments.lexers.asm.LlvmLexer, pygments.lexers.asm.LlvmMirBodyLexer, pygments.lexers.asm.LlvmMirLexer, pygments.lexers.asm.NasmLexer, pygments.lexers.asm.ObjdumpLexer, pygments.lexers.asm.TasmLexer, pygments.lexers.automation.AutohotkeyLexer, pygments.lexers.automation.AutoItLexer, pygments.lexers.bare.BareLexer, pygments.lexers.basic.BBCBasicLexer, pygments.lexers.basic.BlitzBasicLexer, pygments.lexers.basic.BlitzMaxLexer, pygments.lexers.basic.CbmBasicV2Lexer, pygments.lexers.basic.MonkeyLexer, pygments.lexers.basic.QBasicLexer, pygments.lexers.basic.VBScriptLexer, pygments.lexers.bdd.BddLexer, pygments.lexers.bibtex.BibTeXLexer, pygments.lexers.bibtex.BSTLexer, pygments.lexers.boa.BoaLexer, pygments.lexers.business.ABAPLexer, pygments.lexers.business.CobolLexer, pygments.lexers.business.GoodDataCLLexer, pygments.lexers.business.MaqlLexer, pygments.lexers.business.OpenEdgeLexer, pygments.lexers.c_like.ClayLexer, pygments.lexers.c_like.ValaLexer, pygments.lexers.capnproto.CapnProtoLexer, pygments.lexers.cddl.CddlLexer, pygments.lexers.chapel.ChapelLexer, pygments.lexers.clean.CleanLexer, pygments.lexers.configs.ApacheConfLexer, pygments.lexers.configs.AugeasLexer, pygments.lexers.configs.Cfengine3Lexer, pygments.lexers.configs.DockerLexer, pygments.lexers.configs.IniLexer, pygments.lexers.configs.KconfigLexer, pygments.lexers.configs.LighttpdConfLexer, pygments.lexers.configs.NestedTextLexer, pygments.lexers.configs.NginxConfLexer, pygments.lexers.configs.PacmanConfLexer, pygments.lexers.configs.PkgConfigLexer, pygments.lexers.configs.PropertiesLexer, pygments.lexers.configs.RegeditLexer, pygments.lexers.configs.SingularityLexer, pygments.lexers.configs.SquidConfLexer, pygments.lexers.configs.TermcapLexer, pygments.lexers.configs.TerminfoLexer, pygments.lexers.configs.TerraformLexer, pygments.lexers.configs.TOMLLexer, pygments.lexers.console.PyPyLogLexer, pygments.lexers.console.VCTreeStatusLexer, pygments.lexers.crystal.CrystalLexer, pygments.lexers.csound.CsoundDocumentLexer, pygments.lexers.css.CssLexer, pygments.lexers.css.SassLexer, pygments.lexers.css.ScssLexer, pygments.lexers.d.CrocLexer, pygments.lexers.d.DLexer, pygments.lexers.dalvik.SmaliLexer, pygments.lexers.data.YamlLexer, pygments.lexers.devicetree.DevicetreeLexer, pygments.lexers.diff.DarcsPatchLexer, pygments.lexers.diff.DiffLexer, pygments.lexers.diff.WDiffLexer, pygments.lexers.dotnet.BooLexer, pygments.lexers.dotnet.CSharpLexer, pygments.lexers.dotnet.FSharpLexer, pygments.lexers.dotnet.NemerleLexer, pygments.lexers.dotnet.VbNetLexer, pygments.lexers.dsls.AlloyLexer, pygments.lexers.dsls.CrmshLexer, pygments.lexers.dsls.FlatlineLexer, pygments.lexers.dsls.MscgenLexer, pygments.lexers.dsls.PanLexer, pygments.lexers.dsls.ProtoBufLexer, pygments.lexers.dsls.PuppetLexer, pygments.lexers.dsls.RslLexer, pygments.lexers.dsls.SnowballLexer, pygments.lexers.dsls.ThriftLexer, pygments.lexers.dsls.VGLLexer, pygments.lexers.dsls.ZeekLexer, pygments.lexers.dylan.DylanLexer, pygments.lexers.dylan.DylanLidLexer, pygments.lexers.ecl.ECLLexer, pygments.lexers.eiffel.EiffelLexer, pygments.lexers.elm.ElmLexer, pygments.lexers.elpi.ElpiLexer, pygments.lexers.erlang.ElixirLexer, pygments.lexers.erlang.ErlangLexer, pygments.lexers.esoteric.AheuiLexer, pygments.lexers.esoteric.BefungeLexer, pygments.lexers.esoteric.BrainfuckLexer, pygments.lexers.esoteric.CAmkESLexer, pygments.lexers.esoteric.CapDLLexer, pygments.lexers.esoteric.RedcodeLexer, pygments.lexers.ezhil.EzhilLexer, pygments.lexers.factor.FactorLexer, pygments.lexers.fantom.FantomLexer, pygments.lexers.felix.FelixLexer, pygments.lexers.floscript.FloScriptLexer, pygments.lexers.forth.ForthLexer, pygments.lexers.fortran.FortranFixedLexer, pygments.lexers.fortran.FortranLexer, pygments.lexers.foxpro.FoxProLexer, pygments.lexers.futhark.FutharkLexer, pygments.lexers.gcodelexer.GcodeLexer, pygments.lexers.gdscript.GDScriptLexer, pygments.lexers.go.GoLexer, pygments.lexers.grammar_notation.AbnfLexer, pygments.lexers.grammar_notation.BnfLexer, pygments.lexers.grammar_notation.JsgfLexer, pygments.lexers.grammar_notation.PegLexer, pygments.lexers.graph.CypherLexer, pygments.lexers.graphics.AsymptoteLexer, pygments.lexers.graphics.GLShaderLexer, pygments.lexers.graphics.GnuplotLexer, pygments.lexers.graphics.HLSLShaderLexer, pygments.lexers.graphics.PostScriptLexer, pygments.lexers.graphics.PovrayLexer, pygments.lexers.graphviz.GraphvizLexer, pygments.lexers.gsql.GSQLLexer, pygments.lexers.haskell.AgdaLexer, pygments.lexers.haskell.CryptolLexer, pygments.lexers.haskell.HaskellLexer, pygments.lexers.haskell.IdrisLexer, pygments.lexers.haskell.KokaLexer, pygments.lexers.haxe.HaxeLexer, pygments.lexers.haxe.HxmlLexer, pygments.lexers.hdl.SystemVerilogLexer, pygments.lexers.hdl.VerilogLexer, pygments.lexers.hdl.VhdlLexer, pygments.lexers.hexdump.HexdumpLexer, pygments.lexers.html.DtdLexer, pygments.lexers.html.HamlLexer, pygments.lexers.html.HtmlLexer, pygments.lexers.html.PugLexer, pygments.lexers.html.ScamlLexer, pygments.lexers.html.XmlLexer, pygments.lexers.idl.IDLLexer, pygments.lexers.igor.IgorLexer, pygments.lexers.inferno.LimboLexer, pygments.lexers.installers.DebianControlLexer, pygments.lexers.installers.NSISLexer, pygments.lexers.installers.RPMSpecLexer, pygments.lexers.installers.SourcesListLexer, pygments.lexers.int_fiction.Inform6Lexer, pygments.lexers.int_fiction.Inform7Lexer, pygments.lexers.int_fiction.Tads3Lexer, pygments.lexers.iolang.IoLexer, pygments.lexers.j.JLexer, pygments.lexers.javascript.CoffeeScriptLexer, pygments.lexers.javascript.DartLexer, pygments.lexers.javascript.EarlGreyLexer, pygments.lexers.javascript.JavascriptLexer, pygments.lexers.javascript.JuttleLexer, pygments.lexers.javascript.KalLexer, pygments.lexers.javascript.LassoLexer, pygments.lexers.javascript.LiveScriptLexer, pygments.lexers.javascript.MaskLexer, pygments.lexers.javascript.ObjectiveJLexer, pygments.lexers.jslt.JSLTLexer, pygments.lexers.julia.JuliaLexer, pygments.lexers.jvm.CeylonLexer, pygments.lexers.jvm.ClojureLexer, pygments.lexers.jvm.GoloLexer, pygments.lexers.jvm.GosuLexer, pygments.lexers.jvm.GroovyLexer, pygments.lexers.jvm.IokeLexer, pygments.lexers.jvm.JasminLexer, pygments.lexers.jvm.JavaLexer, pygments.lexers.jvm.KotlinLexer, pygments.lexers.jvm.PigLexer, pygments.lexers.jvm.SarlLexer, pygments.lexers.jvm.ScalaLexer, pygments.lexers.jvm.XtendLexer, pygments.lexers.kuin.KuinLexer, pygments.lexers.lisp.CommonLispLexer, pygments.lexers.lisp.CPSALexer, pygments.lexers.lisp.EmacsLispLexer, pygments.lexers.lisp.FennelLexer, pygments.lexers.lisp.HyLexer, pygments.lexers.lisp.NewLispLexer, pygments.lexers.lisp.RacketLexer, pygments.lexers.lisp.SchemeLexer, pygments.lexers.lisp.ShenLexer, pygments.lexers.lisp.XtlangLexer, pygments.lexers.make.BaseMakefileLexer, pygments.lexers.make.CMakeLexer, pygments.lexers.markup.BBCodeLexer, pygments.lexers.markup.GroffLexer, pygments.lexers.markup.MarkdownLexer, pygments.lexers.markup.MoinWikiLexer, pygments.lexers.markup.MozPreprocHashLexer, pygments.lexers.markup.RstLexer, pygments.lexers.markup.TexLexer, pygments.lexers.markup.TiddlyWiki5Lexer, pygments.lexers.matlab.MatlabLexer, pygments.lexers.matlab.OctaveLexer, pygments.lexers.matlab.ScilabLexer, pygments.lexers.maxima.MaximaLexer, pygments.lexers.meson.MesonLexer, pygments.lexers.mime.MIMELexer, pygments.lexers.ml.FStarLexer, pygments.lexers.ml.OcamlLexer, pygments.lexers.ml.OpaLexer, pygments.lexers.ml.ReasonLexer, pygments.lexers.ml.SMLLexer, pygments.lexers.modeling.BugsLexer, pygments.lexers.modeling.JagsLexer, pygments.lexers.modeling.ModelicaLexer, pygments.lexers.modeling.StanLexer, pygments.lexers.modula2.Modula2Lexer, pygments.lexers.monte.MonteLexer, pygments.lexers.mosel.MoselLexer, pygments.lexers.ncl.NCLLexer, pygments.lexers.nimrod.NimrodLexer, pygments.lexers.nit.NitLexer, pygments.lexers.nix.NixLexer, pygments.lexers.oberon.ComponentPascalLexer, pygments.lexers.objective.SwiftLexer, pygments.lexers.ooc.OocLexer, pygments.lexers.parasail.ParaSailLexer, pygments.lexers.parsers.AntlrLexer, pygments.lexers.parsers.EbnfLexer, pygments.lexers.parsers.RagelEmbeddedLexer, pygments.lexers.parsers.RagelLexer, pygments.lexers.pascal.AdaLexer, pygments.lexers.pawn.PawnLexer, pygments.lexers.pawn.SourcePawnLexer, pygments.lexers.perl.Perl6Lexer, pygments.lexers.perl.PerlLexer, pygments.lexers.php.PhpLexer, pygments.lexers.php.ZephirLexer, pygments.lexers.pointless.PointlessLexer, pygments.lexers.pony.PonyLexer, pygments.lexers.praat.PraatLexer, pygments.lexers.procfile.ProcfileLexer, pygments.lexers.prolog.LogtalkLexer, pygments.lexers.prolog.PrologLexer, pygments.lexers.promql.PromQLLexer, pygments.lexers.python.CythonLexer, pygments.lexers.python.DgLexer, pygments.lexers.python.Python2Lexer, pygments.lexers.python.Python2TracebackLexer, pygments.lexers.python.PythonLexer, pygments.lexers.python.PythonTracebackLexer, pygments.lexers.qvt.QVToLexer, pygments.lexers.r.RdLexer, pygments.lexers.r.SLexer, pygments.lexers.rdf.ShExCLexer, pygments.lexers.rdf.SparqlLexer, pygments.lexers.rdf.TurtleLexer, pygments.lexers.rebol.RebolLexer, pygments.lexers.rebol.RedLexer, pygments.lexers.resource.ResourceLexer, pygments.lexers.ride.RideLexer, pygments.lexers.rita.RitaLexer, pygments.lexers.rnc.RNCCompactLexer, pygments.lexers.roboconf.RoboconfGraphLexer, pygments.lexers.roboconf.RoboconfInstancesLexer, pygments.lexers.ruby.FancyLexer, pygments.lexers.ruby.RubyLexer, pygments.lexers.rust.RustLexer, pygments.lexers.sas.SASLexer, pygments.lexers.savi.SaviLexer, pygments.lexers.scdoc.ScdocLexer, pygments.lexers.scripting.AppleScriptLexer, pygments.lexers.scripting.ChaiscriptLexer, pygments.lexers.scripting.EasytrieveLexer, pygments.lexers.scripting.HybrisLexer, pygments.lexers.scripting.JclLexer, pygments.lexers.scripting.LSLLexer, pygments.lexers.scripting.LuaLexer, pygments.lexers.scripting.MiniScriptLexer, pygments.lexers.scripting.MOOCodeLexer, pygments.lexers.scripting.RexxLexer, pygments.lexers.sgf.SmartGameFormatLexer, pygments.lexers.shell.BashLexer, pygments.lexers.shell.BatchLexer, pygments.lexers.shell.ExeclineLexer, pygments.lexers.shell.FishShellLexer, pygments.lexers.shell.PowerShellLexer, pygments.lexers.shell.TcshLexer, pygments.lexers.sieve.SieveLexer, pygments.lexers.smalltalk.NewspeakLexer, pygments.lexers.smalltalk.SmalltalkLexer, pygments.lexers.smithy.SmithyLexer, pygments.lexers.smv.NuSMVLexer, pygments.lexers.snobol.SnobolLexer, pygments.lexers.solidity.SolidityLexer, pygments.lexers.sophia.SophiaLexer, pygments.lexers.spice.SpiceLexer, pygments.lexers.sql.MySqlLexer, pygments.lexers.sql.PlPgsqlLexer, pygments.lexers.sql.PostgresLexer, pygments.lexers.sql.RqlLexer, pygments.lexers.sql.SqlLexer, pygments.lexers.sql.TransactSqlLexer, pygments.lexers.srcinfo.SrcinfoLexer, pygments.lexers.stata.StataLexer, pygments.lexers.supercollider.SuperColliderLexer, pygments.lexers.tcl.TclLexer, pygments.lexers.teal.TealLexer, pygments.lexers.templates.Angular2Lexer, pygments.lexers.templates.CheetahLexer, pygments.lexers.templates.ColdfusionLexer, pygments.lexers.templates.DjangoLexer, pygments.lexers.templates.EvoqueLexer, pygments.lexers.templates.GenshiTextLexer, pygments.lexers.templates.HandlebarsLexer, pygments.lexers.templates.LiquidLexer, pygments.lexers.templates.MakoLexer, pygments.lexers.templates.MasonLexer, pygments.lexers.templates.MyghtyLexer, pygments.lexers.templates.SmartyLexer, pygments.lexers.templates.TwigLexer, pygments.lexers.templates.VelocityLexer, pygments.lexers.teraterm.TeraTermLexer, pygments.lexers.testing.GherkinLexer, pygments.lexers.testing.TAPLexer, pygments.lexers.textedit.AwkLexer, pygments.lexers.textedit.SedLexer, pygments.lexers.textedit.VimLexer, pygments.lexers.textfmts.GettextLexer, pygments.lexers.textfmts.HttpLexer, pygments.lexers.textfmts.IrcLogsLexer, pygments.lexers.textfmts.KernelLogLexer, pygments.lexers.textfmts.NotmuchLexer, pygments.lexers.textfmts.TodotxtLexer, pygments.lexers.theorem.CoqLexer, pygments.lexers.theorem.IsabelleLexer, pygments.lexers.theorem.LeanLexer, pygments.lexers.thingsdb.ThingsDBLexer, pygments.lexers.trafficscript.RtsLexer, pygments.lexers.typoscript.TypoScriptCssDataLexer, pygments.lexers.typoscript.TypoScriptHtmlDataLexer, pygments.lexers.typoscript.TypoScriptLexer, pygments.lexers.unicon.IconLexer, pygments.lexers.unicon.UcodeLexer, pygments.lexers.unicon.UniconLexer, pygments.lexers.urbi.UrbiscriptLexer, pygments.lexers.usd.UsdLexer, pygments.lexers.varnish.VCLLexer, pygments.lexers.verification.BoogieLexer, pygments.lexers.verification.SilverLexer, pygments.lexers.webassembly.WatLexer, pygments.lexers.webidl.WebIDLLexer, pygments.lexers.webmisc.CirruLexer, pygments.lexers.webmisc.DuelLexer, pygments.lexers.webmisc.QmlLexer, pygments.lexers.webmisc.SlimLexer, pygments.lexers.webmisc.XQueryLexer, pygments.lexers.whiley.WhileyLexer, pygments.lexers.x10.X10Lexer, pygments.lexers.xorg.XorgLexer, pygments.lexers.yang.YangLexer, pygments.lexers.zig.ZigLexer, pygments.lexers.archetype.AtomsLexer, pygments.lexers.c_cpp.CFamilyLexer, pygments.lexers.csound.CsoundLexer, pygments.lexers.dotnet.GenericAspxLexer, pygments.lexers.email.EmailHeaderLexer, pygments.lexers.jvm.TeaLangLexer, pygments.lexers.parsers.TreetopBaseLexer, pygments.lexers.slash.SlashLanguageLexer, pygments.lexers.sql.PsqlRegexLexer, pygments.lexers.templates.ColdfusionMarkupLexer, pygments.lexers.templates.GenshiMarkupLexer, pygments.lexers.templates.JspRootLexer, pygments.lexers.templates.TeaTemplateRootLexer

Undocumented