fst.parsex

Extended AST parse. Allows parsing bits and pieces of valid python AST trees which are not normally parsable individually with ast.parse().

The parse functions in this module are oriented towards parsing all valid elements which they name which may include parsing things which are not those elements, common sense is applied liberally though. And just because an element is parsed successfully doesn't mean it may not need parentheses if used in the way intended. Or that it needs a trailing newline due to comment ending without an explicit trailing newline.

Mode = typing.Union[typing.Literal['all', 'strict', 'exec', 'eval', 'single', 'stmts', 'stmt', 'ExceptHandler', '_ExceptHandlers', 'match_case', '_match_cases', 'expr', 'expr_all', 'expr_arglike', 'expr_slice', 'Tuple_elt', 'Tuple', '_Assign_targets', '_decorator_list', 'boolop', 'operator', 'unaryop', 'cmpop', 'comprehension', '_comprehensions', '_comprehension_ifs', 'arguments', 'arguments_lambda', 'arg', 'keyword', 'alias', '_aliases', 'Import_name', '_Import_names', 'ImportFrom_name', '_ImportFrom_names', 'withitem', '_withitems', 'pattern', 'type_param', '_type_params', '_expr_arglikes'], type[ast.AST]]

Extended parse modes:

  • 'all': Check all possible parse modes (from most likely to least). There is syntax overlap so certain types will never be returned, for example TypeVar is always shadowed by AnnAssign. Since this attempts many parses before failing it is slower to do so than other modes, though the most likely success is just as fast. Will never return an Expression or Interactive.
  • 'strict': Attempt parse minumum valid parsable code. If only one statement then return the statement itself instead of the Module. If that statement is an Expr then return the expression instead of the statement. If nothing present then return empty Module. Doesn't attempt any of the other parse modes which would not normally be parsable by python, just anything that can be parsed natively by ast.parse().
  • 'exec': Parse to a Module. Same as passing Module type or 'stmts'. Same as ast.parse() mode 'exec'.
  • 'eval': Parse to an Expression. Same as passing Expression type. Same as ast.parse() mode 'eval'.
  • 'single': Parse to an Interactive. Same as passing Interactive type. Same as ast.parse() mode 'single'.
  • 'stmts': Parse zero or more stmts returned in a Module. Same as passing 'exec' or Module.
  • 'stmt': Parse a single stmt returned as itself. Same as passing stmt type.
  • 'ExceptHandler': Parse as a single ExceptHandler returned as itself. Same as passing ExceptHandler type.
  • '_ExceptHandlers': Parse zero or more ExceptHandlers returned in an _ExceptHandlers SPECIAL SLICE.
  • 'match_case': Parse a single match_case returned as itself. Same as passing match_case type.
  • '_match_cases': Parse zero or more match_cases returned in a _match_cases SPECIAL SLICE.
  • 'expr': "expression", parse a single expr returned as itself. This is differentiated from the following modes by the handling of slices and starred expressions. In this mode a:b and *not v are syntax errors. Same as passing expr type.
  • 'expr_all': Parse to any kind of expression including Slice, *not a or Tuple of any of those combined (but unparenthesized in that case).
  • 'expr_arglike': Accept special syntax for Starred (call argument, class base definition, slice implicit tuple), same as 'expr' except that in this mode a:b is a syntax error and *not v parses to a starred expression *(not v).
  • 'expr_slice': "slice expression", same as 'expr' except that in this mode a:b parses to a Slice and *not v parses to a single element tuple containing a starred expression (*(not v),).
  • 'Tuple_elt': "tuple element expression" as in a Tuple that can be anywhere including a Subscript.slice. Same as 'expr' except that in this mode a:b parses to a Slice and *not v parses to a starred expression *(not v). Tuples are parsed but cannot contain Slices or arglike expressions.
  • 'Tuple': Parse to a Tuple which may contain anything that a tuple can contain like multiple Slices and arglike expressions. If it contains Slices or arglikes then it must not be parenthesized (as per python syntax).
  • '_Assign_targets': Parse zero or more Assign targets returned in a _Assign_targets SPECIAL SLICE, with = as separators and an optional trailing =.
  • '_decorator_list': Parse zero or more decorators returned in a _decorator_list SPECIAL SLICE. Each decorator must have a leading @.
  • 'boolop': Parse to a boolop operator.
  • 'operator': Parse to an operator operator.
  • 'unaryop': Parse to a unaryop operator.
  • 'cmpop': Parse to a cmpop compare operator.
  • 'comprehension': Parse a single comprehension returned as itself. Same as passing comprehension type.
  • '_comprehensions': Parse zero or more comprehensions returned in a _comprehensions SPECIAL SLICE.
  • '_comprehension_ifs': Parse zero or more comprehension ifs returned in a _comprehension_ifs SPECIAL SLICE.
  • 'arguments': Parse as arguments for a FunctionDef or AsyncFunctionDef returned as itself. In this mode type annotations are allowed for the arguments. Same as passing arguments type.
  • 'arguments_lambda': Parse as arguments for a Lambda returned as itself. In this mode type annotations are not allowed for the arguments.
  • 'arg': Parse as a single arg returned as itself. Same as passing arg type.
  • 'keyword': Parse as a single keyword returned as itself. Same as passing keyword type.
  • 'alias': Parse as a single alias returned as itself. Either starred or dotted versions are accepted. Same as passing alias type.
  • '_aliases': Parse zero or more aliases returned in a _aliases SPECIAL SLICE. Either starred or dotted versions are accepted. Does not need trailing comma for a single element.
  • 'Import_name': Parse as a single alias returned as itself, with starred version being a syntax error. This is the alias used in Import.names.
  • '_Import_names': Parse zero or more alias returned in a _aliases SPECIAL SLICE, with starred version being a syntax error. Does not need trailing comma for a single element. This is the alias used in Import.names.
  • 'ImportFrom_name': Parse as a single alias returned as itself, with dotted version being a syntax error. This is the alias used in ImportFrom.names.
  • '_ImportFrom_names': Parse zero or more alias returned in a _aliases SPECIAL SLICE, with dotted version being a syntax error. Does not need trailing comma for a single element. This is the alias used in ImportFrom.names.
  • 'withitem': Parse as a single withitem returned as itself. Same as passing withitem type.
  • '_withitems': Parse zero or more withitems returned in a _withitems SPECIAL SLICE.
  • 'pattern': Parse as a a single pattern returned as itself. Same as passing pattern type.
  • 'type_param': Parse as a single type_param returned as itself, either TypeVar, ParamSpec or TypeVarTuple. Same as passing type_param type.
  • '_type_params': Parse zero or more type_params returned in a _type_params SPECIAL SLICE. Does not need trailing comma for a single element.
  • type[AST]: If an AST type is passed then will attempt to parse to this type. This can be used to narrow the scope of desired return, for example Constant will parse as an expression but fail if the expression is not a Constant. These overlap with the string specifiers to an extent but not all of them. For example AST type expr is the same as passing 'expr'. Not all string specified modes are can be matched, for example 'arguments_lambda'. Likewise 'exec' and 'stmts' specify the same parse mode. Tuple parse also allows parsing Slices in the Tuple as well as otherwise invalid star notation *not a.
  • str(type[AST]): Same as type[AST].
class ParseError(builtins.SyntaxError):

Not technically a syntax error but mostly not the code we were expecting.