_parse module

class pyflyby._parse.AstNodeContext(parent, field, index)
_asdict()

Return a new OrderedDict which maps field names to their values.

_field_defaults = {}
_fields = ('parent', 'field', 'index')
_fields_defaults = {}
classmethod _make(iterable)

Make a new AstNodeContext object from a sequence or iterable

_replace(**kwds)

Return a new AstNodeContext object replacing specified fields with new values

property field

Alias for field number 1

property index

Alias for field number 2

property parent

Alias for field number 0

class pyflyby._parse.IgnoreOptionsDocTestParser
_find_options(source, name, lineno)

Return a dictionary containing option overrides extracted from option directives in the given source string.

name is the string’s name, and lineno is the line number where the example starts; both are used for error messages.

class pyflyby._parse._DummyAst_Node
pyflyby._parse._annotate_ast_context(ast_node)

Recursively annotate context on ast nodes, setting context to a AstNodeContext named tuple with values (parent, field, index). Each ast_node satisfies parent.<field>[<index>] is ast_node.

For non-list fields, the index part is None.

pyflyby._parse._annotate_ast_nodes(ast_node)
Annotate AST with:
  • startpos and endpos

  • [disabled for now: context as AstNodeContext ]

Parameters

ast_node (ast.AST) – AST node returned by _parse_ast_nodes

Returns

None

pyflyby._parse._annotate_ast_startpos(ast_node, parent_ast_node, minpos, text, flags)

Annotate ast_node. Set ast_node.startpos to the starting position of the node within text.

For “typical” nodes, i.e. those other than multiline strings, this is simply FilePos(ast_node.lineno, ast_node.col_offset+1), but taking text.startpos into account.

For multiline string nodes, this function works by trying to parse all possible subranges of lines until finding the range that is syntactically valid and matches value. The candidate range is text[min_start_lineno:lineno+text.startpos.lineno+1].

This function is unfortunately necessary because of a flaw in the output produced by the Python built-in parser. For some crazy reason, the ast_node.lineno attribute represents something different for multiline string literals versus all other statements. For multiline string literal nodes and statements that are just a string expression (or more generally, nodes where the first descendant leaf node is a multiline string literal), the compiler attaches the ending line number as the value of the lineno attribute. For all other than AST nodes, the compiler attaches the starting line number as the value of the lineno attribute. This means e.g. the statement “’’’foonbar’’’” has a lineno value of 2, but the statement “x=’’’foonbar’’’” has a lineno value of 1.

Parameters
  • minpos (FilePos) – Earliest position to check, in the number space of text.

  • text (FileText) – Source text that was used to parse the AST, whose startpos should be used in interpreting ast_node.lineno (which always starts at 1 for the subset that was parsed).

  • flags (CompilerFlags) – Compiler flags to use when re-compiling code.

Returns

True if this node is a multiline string literal or the first child is such a node (recursively); False otherwise.

Raises

ValueError – Could not find the starting line number.

pyflyby._parse._ast_node_is_in_docstring_position(ast_node)

Given a Str AST node, return whether its position within the AST makes it eligible as a docstring.

The main way a Str can be a docstring is if it is a standalone string at the beginning of a Module, FunctionDef, or ClassDef.

We also support variable docstrings per Epydoc:

  • If a variable assignment statement is immediately followed by a bare string literal, then that assignment is treated as a docstring for that variable.

Parameters

ast_node (ast.Str) – AST node that has been annotated by _annotate_ast_nodes.

Return type

bool

Returns

Whether this string ast node is in docstring position.

pyflyby._parse._ast_str_literal_value(node)
pyflyby._parse._flags_to_try(source, flags, auto_flags, mode)

Flags to try for auto_flags.

If auto_flags is False, then only yield flags. If auto_flags is True, then yield flags and flags ^ print_function.

pyflyby._parse._flatten_ast_nodes(arg)
pyflyby._parse._is_comment_or_blank(line)

Returns whether a line of python code contains only a comment is blank.

>>> _is_comment_or_blank("foo\n")
False
>>> _is_comment_or_blank("  # blah\n")
True
pyflyby._parse._iter_child_nodes_in_order(node)

Yield all direct child nodes of node, that is, all fields that are nodes and all items of fields that are lists of nodes.

_iter_child_nodes_in_order yields nodes in the same order that they appear in the source.

ast.iter_child_nodes does the same thing, but not in source order. e.g. for Dict s, it yields all key nodes before all value nodes.

pyflyby._parse._iter_child_nodes_in_order_internal_1(node)
pyflyby._parse._parse_ast_nodes(text, flags, auto_flags, mode)

Parse a block of lines into an AST.

Also annotate input_flags, source_flags, and flags on the resulting ast node.

Parameters
  • auto_flags (bool) – Whether to guess different flags if text can’t be parsed with flags.

  • mode – Compilation mode: “exec”, “single”, or “eval”.

Return type

ast.Module

pyflyby._parse._split_code_lines(ast_nodes, text)

Split the given ast_nodes and corresponding text by code/noncode statement.

Yield tuples of (nodes, subtext). nodes is a list of ast.AST nodes, length 0 or 1; subtext is a FileText sliced from text.

FileText(…))} for code lines and (None, FileText(...)) for non-code lines (comments and blanks).

pyflyby._parse._test_parse_string_literal(text, flags)

Attempt to parse text. If it parses cleanly to a single string literal, return its value. Otherwise return None.

>>> _test_parse_string_literal(r'"foo\n" r"\nbar"', 0)
'foo\n\\nbar'
pyflyby._parse._walk_ast_nodes_in_order(node)

Recursively yield all child nodes of node, in the same order that the node appears in the source.

ast.walk does the same thing, but yields nodes in an arbitrary order.

pyflyby._parse.infer_compile_mode(arg)

Infer the mode needed to compile arg.

Return type

str