_autoimp module

exception pyflyby._autoimp.LoadSymbolError
class pyflyby._autoimp.ScopeStack(arg)

A stack of namespace scopes, as a tuple of dict s.

Each entry is a dict.

Ordered from most-global to most-local. Builtins are always included. Duplicates are removed.

_abc_impl = <_abc_data object>
_cached_has_star_import = False
clone_top()

Return a new ScopeStack referencing the same namespaces as self, but cloning the topmost namespace (and aliasing the others).

has_star_import()

Return whether there are any star-imports in this ScopeStack. Only relevant in AST-based static analysis mode.

merged_to_two()

Return a 2-tuple of dicts.

These can be used for functions that take a globals and locals argument, such as eval.

If there is only one entry, then return it twice.

If there are more than two entries, then create a new dict that merges the more-global ones. The most-local stack will alias the dict from the existing ScopeStack.

Return type

tuple of (dict, dict)

with_new_scope(include_class_scopes=False, new_class_scope=False)

Return a new ScopeStack with an additional empty scope.

Parameters
  • include_class_scopes – Whether to include previous scopes that are meant for ClassDefs.

  • new_class_scope – Whether the new scope is for a ClassDef.

Return type

ScopeStack

class pyflyby._autoimp._ClassScope
pyflyby._autoimp._IMPORT_FAILED = {}

Set of imports we’ve already attempted and failed.

class pyflyby._autoimp._MissingImportFinder(scopestack, find_unused_imports=False, parse_docstrings=False)

A helper class to be used only by _find_missing_imports_in_ast.

This class visits every AST node and collects symbols that require importing. A symbol requires importing if it is not already imported or otherwise defined/assigned in this scope.

For attributes like “foo.bar.baz”, we need to be more sophisticated:

Suppose the user imports “foo.bar” and then accesses “foo.bar.baz.quux”. Baz may be already available just by importing foo.bar, or it may require further import. We decide as follows. If foo.bar is not a module, then we assume whatever’s under it can’t be imported. If foo.bar is a module but does not have a ‘baz’ attribute, then it does require import.

_NewScopeCtx(**kwargs)

Context manager that temporarily pushes a new empty namespace onto the stack of namespaces.

_UpScopeCtx()

Context manager that temporarily moves up one in the scope stack

_check_load(fullname, scopestack, lineno)
_finish_deferred_load_checks()
_scan_node(node)
_scan_unused_imports()
_visit_Load(fullname)
_visit_Store(fullname, value=None)
_visit_StoreImport(node, modulename)
_visit_fullname(fullname, ctx)
find_missing_imports(node)
generic_visit(node)

Generic visitor that visits all of the node’s field values, in the order declared by node._fields.

Called if no explicit visitor function exists for a node.

scan_for_import_issues(codeblock)
visit(node)

Visit a node.

visit_Assign(node)
visit_Attribute(node)
visit_ClassDef(node)
visit_Delete(node)
visit_Dict(node)
visit_DictComp(node)
visit_ExceptHandler(node)
visit_FunctionDef(node)
visit_GeneratorExp(node)
visit_ImportFrom(node)
visit_Lambda(node)
visit_ListComp(node)
visit_Name(node)
visit_SetComp(node)
visit_alias(node, modulename=None)
visit_arg(node)
visit_arguments(node)
visit_comprehension(node)
class pyflyby._autoimp._UseChecker(name, source, lineno)

An object that can check whether it was used.

used = False
pyflyby._autoimp._find_earliest_backjump_label(bytecode)

Find the earliest target of a backward jump.

These normally represent loops.

For example, given the source code:

>>> def f():
...     if foo1():
...         foo2()
...     else:
...         foo3()
...     foo4()
...     while foo5():  # L7
...         foo6()

In python 2.6, the disassembled bytecode is:

>>> import dis
>>> dis.dis(f) 
  2           0 LOAD_GLOBAL              0 (foo1)
              3 CALL_FUNCTION            0
              6 JUMP_IF_FALSE           11 (to 20)
              9 POP_TOP

  3          10 LOAD_GLOBAL              1 (foo2)
             13 CALL_FUNCTION            0
             16 POP_TOP
             17 JUMP_FORWARD             8 (to 28)
        >>   20 POP_TOP

  5          21 LOAD_GLOBAL              2 (foo3)
             24 CALL_FUNCTION            0
             27 POP_TOP

  6     >>   28 LOAD_GLOBAL              3 (foo4)
             31 CALL_FUNCTION            0
             34 POP_TOP

  7          35 SETUP_LOOP              22 (to 60)
        >>   38 LOAD_GLOBAL              4 (foo5)
             41 CALL_FUNCTION            0
             44 JUMP_IF_FALSE           11 (to 58)
             47 POP_TOP

  8          48 LOAD_GLOBAL              5 (foo6)
             51 CALL_FUNCTION            0
             54 POP_TOP
             55 JUMP_ABSOLUTE           38
        >>   58 POP_TOP
             59 POP_BLOCK
        >>   60 LOAD_CONST               0 (None)
             63 RETURN_VALUE

The earliest target of a backward jump would be the ‘while’ loop at L7, at bytecode offset 38:

>>> _find_earliest_backjump_label(f.__code__.co_code) 
38

Note that in this example there are earlier targets of jumps at bytecode offsets 20 and 28, but those are targets of _forward_ jumps, and the clients of this function care about the earliest _backward_ jump.

If there are no backward jumps, return an offset that points after the end of the bytecode.

Parameters

bytecode (bytes) – Compiled bytecode, e.g. function.__code__.co_code.

Return type

int

Returns

The earliest target of a backward jump, as an offset into the bytecode.

pyflyby._autoimp._find_loads_without_stores_in_code(co, loads_without_stores)

Find global LOADs without corresponding STOREs, by disassembling code. Recursive helper for _find_missing_imports_in_code.

Parameters
  • co (types.CodeType) – Code object, e.g. function.__code__

  • loads_without_stores (set) – Mutable set to which we add loads without stores.

Returns

None

pyflyby._autoimp._find_missing_imports_in_ast(node, namespaces)

Find missing imports in an AST node. Helper function to find_missing_imports.

>>> node = ast.parse("import numpy; numpy.arange(x) + arange(x)")
>>> _find_missing_imports_in_ast(node, [{}])
[DottedIdentifier('arange'), DottedIdentifier('x')]
Return type

list of DottedIdentifier

pyflyby._autoimp._find_missing_imports_in_code(co, namespaces)

Find missing imports in a code object. Helper function to find_missing_imports.

>>> f = lambda: foo.bar(x) + baz(y)
>>> [str(m) for m in _find_missing_imports_in_code(f.__code__, [{}])]
['baz', 'foo.bar', 'x', 'y']
>>> f = lambda x: (lambda: x+y)
>>> _find_missing_imports_in_code(f.__code__, [{}])
[DottedIdentifier('y')]
Return type

list of str

pyflyby._autoimp._op(c)
pyflyby._autoimp._try_import(imp, namespace)

Try to execute an import. Import the result into the namespace namespace.

Print to stdout what we’re about to do.

Only import into namespace if we won’t clobber an existing definition.

Parameters
  • imp (Import or str) – The import to execute, e.g. “from numpy import arange”

  • namespace (dict) – Namespace to import into.

Returns

True on success, False on failure

pyflyby._autoimp.auto_import_symbol(fullname, namespaces, db=None, autoimported=None, post_import_hook=None)

Try to auto-import a single name.

Parameters
  • fullname (str) – Fully-qualified module name, e.g. “sqlalchemy.orm”.

  • namespaces (list of dict, e.g. [globals()].) – Namespaces to check. Namespace[-1] is the namespace to import into.

  • db (ImportDB) – Import database to use.

  • autoimported – If not None, then a dictionary of identifiers already attempted. auto_import will not attempt to auto-import symbols already in this dictionary, and will add attempted symbols to this dictionary, with value True if the autoimport succeeded, or False if the autoimport did not succeed.

  • post_import_hook (callable) – A callable that is invoked if an import was successfully made. It is invoked with the Import object representing the successful import

Return type

bool

Returns

True if the symbol was already in the namespace, or the auto-import succeeded; False if the auto-import failed.

pyflyby._autoimp.clear_failed_imports_cache()

Clear the cache of previously failed imports.

pyflyby._autoimp.get_known_import(fullname, db=None)

Get the deepest known import.

For example, suppose:

  • The user accessed “foo.bar.baz”,

  • We know imports for “foo”, “foo.bar”, and “foo.bar.quux”.

Then we return “import foo.bar”.

Parameters

fullname (DottedIdentifier) – Fully-qualified name, such as “scipy.interpolate”

pyflyby._autoimp.load_symbol(fullname, namespaces, autoimport=False, db=None, autoimported=None)

Load the symbol fullname.

>>> import os
>>> load_symbol("os.path.join.__name__", {"os": os})
'join'
>>> load_symbol("os.path.join.asdf", {"os": os})
Traceback (most recent call last):
...
pyflyby._autoimp.LoadSymbolError: os.path.join.asdf: AttributeError: 'function' object has no attribute 'asdf'
>>> load_symbol("os.path.join", {})
Traceback (most recent call last):
...
pyflyby._autoimp.LoadSymbolError: os.path.join: NameError: os
Parameters
  • fullname (str) – Fully-qualified symbol name, e.g. “os.path.join”.

  • namespaces (dict or list of dict) – Namespaces to check.

  • autoimport – If False (default), the symbol must already be imported. If True, then auto-import the symbol first.

  • db (ImportDB) – Import database to use when autoimport=True.

  • autoimported – If not None, then a dictionary of identifiers already attempted. auto_import will not attempt to auto-import symbols already in this dictionary, and will add attempted symbols to this dictionary, with value True if the autoimport succeeded, or False if the autoimport did not succeed.

Returns

Object.

Raises

LoadSymbolError – Object was not found or there was another exception.

pyflyby._autoimp.scan_for_import_issues(codeblock, find_unused_imports=True, parse_docstrings=False)

Find missing and unused imports, by lineno.

>>> arg = "import numpy, aa.bb as cc\nnumpy.arange(x)\narange(x)"
>>> missing, unused = scan_for_import_issues(arg)
>>> missing
[(2, DottedIdentifier('x')), (3, DottedIdentifier('arange')), (3, DottedIdentifier('x'))]
>>> unused
[(1, Import('from aa import bb as cc'))]
Parameters

parse_docstrings

Whether to parse docstrings. Compare the following examples. When parse_docstrings=True, ‘bar’ is not considered unused because there is a string that references it in braces:

>>> scan_for_import_issues("import foo as bar, baz\n'{bar}'\n")
([], [(1, Import('import baz')), (1, Import('import foo as bar'))])
>>> scan_for_import_issues("import foo as bar, baz\n'{bar}'\n", parse_docstrings=True)
([], [(1, Import('import baz'))])

pyflyby._autoimp.symbol_needs_import(fullname, namespaces)

Return whether fullname is a symbol that needs to be imported, given the current namespace scopes.

A symbol needs importing if it is not previously imported or otherwise assigned. namespaces normally includes builtins and globals as well as symbols imported/assigned locally within the scope.

If the user requested “foo.bar.baz”, and we see that “foo.bar” exists and is not a module, we assume nothing under foo.bar needs import. This is intentional because (1) the import would not match what is already in the namespace, and (2) we don’t want to do call getattr(foo.bar, “baz”), since that could invoke code that is slow or has side effects.

Parameters
  • fullname (DottedIdentifier) – Fully-qualified symbol name, e.g. “os.path.join”.

  • namespaces (list of dict) – Stack of namespaces to search for existing items.

Return type

bool

Returns

True if fullname needs import, else False