Detect Unused Imports

This is a very basic usage: look for DEF without any USE, and warn about them, focusing on imported values.

>>> import beniget, gast as ast

# parse some simple statements
>>> code = "from math import cos, sin; print(cos(3))"
>>> module = ast.parse(code)

# compute the def-use chains at module level
>>> duc = beniget.DefUseChains()
>>> duc.visit(module)

# grab the import statement
>>> imported = module.body[0].names

# inspect the users of each imported name
>>> for name in imported:
...   ud = duc.chains[name]
...   if not ud.users():
...     print("Unused import: {}".format(ud.name()))
Unused import: sin

At that point, the astute reader has already noted that due to the dynamic nature of Python, one can fool this analysis by calling the eval function, eventually through an indirection, or by performing a lookup into globals(). More about this later.

Compute a Function's Closure

In Python, inner functions (and lambdas) can capture identifiers defined in the outer scope. This analysis computes such identifiers by registering all USE from a local DEF, then walking through all identifier and checking whether they're one of the USE.

An alternative approach would be to rely on the use-def chains to inspect all the DEF of each USE and ensure the DEF come from the visited function.

>>> import gast as ast
>>> import beniget
>>> class Capture(ast.NodeVisitor):
...
...     def __init__(self, module_node):
...         # initialize def-use chains
...         self.chains = beniget.DefUseChains()
...         self.chains.visit(module_node)
...         self.users = set()  # users of local definitions
...         self.captured = set()  # identifiers that don't belong to local users
...
...     def visit_FunctionDef(self, node):
...         # initialize the set of node using a local variable
...         for def_ in self.chains.locals[node]:
...             self.users.update(use.node for use in def_.users())
...         self.generic_visit(node)
...
...     def visit_Name(self, node):
...         # register load of identifiers not locally defined
...         if isinstance(node.ctx, ast.Load):
...             if node not in self.users:
...                 self.captured.add(node.id)

>>> code = 'def foo(x):\n def bar(): return x\n return bar'
>>> module = ast.parse(code)
>>> inner_function = module.body[0].body[0]
>>> capture = Capture(module)
>>> capture.visit(inner_function)
>>> list(capture.captured)
['x']

Detecting NameError

Any USE without DEF is probably (and no, not certainly) an error. But even if there's an associated DEF, it could be an error; Consider the following:

from random import random
if random() > 0.5:
    a = 1
print(a)

There's a chance that a is unbound while executing print(a). It would be possible to combine beniget with a dummy code transformation to detect this issue by generating dummy top-level definitions and checking if they have any USE:

a = random = None  # if any of those have a USE, then we have a potential issue
from random import random
if random() > 0.5:
    a = 1
print(a)

Limitations

It's Python. So introspection and lazy binding are a pain for any static analysis tool. There's nothing we can do against

• eval and exec
• star import from some_module import * even if beniget, in a very conservative way, assumes that such import can define any identifier, which means it's likely to have a lot of USE!
• assigning to globals() or locals()

And plenty of other similar stuff. I can't blame you for using these features, that's part of Python nature

Also, note that beniget analysis is not data dependant, so if 1: a = 1 does not unconditionally defines a!

Installation and stuff

beniget is available on PyPI and GitHub under BSD 3-clause.

It's tested using tox on Python 2.7 and 3.6.

It's already used as a foundation package of Pythran and Transonic.

Acknowledgment

Thanks a lot to Pierre Augier for motivating the project and LEGI for funding it!

Also, thanks a bunch to Ashwin Vishnu, Pierrick Brunet and Jean Laroche for proof reading this post o/