100 Days of Learning: Day 24 – Wrapping up on the Python importing process

Photo by Boitumelo Phetla on Unsplash

Here is my Log book.

Code for today.

A handy function

The Definitive Guide to Python import Statements has a neat little bit of code that will show you all the modules that you can import. For my exploration I will wrap this into a function.

Edit example.py and add the following function

def all_available_modules(search_path = ['.']):
    import pkgutil
    all_modules = [x[1] for x in pkgutil.iter_modules(path=search_path)]
    print('All available modules:')
if __name__ == '__main__':
# The file tree at the moment
  ├── example.py
  ├── module1.py
  ├── package1
  │   ├── __init__.py
  │   └── hello.py
  └── time.py

# Run example.py
$ python example.py
All available modules:
['example', 'module1', 'package1', 'time']

# Modify the call to be this and then run it again:

All available modules:
['example', 'module1', 'package1', 'time', '__future__', '_aix_support',
 '_bootlocale', '_bootsubprocess', '_collections_abc', '_compat_pickle',
 '_compression', '_markupbase', '_osx_support', '_py_abc', '_pydecimal',
 '_pyio', '_sitebuiltins', '_strptime', '_sysconfigdata__darwin_darwin',
 '_threading_local', '_weakrefset', 'abc', 'aifc', 'antigravity', 'argparse',
 'ast', 'asynchat', 'asyncio', 'asyncore', 'base64', 'bdb', 'binhex',
... # and the list goes on and on

What is the difference between absolute and relative imports?

  • Absolute imports uses the full path from the project’s root directory to the module to be imported. For example "import package1.subpackage1.module1" or "from package1.subpackage1 import module1"
  • Relative imports uses the relative path from the current module to the module to be imported. Syntax is from .module/package import abc Where each . represents a directory up the chain. For example "from ..abc import xyz" would look for a module named "abc" in the parent directory of the current module.

Note that with a relative import you can only go up the chain to the directory containing the script but not including that directory. You will get ImportError: attempted relative import beyond top-level package if you try.

Let’s explore this. I have created a new package2.subpackage2.subzero.py.

  ├── example.py
  ├── module1.py
  ├── package1
  │   ├── __init__.py
  │   └── hello.py
  ├── package2
  │   └── subpackage2
  │       └── subzero.py
  └── time.py
# hello.py (inside package1)
# Added the following function
def say_hello(name):
    print(f'Hello {name}!')
# subzero.py
import sys
print(f'{__name__} sys.path: {sys.path}')

# This will throw ImportError: attempted relative import beyond top-level package
#from ...package1.hello import say_hello

print('subzero.py will import package1/hello.py absolutely')
from package1.hello import say_hello

$ python example.py
subzero.py will import package1/hello.py absolutely
Hello subzero!

So package2.subpackage2.subzero is able to use an absolute import to import the function say_hello from package1.hello. This is because the sys.path[0] is set to the directory that contains the script being run, which in this case is project.

If you try to do a relative import by walking up the directory tree using …, then you will get the ImportError mentioned earlier.

Key points:

  • Absolute imports is preferred over relative imports.
  • It is better to only import what you need. E.g. from abc import x, y, z . It also makes it clear to other people where things came from.

Wrapping up

The Definitive Guide to Python import Statements has a very good example under "Case 2" of where you run different modules as scripts and thus sys.path will change and you will run into issues. I have personally hit this issue a couple of times now and after my learning the past 3 days I now understand why this fails.

RealPython.com has a really good tutorial about the different ways of importing. Well worth the read. There is also this more advanced tutorial.

How do check if a module has been imported or not? SO post