How Python Finds Modules

Mental Model

Imagine Python's module search path (sys.path) as a list of specific drawers it checks for modules. When you run python your_script.py, only the your_script.py's directory is automatically put into the first drawer. If your app module is in a higher-level drawer, Python won't look there unless explicitly told.

Rule: Never execute nested module files directly as scripts; instead, invoke them as modules using the -m flag from your project root.

The Setup

You are running a script inside a nested subdirectory (tests/integration/test_api.py) and it fails to import a module from the root folder, even though running from the root folder works perfectly.

What Does This Print?

⚠ Broken code

Python

# file: project_root/tests/integration/test_api.py
import sys

try:
    # The developer attempts to import from project_root/app/core.py
    from app.core import DatabaseClient
    print("Database client imported successfully.")
except ModuleNotFoundError as e:
    print(f"Failed! sys.path is: {sys.path[:1]}")
    print(f"Error: {e}")

If you run python tests/integration/test_api.py directly from the terminal, will the import of app.core succeed?

The Output

What actually happens

Failed! sys.path is: ['/project_root/tests/integration'] Error: No module named 'app'

The import fails because Python cannot locate the app package in its search paths. When you execute a script directly, Python automatically inserts the directory containing that script into the first position of sys.path. Since the directory /project_root/tests/integration does not contain a subdirectory named app, the lookup fails, completely ignoring the root project folder. Python does not automatically resolve imports relative to your current terminal shell directory, leading to a breakdown in package path routing when nested utility files are executed. This behavior causes discrepancies between local testing runs and automated module execution pathways.

Why Python Does This

CPython resolves imports by systematically iterating through the list of directory paths stored in sys.path. When executing an interactive shell or utilizing the module flag -m, CPython populates sys.path[0] with an empty string, which directs the interpreter to look inside the current working directory first. Conversely, running a script file directly triggers CPython to locate the absolute path of the directory containing that script, placing it at sys.path[0]. This isolation mechanism prevents localized scripts from accidentally masking system library imports, but prevents nested components from referencing parent-level directories unless the workspace is installed as an editable package.

The Fix

✓ Corrected pattern

Python

# Run the script as a module from the project root instead of altering sys.path programmatically:
# Command: python -m tests.integration.test_api

# Or programmatically handle it safely using pathlib inside the script:
import sys
from pathlib import Path

root_path = str(Path(__file__).resolve().parents[2])
if root_path not in sys.path:
    sys.path.insert(0, root_path)

# Now the relative resolution functions perfectly
# from app.core import DatabaseClient

Using python -m <module_name> tells Python to treat the specified module as a package and adds the current working directory (where you run the command from) to sys.path. If you run it from your project root, the root itself is added, allowing Python to correctly find all subpackages like app.

How This Fails in Real Systems

A cron job script located in scripts/cron/backup.py failed nightly with ModuleNotFoundError because the system crontab executed the script directly. The team spent 4 days trying to fix environment variables when the root cause was that execution of a nested script changes sys.path[0] to that script's directory, bypassing the project's root site-packages.

Key Takeaway

Never execute nested module files directly as scripts; instead, invoke them as modules using the -m flag from your project root.

Common mistake: Developers assume that Python will automatically figure out the project root and add it to sys.path when executing a script from a nested directory, leading to ModuleNotFoundError for modules outside the current script's immediate path.