0%

41- Working with Modules & Packages

Import, create, and organize modules and packages. Master the full power of Python’s module system for code organization and reuse.

You understand what modules and packages are. Now it is time to work with them in depth. How do you organize a real project across multiple files? How do you handle imports between files? How do you avoid circular imports? How do you distribute your code as a package? This lesson covers practical techniques for working with modules and packages. You will learn about intra-package imports, the __init__.py file patterns, namespace packages, and how to structure a project for maintainability. You will also learn about the standard library modules that can save you hours of work. By the end of this lesson, you will be able to structure any Python project with confidence.

🕯️ Magic Note

Python’s module system is one of its simplest yet most powerful features. A file is a module. A folder with __init__.py is a package. That is almost all the syntax you need. Yet this simple system enables large projects like Django, NumPy, and TensorFlow to organize millions of lines of code.

Intra-Package Imports (Importing Between Modules in a Package)
When you have multiple modules in a package, you need to import from one module to another. Use relative imports or absolute imports.

Python

# Package structure:

# my_package/

# __init__.py

# math_utils.py

# string_utils.py

# calculator.py

# File: my_package/math_utils.py

def add(a, b):

return a + b

def multiply(a, b):

return a * b

# File: my_package/calculator.py (uses math_utils)

# Absolute import (recommended for clarity)

from my_package.math_utils import add, multiply

def double_add(a, b):

return add(a, b) * 2

# Relative import (shorter, but only works inside a package)

from .math_utils import add, multiply

def triple_add(a, b):

return add(a, b) * 3

💡 Use absolute imports for clarity, especially in larger projects. Use relative imports for convenience within a package, but be aware they cannot be used in scripts run directly as __main__.
The __init__.py File: Advanced Patterns
The __init__.py file is more than just a marker. It can control what gets exposed, run initialization code, and create a clean public interface.

Python

# File: my_package/__init__.py

# Pattern 1: Expose a clean API

from .math_utils import add, multiply, subtract, divide

from .string_utils import reverse, capitalize, is_palindrome

from .calculator import calculate

# Pattern 2: Define what gets imported with star import

__all__ = [“add”, “multiply”, “subtract”, “divide”, “reverse”, “capitalize”, “is_palindrome”, “calculate”]

# Pattern 3: Package-level variables (like version)

__version__ = “1.0.0”

__author__ = “Feloriya”

# Pattern 4: Run initialization code

print(f”Initializing my_package version {__version__}”)

# Pattern 5: Set up logging configuration

import logging

logging.getLogger(__name__).addHandler(logging.NullHandler())

# Now users can write:

# import my_package

# my_package.add(5, 3)

# print(my_package.__version__)

🕯️ Magic Note

The __init__.py file is executed once when the package is first imported. This makes it a perfect place for one-time setup, such as reading configuration files, setting up logging, or registering submodules.

Namespace Packages (Packages Without __init__.py)
Python 3.3 introduced namespace packages. These are packages that span multiple directories and do not require __init__.py files.

Python

# Directory structure:

# project1/

# my_namespace/

# module_a.py

#

# project2/

# my_namespace/

# module_b.py

# If both directories are in sys.path, you can import:

# from my_namespace import module_a

# from my_namespace import module_b

# The two directories combine into a single logical package

⚠️ Namespace packages are an advanced feature. Most projects should stick with regular packages (with __init__.py) for simplicity and clarity.
The Module Cache and Reloading
Modules are cached in sys.modules after import. Importing the same module again does not reload it. Use importlib.reload() to force a reload.

Python

import sys

import importlib

# Modules are cached in sys.modules

print(sys.modules.keys()) # Shows all imported modules

# If you modify a module after import, you can reload it

import my_module

my_module.do_something()

# After changing my_module.py on disk

importlib.reload(my_module) # Force reload

my_module.do_something() # Now uses updated code

🕯️ Magic Note

The module cache is why Python is so fast at importing. Once a module is loaded, subsequent imports are nearly instantaneous. This is why you can import math many times without performance cost.

The dir() Function and Module Inspection
Use dir() to see what names a module defines. It is useful for exploration and debugging.

Python

import math

# List all names defined in the math module

print(dir(math))

# [‘__doc__’, ‘__loader__’, ‘__name__’, ‘__package__’, ‘acos’, ‘acosh’, ‘add’, ‘asin’, …]

# Filter to show only functions (not dunder methods)

print([name for name in dir(math) if not name.startswith(“__”)])

# Get help for a specific function in the module

help(math.sqrt)

Essential Standard Library Modules
Python comes with a rich standard library. Here are some of the most useful modules you should know.
ModulePurposeCommon Functions
osOperating system interfacelistdir(), path.join(), getenv()
sysSystem-specific parametersargv, exit(), path
jsonJSON data handlingload(), dump(), loads(), dumps()
reRegular expressionssearch(), findall(), sub()
datetimeDates and timesdatetime.now(), timedelta
randomRandom number generationrandint(), choice(), shuffle()
collectionsAdditional data structuresdefaultdict, Counter, deque
itertoolsIterator toolschain(), cycle(), product()
functoolsHigher-order functionslru_cache, partial, wraps
pathlibObject-oriented file pathsPath(), read_text(), glob()

Python

# Example: Using multiple standard library modules together

import json

import os

from pathlib import Path

from datetime import datetime

# Read a JSON configuration file

config_path = Path(“config.json”)

if config_path.exists():

with open(config_path, “r”) as f:

config = json.load(f)

print(f”Loaded config at {datetime.now()}”)

else:

print(“Config file not found”)

Creating Executable Modules
You can make a module both importable and executable. Use the __main__ guard for this.

Python

# File: cli_tool.py

import sys

def main():

“””Main entry point for the command-line tool.”””

args = sys.argv[1:]

if not args:

print(“Usage: python cli_tool.py <name>”)

sys.exit(1)

print(f”Hello, {args[0]}!”)

def helper_function():

return “This can be imported too”

if __name__ == “__main__”:

main()

# You can now:

# 1. Run as script: python cli_tool.py Ali

# 2. Import as module: from cli_tool import helper_function

Using __all__ to Control Public API
Define __all__ to specify which names are public. This affects from module import * and helps tools like linters.

Python

# File: api.py

__all__ = [“public_function”, “PublicClass”, “CONSTANT”]

def public_function():

return “This is public”

def _internal_function():

return “This is private by convention”

class PublicClass:

pass

class _InternalClass:

pass

CONSTANT = 42

_INTERNAL_CONSTANT = 99

# Now, from api import * will only import:

# public_function, PublicClass, and CONSTANT

🕯️ Magic Note

The __all__ variable is a promise, not a security measure. Users can still access _internal_function if they know its name. The underscore convention is equally important. Together, they communicate intent clearly.

Circular Imports and How to Avoid Them
A circular import happens when module A imports module B, and module B imports module A. Python can sometimes handle this, but it often causes errors. Avoid them by restructuring your code.

Python

# Problem: Circular import

# module_a.py

# from module_b import func_b

#

# def func_a():

# return func_b()

# module_b.py

# from module_a import func_a

#

# def func_b():

# return func_a()

# Solutions:

# 1. Move shared code to a third module

# 2. Import inside a function (lazy import)

# 3. Reorganize to avoid mutual dependency

# Lazy import solution (not ideal but works)

# module_a.py

def func_a():

from module_b import func_b # Import inside function

return func_b()

⚠️ Circular imports are a design smell. If you encounter them, take it as a sign that your code has too much coupling. Refactor to break the cycle.
Practical Example: Building a Utility Package
Here is a complete example of a small utility package with multiple modules.

Python

# Directory structure:

# feloriya_utils/

# __init__.py

# strings.py

# numbers.py

# lists.py

# File: feloriya_utils/strings.py

def reverse(text):

return text[::-1]

def is_palindrome(text):

cleaned = text.lower().replace(” “, “”)

return cleaned == cleaned[::-1]

# File: feloriya_utils/numbers.py

def is_even(n):

return n % 2 == 0

def is_prime(n):

if n < 2:

return False

for i in range(2, int(n ** 0.5) + 1):

if n % i == 0:

return False

return True

# File: feloriya_utils/lists.py

def unique(items):

return list(set(items))

def flatten(nested_list):

result = []

for item in nested_list:

if isinstance(item, list):

result.extend(flatten(item))

else:

result.append(item)

return result

# File: feloriya_utils/__init__.py

from .strings import reverse, is_palindrome

from .numbers import is_even, is_prime

from .lists import unique, flatten

__version__ = “1.0.0”

__all__ = [“reverse”, “is_palindrome”, “is_even”, “is_prime”, “unique”, “flatten”]

# Usage in another script:

# import feloriya_utils

#

# print(feloriya_utils.reverse(“hello”))

# print(feloriya_utils.is_prime(17))

# print(feloriya_utils.flatten([[1, 2], [3, [4, 5]]]))

Common Mistakes with Modules and Packages
  • Name conflicts with standard library (naming your file random.py breaks import random)
  • Circular imports that cause AttributeError or ImportError
  • Running a script inside a package with relative imports directly (use python -m package.module instead)
  • Modifying sys.path in production code instead of proper packaging
  • Forgetting to include __init__.py in Python 3.2 and earlier (pre-3.3)
  • Using from module import * which obscures the origin of names
Check Your Understanding
  • What is the difference between absolute and relative imports?
  • How do you make a module both importable and executable?
  • What is the purpose of the __init__.py file?
  • Create a package named geometry with modules circle.py and rectangle.py, each containing an area() function. Expose these functions at the package level.
  • What is a circular import and how can you avoid it?
  • Name five useful modules from the standard library and what they do.

⚡ Whisper

A well-organized module is a well-organized mind. Each file has a purpose. Each import is clear. There are no circles. There are no hidden dependencies. The package structure tells a story about how the code is meant to be used. The __init__.py file announces what is public. The __all__ list confirms the promise. The if __name__ == “__main__” guard says “I can stand alone.” This is not just organization. This is respect—for the code, for the reader, and for the future. Projects grow. Teams change. Code lives on. The modules you write today will be imported by someone tomorrow. Make the import simple. Make the interface clear. Make the structure obvious. Your future self will thank you. So will every developer who comes after.

Related posts