0%

52- Introduction to Advanced Modules

Explore Python’s powerful standard library modules for specialized tasks. Datetime, math, random, re, and more. Expand your toolkit beyond the basics.

Python comes with a rich standard library. You have already used some modules like random, math, and datetime. But there are many more. Advanced modules solve specific problems. Need to work with dates and times? datetime has you covered. Need regular expressions for pattern matching? re is your tool. Need efficient data structures? collections provides them. Need to generate random numbers? random and secrets are essential. This lesson introduces the most useful advanced modules in Python’s standard library. You will learn what each module does and when to reach for it. These modules will save you hours of reinventing the wheel. They are battle-tested, well-documented, and ready to use.

🕯️ Magic Note

The Zen of Python says: “There should be one—and preferably only one—obvious way to do it.” The standard library embodies this principle. For most common tasks, the standard library provides a solution. Before writing a complex function, check if the standard library already has it.

The os Module: Operating System Interface
The os module provides functions for interacting with the operating system: file system operations, environment variables, and process management.

Python

import os

from pathlib import Path

# Working with directories

current_dir = os.getcwd() # Get current working directory

print(f”Current directory: {current_dir}”)

os.listdir(“.”) # List files in directory

os.mkdir(“new_folder”) # Create a new directory

os.rmdir(“empty_folder”) # Remove empty directory

os.makedirs(“parent/child/grandchild”, exist_ok=True) # Create nested directories

# Working with files

os.path.exists(“file.txt”) # Check if file exists

os.path.isfile(“file.txt”) # Check if it is a file

os.path.isdir(“folder”) # Check if it is a directory

os.path.getsize(“file.txt”) # Get file size in bytes

os.path.getmtime(“file.txt”) # Last modification time

# Joining paths (platform independent)

path = os.path.join(“folder”, “subfolder”, “file.txt”)


# On Windows: folder\subfolder\file.txt

# On Linux/Mac: folder/subfolder/file.txt

# Environment variables

home = os.environ.get(“HOME”, “/default/path”)

os.environ[“MY_VAR”] = “some_value” # Set environment variable

# Running shell commands

os.system(“echo Hello”) # Run shell command

🕯️ Magic Note

pathlib is a newer, object-oriented alternative to os.path. It is often more intuitive. Both are useful, but many modern Python projects prefer pathlib for path operations.

The sys Module: System-Specific Parameters
The sys module provides access to interpreter-level variables and functions.

Python

import sys

# Command-line arguments

print(f”Script name: {sys.argv[0]}”)

print(f”Arguments: {sys.argv[1:]}”)

# Python version

print(f”Python version: {sys.version}”)

print(f”Version info: {sys.version_info}”)

# Exiting the program

if len(sys.argv) < 2:

print(“Usage: python script.py <argument>”)

sys.exit(1) # Exit with error code

# Module search path

print(f”Module search path: {sys.path}”)

# Platform information

print(f”Platform: {sys.platform}”) # ‘win32’, ‘linux’, ‘darwin’, etc.

# Standard input/output/error streams

sys.stdout.write(“Hello”) # Same as print()

sys.stderr.write(“Error message”)

# Maximum recursion depth

print(f”Recursion limit: {sys.getrecursionlimit()}”)

sys.setrecursionlimit(10000) # Increase limit (use carefully)

The datetime Module: Dates and Times
The datetime module provides classes for manipulating dates and times.

Python

from datetime import datetime, date, time, timedelta

# Current date and time

now = datetime.now()

print(f”Now: {now}”)

print(f”Date: {now.date()}”)

print(f”Time: {now.time()}”)

print(f”Year: {now.year}, Month: {now.month}, Day: {now.day}”)

# Creating specific dates

birthday = date(2025, 5, 15)

meeting = datetime(2025, 6, 20, 14, 30, 0)

# Date arithmetic (timedelta)

tomorrow = now + timedelta(days=1)

next_week = now + timedelta(weeks=1)

three_hours_later = now + timedelta(hours=3)

yesterday = now – timedelta(days=1)

# Formatting dates (strftime)

formatted = now.strftime(“%Y-%m-%d %H:%M:%S”)

print(f”Formatted: {formatted}”) # 2025-05-10 14:30:00

formatted_readable = now.strftime(“%A, %B %d, %Y”)

print(f”Readable: {formatted_readable}”) # Saturday, May 10, 2025

# Parsing strings to dates (strptime)

date_string = “2025-12-25 09:30:00”

parsed = datetime.strptime(date_string, “%Y-%m-%d %H:%M:%S”)

print(f”Parsed: {parsed}”)

# Comparing dates

if now > meeting:

print(“Meeting has passed”)

else:

days_left = (meeting – now).days

print(f”Meeting in {days_left} days”)

The math Module: Mathematical Functions
The math module provides mathematical functions and constants.

Python

import math

# Constants

print(f”π = {math.pi}”)

print(f”e = {math.e}”)

print(f”τ = {math.tau}”) # 2π

print(f”Infinity: {math.inf}”)

print(f”Not a Number: {math.nan}”)

# Basic functions

print(f”ceil(3.2) = {math.ceil(3.2)}”) # 4 (rounds up)

print(f”floor(3.9) = {math.floor(3.9)}”) # 3 (rounds down)

print(f”round(3.14159, 2) = {round(3.14159, 2)}”) # 3.14 (built-in round)

print(f”trunc(3.7) = {math.trunc(3.7)}”) # 3 (removes decimal)

# Powers and roots

print(f”sqrt(16) = {math.sqrt(16)}”) # 4.0

print(f”pow(2, 3) = {math.pow(2, 3)}”) # 8.0

print(f”hypot(3, 4) = {math.hypot(3, 4)}”) # 5.0 (Euclidean distance)

# Exponential and logarithmic

print(f”exp(2) = {math.exp(2)}”) # e² ≈ 7.389

print(f”log(100, 10) = {math.log(100, 10)}”) # 2.0 (log base 10)

print(f”log2(8) = {math.log2(8)}”) # 3.0

print(f”log10(1000) = {math.log10(1000)}”) # 3.0

# Trigonometric functions (angles in radians)

angle = math.radians(60) # Convert 60 degrees to radians

print(f”sin(60°) = {math.sin(angle)}”)

print(f”cos(60°) = {math.cos(angle)}”)

print(f”tan(45°) = {math.tan(math.radians(45))}”)

# Angular conversion

print(math.degrees(math.pi)) # 180.0

print(math.radians(180)) # 3.14159…

The random Module: Random Number Generation
The random module provides functions for generating random numbers and selecting random items.

Python

import random

# Basic random number generation

print(f”Random float [0.0, 1.0): {random.random()}”)

print(f”Random integer [1, 100]: {random.randint(1, 100)}”)

print(f”Random float [5.0, 10.0]: {random.uniform(5.0, 10.0)}”)

# Choosing random elements

colors = [“red”, “green”, “blue”, “yellow”, “purple”]

print(f”Random choice: {random.choice(colors)}”)

print(f”Multiple random choices (with replacement): {random.choices(colors, k=3)}”)

print(f”Multiple random choices (without replacement): {random.sample(colors, k=3)}”)

# Shuffling sequences

cards = list(range(1, 11))

random.shuffle(cards)

print(f”Shuffled cards: {cards}”)

# Weighted choices

items = [“apple”, “banana”, “cherry”]

weights = [0.5, 0.3, 0.2]

print(f”Weighted choice: {random.choices(items, weights=weights, k=1)[0]}”)

# Setting seed for reproducibility

random.seed(42)

print(f”Deterministic random: {random.randint(1, 100)}”) # Same each run

⚠️ The random module is suitable for simulations and games, but not for security (passwords, tokens, cryptography). For security-sensitive applications, use the secrets module instead.
The secrets Module: Cryptographically Secure Randomness
The secrets module generates cryptographically strong random numbers for security applications.

Python

import secrets

import string

# Generate secure random numbers

print(f”Secure random integer: {secrets.randbelow(100)}”) # 0-99

print(f”Secure random bits: {secrets.randbits(16)}”) # 0-65535

# Generate secure tokens

print(f”URL-safe token: {secrets.token_urlsafe(32)}”)

print(f”Hex token: {secrets.token_hex(16)}”)

print(f”Bytes token: {secrets.token_bytes(16)}”)

# Generate random password

alphabet = string.ascii_letters + string.digits + string.punctuation

password = “”.join(secrets.choice(alphabet) for _ in range(12))

print(f”Secure password: {password}”)

# Timing-safe comparison (prevents timing attacks)

input_password = “user_password”

stored_hash = “stored_hash_value”

if secrets.compare_digest(input_password, stored_hash):

print(“Passwords match (securely)”)

🕯️ Magic Note

Use random for simulations, games, and testing. Use secrets for passwords, tokens, and anything security-related. The difference is critical.

The re Module: Regular Expressions
The re module provides regular expression matching operations for powerful pattern-based string processing.

Python

import re

# Searching for patterns

text = “The price is $19.99 and $29.99”

match = re.search(r”\$\d+\.\d{2}”, text)

if match:

print(f”Found: {match.group()}”) # $19.99 (first match only)

# Find all matches

all_prices = re.findall(r”\$\d+\.\d{2}”, text)

print(f”All prices: {all_prices}”) # [‘$19.99’, ‘$29.99’]

# Replacing patterns

replaced = re.sub(r”\$\d+\.\d{2}”, “$XX.XX”, text)

print(f”Redacted: {replaced}”)

# Splitting with regular expressions

data = “apple, banana; cherry: date”

split_data = re.split(r”[,;:]”, data)

print(f”Split: {[s.strip() for s in split_data]}”)

# Compiling patterns for performance (reuse)

email_pattern = re.compile(r”[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}”)

text_with_emails = “Contact us at support@example.com or admin@test.org”

emails = email_pattern.findall(text_with_emails)

print(f”Emails: {emails}”)

💡 Regular expressions are powerful but can be hard to read. Use raw strings (r”…”) for regex patterns to avoid escaping backslashes. Test your patterns with tools like regex101.com before coding.
The collections Module: Specialized Data Structures
The collections module provides alternative data structures beyond lists, tuples, and dictionaries.

Python

from collections import defaultdict, Counter, deque, namedtuple, OrderedDict

# defaultdict: dictionary with default values

dd = defaultdict(list)

dd[“fruits”].append(“apple”)

dd[“fruits”].append(“banana”)

dd[“vegetables”].append(“carrot”)

print(dict(dd)) # {‘fruits’: [‘apple’, ‘banana’], ‘vegetables’: [‘carrot’]}

# Counter: count occurrences

words = [“apple”, “banana”, “apple”, “cherry”, “banana”, “apple”]

word_count = Counter(words)

print(word_count) # Counter({‘apple’: 3, ‘banana’: 2, ‘cherry’: 1})

print(f”Most common: {word_count.most_common(2)}”)

# deque: double-ended queue (efficient append/pop from both ends)

dq = deque([1, 2, 3])

dq.appendleft(0)

dq.append(4)

print(dq) # deque([0, 1, 2, 3, 4])

print(dq.pop()) # 4

print(dq.popleft()) # 0

dq.rotate(1) # Rotate right

print(dq) # deque([3, 1, 2])

# namedtuple: tuple with named fields

Point = namedtuple(“Point”, [“x”, “y”])

p = Point(10, 20)

print(f”Point: x={p.x}, y={p.y}”)

print(f”Index access: {p[0]}, {p[1]}”)

🕯️ Magic Note

The defaultdict eliminates if key in dict checks. Counter is perfect for frequency analysis. deque is ideal for queues and stacks. namedtuple gives you lightweight objects without writing a full class.

The pathlib Module: Object-Oriented File Paths
The pathlib module provides an object-oriented interface for working with file paths.

Python

from pathlib import Path

# Creating paths

home = Path.home()

current = Path.cwd()

file_path = Path(“data”, “config.json”)

print(f”Home: {home}”)

# Path operations (using / operator)

config = home / “.config” / “myapp” / “settings.ini”

print(f”Config path: {config}”)

# Checking existence and properties

if config.exists():

print(f”File exists, size: {config.stat().st_size} bytes”)

print(f”Is file: {config.is_file()}”)

print(f”Parent: {config.parent}”)

print(f”Name: {config.name}”)

print(f”Suffix: {config.suffix}”)

print(f”Stem (without suffix): {config.stem}”)

# Reading and writing (convenience methods)

file = Path(“example.txt”)

file.write_text(“Hello, world!”)

content = file.read_text()

print(f”Content: {content}”)

# Directory operations

Path(“new_folder”).mkdir(exist_ok=True)

for item in Path(“.”).iterdir():

print(f” {item.name} ({‘DIR’ if item.is_dir() else ‘FILE’})”)

# Glob patterns (recursive search)

for py_file in Path(“.”).glob(“**/*.py”):

print(f”Python file: {py_file}”)

💡 pathlib is more intuitive and less error-prone than os.path. For new code, prefer pathlib for path operations. It works on all platforms.
The json Module: JSON Data Handling
The json module encodes and decodes JSON data, essential for APIs and configuration files.

Python

import json

# Python data to JSON (serialization)

data = {

“name”: “Feloriya”,

“age”: 25,

“skills”: [“Python”, “Web Design”],

“is_active”: True,

“score”: 95.5

}

json_string = json.dumps(data, indent=2)

print(f”JSON string:\n{json_string}”)

# JSON to Python data (deserialization)

parsed = json.loads(json_string)

print(f”Parsed name: {parsed[‘name’]}”)

# Reading/writing JSON files

with open(“data.json”, “w”) as f:

json.dump(data, f, indent=2)

with open(“data.json”, “r”) as f:

loaded = json.load(f)

print(f”Loaded: {loaded[‘name’]}”)

Common Mistakes with Advanced Modules
  • Using random for security-critical applications (use secrets)
  • Manually manipulating paths with string concatenation (use pathlib or os.path.join)
  • Not handling timezone issues with datetime (use pytz or zoneinfo for timezone-aware datetime)
  • Forgetting to compile regex patterns when used repeatedly (performance)
  • Not checking return values of re.search before calling .group()
  • Using lists when defaultdict or Counter would be cleaner
Check Your Understanding
  • How do you find all files with a .txt extension in a directory using pathlib?
  • What is the difference between random and secrets?
  • Write a regex that matches email addresses.
  • What does collections.Counter do?
  • How do you get the current date and time in datetime?
  • When would you use a deque instead of a list?

⚡ Whisper

The standard library is a treasure chest. os for the system. sys for the interpreter. datetime for time. math for calculations. random for chance. re for patterns. collections for better containers. pathlib for paths. json for data exchange. Each module is a tool. Each solves a common problem. The best Python programmers do not write everything from scratch. They know the standard library. They reach for defaultdict instead of checking keys manually. They use pathlib instead of string concatenation. They reach for Counter instead of counting manually. Do not reinvent the wheel. The standard library wheel is already there, tested, documented, and ready. Learn these modules. Practice them. Soon you will reach for them without thinking. And your code will be shorter, clearer, and more reliable. This is not laziness. This is wisdom.

Related posts