Caching stores the result of a function call. Next time the same arguments appear, return the stored result instead of recomputing. This is memoization. It is one of the most powerful optimization techniques.
Python provides functools.lru_cache (Least Recently Used cache). It is a decorator that caches function results automatically. You add one line. Your function becomes faster. This tutorial teaches you to use caching effectively—and when not to use it.
🕯️ Magic Note
Memoization was popularized by Donald Michie in 1968. The term combines “memorandum” and “memorization”. Python’s `lru_cache` implementation is thread-safe and can cache thousands of results efficiently.
Python (slow_fibonacci.py)
import time
def fibonacci(n):
if n < 2:
return n
return fibonacci(n – 1) + fibonacci(n – 2)
start = time.perf_counter()
result = fibonacci(35)
elapsed = time.perf_counter() – start
print(f”fibonacci(35) = {result}”)
print(f”Time: {elapsed:.2f} seconds”)
# Time: ~4-5 seconds (depending on hardware)
🕯️ Magic Note
Fibonacci without caching has exponential time complexity O(2ⁿ). It recomputes the same values thousands of times. For n=35, it makes over 18 million function calls. For n=40, over 200 million. This is why caching is essential for recursive algorithms.
Python (fast_fibonacci.py)
from functools import lru_cache
import time
@lru_cache(maxsize=128)
def fibonacci(n):
if n < 2:
return n
return fibonacci(n – 1) + fibonacci(n – 2)
start = time.perf_counter()
result = fibonacci(100)
elapsed = time.perf_counter() – start
print(f”fibonacci(100) = {result}”)
print(f”Time: {elapsed:.6f} seconds”)
# Time: ~0.0005 seconds (instant!)
print(f”Cache info: {fibonacci.cache_info()}”)
# CacheInfo(hits=98, misses=101, maxsize=128, currsize=101)
🕯️ Magic Note
The `@lru_cache` decorator stores results in a dictionary keyed by arguments. `maxsize=128` means keep the 128 most recent results (discard older ones). For Fibonacci, the cache turns O(2ⁿ) into O(n). This is a dramatic improvement.
Python (basic_cache.py)
from functools import lru_cache
call_count = 0
@lru_cache(maxsize=32)
def expensive_function(n):
global call_count
call_count += 1
print(f”Computing for {n}…”)
return n * n
print(expensive_function(5)) # Computing for 5… 25
print(expensive_function(5)) # 25 (from cache, no computation)
print(expensive_function(10)) # Computing for 10… 100
print(expensive_function(5)) # 25 (still cached)
print(f”Actual computations: {call_count}”)
print(f”Cache stats: {expensive_function.cache_info()}”)
# CacheInfo(hits=2, misses=2, maxsize=32, currsize=2)
Python (cache_management.py)
from functools import lru_cache
@lru_cache(maxsize=4)
def square(x):
print(f”Computing square({x})”)
return x * x
# Fill the cache
for i in range(1, 5):
square(i)
print(f”Cache info: {square.cache_info()}”)
# CacheInfo(hits=0, misses=4, maxsize=4, currsize=4)
# This will cause LRU eviction (oldest: 1 is removed)
square(5) # Computing square(5)
square(1) # Computing square(1) again (was evicted)
print(f”Cache info after eviction: {square.cache_info()}”)
# Clear the cache entirely
square.cache_clear()
print(f”After clear: {square.cache_info()}”)
# CacheInfo(hits=0, misses=0, maxsize=4, currsize=0)
Python (cached_api_client.py)
import requests
import time
from functools import lru_cache
@lru_cache(maxsize=100)
def get_user(user_id):
“””Fetch user from API (cached).”””
print(f”🌐 Fetching user {user_id} from API…”)
response = requests.get(f”https://jsonplaceholder.typicode.com/users/{user_id}”)
response.raise_for_status()
return response.json()
print(get_user(1)) # Fetches from network
print(get_user(1)) # Returns cached result
print(get_user(2)) # Fetches from network
print(f”Cache stats: {get_user.cache_info()}”)
🕯️ Magic Note
In production, you might want a time-based expiration for API caches. `lru_cache` does not support TTL (time-to-live). For TTL caches, consider `cachetools` library or implement your own.
Python (cached_db_queries.py)
import sqlite3
from functools import lru_cache
def get_db_connection():
conn = sqlite3.connect(“example.db”)
conn.row_factory = sqlite3.Row
return conn
@lru_cache(maxsize=50)
def get_product(product_id):
“””Get product by ID (cached).”””
print(f”📊 Querying database for product {product_id}…”)
with get_db_connection() as conn:
cursor = conn.execute(“SELECT * FROM products WHERE id = ?”, (product_id,))
row = cursor.fetchone()
return dict(row) if row else None
# First call: queries database
product = get_product(1)
# Second call: returns from cache
same_product = get_product(1)
print(f”Cache stats: {get_product.cache_info()}”)
- **Arguments are not hashable:** (lists, dicts, sets as arguments)
- **Function has side effects:** (modifies global state, writes to files)
- **Return value changes over time:** (time-dependent, file content)
- **Function is already fast:** (overhead of caching may exceed benefit)
- **Memory is extremely limited:** (cache stores all results)
Example where caching fails (unhashable argument)
from functools import lru_cache
@lru_cache(maxsize=32)
def process_list(items): # items is a list (unhashable)
return sum(items)
# This will raise TypeError
# process_list([1, 2, 3]) # TypeError: unhashable type: ‘list’
# Solution: convert to tuple
@lru_cache(maxsize=32)
def process_tuple(items): # items is a tuple (hashable)
return sum(items)
result = process_tuple(tuple([1, 2, 3]))
print(result)
Python (cache_decorator.py)
from functools import cache # Python 3.9+
@cache # Unlimited cache size
def factorial(n):
if n <= 1:
return 1
return n * factorial(n – 1)
print(factorial(10)) # Computes all intermediate values
print(factorial(10)) # Instant (cached)
print(factorial(20)) # Computes only new values (11-20)
Python (ttl_cache.py)
import time
from functools import wraps
from collections import OrderedDict
def ttl_cache(maxsize=128, ttl=60):
“””Cache with time-to-live expiration.”””
def decorator(func):
cache = OrderedDict()
@wraps(func)
def wrapper(*args, **kwargs):
key = (args, tuple(sorted(kwargs.items())))
now = time.time()
# Check if key is in cache and not expired
if key in cache:
value, timestamp = cache[key]
if now – timestamp < ttl:
# Move to end (mark as recently used)
cache.move_to_end(key)
return value
else:
# Expired, remove from cache
del cache[key]
# Compute new value
result = func(*args, **kwargs)
cache[key] = (result, now)
# Enforce maxsize (remove oldest)
if len(cache) > maxsize:
cache.popitem(last=False)
return result
return wrapper
return decorator
@ttl_cache(maxsize=10, ttl=5)
def get_weather(city):
print(f”🌤️ Fetching weather for {city}…”)
return {“city”: city, “temperature”: 22, “condition”: “sunny”}
print(get_weather(“Tehran”)) # Fetches
print(get_weather(“Tehran”)) # From cache
time.sleep(6)
print(get_weather(“Tehran”)) # Expired, fetches again
🕯️ Magic Note
TTL (Time-To-Live) caches are essential for data that changes over time: stock prices, weather, user profiles. The built-in `lru_cache` does not support TTL, so you implement it yourself or use third-party libraries like `cachetools`.
| Library | Features | Use Case |
|---|---|---|
| cachetools | TTL cache, LFU cache, unlimited cache | Advanced caching without external dependencies |
| diskcache | Disk-based cache, persistent across restarts | Large datasets, cross-process caching |
| redis-py | Distributed cache, TTL, pub/sub | Multi-server applications, session storage |
| aiocache | Asyncio support, multiple backends | Async applications |
Python (cache_metrics.py)
from functools import lru_cache
import random
@lru_cache(maxsize=100)
def process_request(user_id, request_type):
# Simulate expensive operation
return f”Processed {request_type} for user {user_id}”
# Simulate requests with some repetition
requests = []
for _ in range(1000):
user_id = random.randint(1, 50)
req_type = random.choice([“login”, “data”, “update”, “delete”])
requests.append((user_id, req_type))
# Process all requests
for user_id, req_type in requests:
process_request(user_id, req_type)
stats = process_request.cache_info()
total = stats.misses + stats.hits
hit_rate = stats.hits / total * 100
print(f”Cache stats: {stats}”)
print(f”Hit rate: {hit_rate:.1f}%”)
print(f”Time saved: {stats.hits} recomputations avoided”)
- Caching functions with side effects (results may be stale)
- Using `lru_cache` with unhashable arguments (TypeError)
- Forgetting that default arguments are part of the cache key
- Caching functions that depend on global state
- Not clearing cache when underlying data changes
Default arguments affect cache keys
from functools import lru_cache
@lru_cache(maxsize=32)
def greet(name, greeting=”Hello”):
print(f”Computing greeting for {name}”)
return f”{greeting}, {name}!”
# These are different cache keys
print(greet(“Ali”)) # Computes
print(greet(“Ali”, “Hello”)) # Computes again (same as default but explicit)
print(greet(“Ali”, greeting=”Hello”)) # Computes again (keyword argument)
print(f”Cache misses: {greet.cache_info().misses}”) # 3
- What does `lru_cache` stand for?
- Write a cached Fibonacci function using `lru_cache`.
- How do you clear the cache of a cached function?
- Why can’t you cache a function that takes a list argument?
- What is the difference between `@cache` and `@lru_cache(maxsize=None)`?
- When would you NOT want to use caching?
⚡ Whisper
Caching is remembering. The function says: “I have seen these arguments before. Here is the answer.” No recomputation. No waiting. No wasted cycles. `@lru_cache` is one line. One line that turns exponential into linear. One line that makes API calls instant. One line that saves database queries. Use it for recursive functions. Use it for expensive calculations. Use it for repeated lookups. But cache wisely. Do not cache side effects. Do not cache time-dependent data. Do not cache unbounded. Learn `cache_info()`. Learn `cache_clear()`. Your code will be faster. Your CPU will be grateful. Your users will notice. Remember the result. Reuse it. That is caching. That is efficiency.
