Python’s csv module provides tools for reading and writing CSV files. You can parse CSV data into dictionaries or lists, handle different delimiters, deal with quotes and escapes, and write data back to files.
This lesson covers everything you need to work with CSV files: reading rows as lists or dictionaries, writing data, handling different delimiters, managing headers, and processing large CSV files efficiently. These skills are essential for data analysis, reporting, and system integration.
🕯️ Magic Note
The CSV format is not standardized. Different programs use different delimiters (comma, semicolon, tab), different quote characters, and different escape conventions. Python’s csv module handles these variations gracefully, making it the most robust tool for CSV processing.
Python
import csv
# Sample CSV content (data.csv)
# name,age,city
# Ali,25,Tehran
# Sara,30,Shiraz
# Reza,28,Isfahan
with open(“data.csv”, “r”, encoding=”utf-8″) as f:
reader = csv.reader(f)
for row in reader:
print(row)
# Output:
# [‘name’, ‘age’, ‘city’]
# [‘Ali’, ’25’, ‘Tehran’]
# [‘Sara’, ’30’, ‘Shiraz’]
# [‘Reza’, ’28’, ‘Isfahan’]
Python
import csv
with open(“data.csv”, “r”, encoding=”utf-8″) as f:
reader = csv.DictReader(f)
for row in reader:
print(f”{row[‘name’]} is {row[‘age’]} years old and lives in {row[‘city’]}”)
# Output:
# Ali is 25 years old and lives in Tehran
# Sara is 30 years old and lives in Shiraz
# Reza is 28 years old and lives in Isfahan
# Access fieldnames
print(reader.fieldnames) # [‘name’, ‘age’, ‘city’]
🕯️ Magic Note
DictReader is preferred over reader because it makes your code self-documenting. Instead of row[0] (what is column 0?), you write row[‘name’] (clearly the name field).
Python
import csv
data = [
[“name”, “age”, “city”],
[“Ali”, 25, “Tehran”],
[“Sara”, 30, “Shiraz”],
[“Reza”, 28, “Isfahan”]
]
with open(“output.csv”, “w”, newline=””, encoding=”utf-8″) as f:
writer = csv.writer(f)
writer.writerows(data)
# Write one row at a time
with open(“output2.csv”, “w”, newline=””, encoding=”utf-8″) as f:
writer = csv.writer(f)
writer.writerow([“name”, “age”, “city”])
writer.writerow([“Ali”, 25, “Tehran”])
writer.writerow([“Sara”, 30, “Shiraz”])
Python
import csv
fieldnames = [“name”, “age”, “city”]
data = [
{“name”: “Ali”, “age”: 25, “city”: “Tehran”},
{“name”: “Sara”, “age”: 30, “city”: “Shiraz”},
{“name”: “Reza”, “age”: 28, “city”: “Isfahan”}
]
with open(“output_dict.csv”, “w”, newline=””, encoding=”utf-8″) as f:
writer = csv.DictWriter(f, fieldnames=fieldnames)
writer.writeheader()
writer.writerows(data)
Python
import csv
# Semicolon-delimited file (common in Excel exports when comma is decimal separator)
with open(“data_semicolon.csv”, “r”, encoding=”utf-8″) as f:
reader = csv.reader(f, delimiter=”;”)
for row in reader:
print(row)
# Tab-delimited (TSV format)
with open(“data.tsv”, “r”, encoding=”utf-8″) as f:
reader = csv.reader(f, delimiter=”\t”)
for row in reader:
print(row)
# Pipe-delimited
with open(“data_pipe.txt”, “r”, encoding=”utf-8″) as f:
reader = csv.reader(f, delimiter=”|”)
for row in reader:
print(row)
# Writing with custom delimiter
with open(“output_tsv.tsv”, “w”, newline=””, encoding=”utf-8″) as f:
writer = csv.writer(f, delimiter=”\t”)
writer.writerow([“name”, “age”, “city”])
writer.writerow([“Ali”, 25, “Tehran”])
Python
import csv
data = [
[“name”, “description”],
[“Ali”, “Loves programming, coffee, and magic”], # Contains comma
[“Sara”, ‘She said “Hello” to everyone’], # Contains quotes
[“Reza”, “Line 1\\nLine 2”] # Contains newline
]
# Python handles quoting automatically
with open(“quoted.csv”, “w”, newline=””, encoding=”utf-8″) as f:
writer = csv.writer(f, quoting=csv.QUOTE_MINIMAL)
writer.writerows(data)
# Different quoting options:
# csv.QUOTE_MINIMAL – quote only when necessary (default)
# csv.QUOTE_ALL – quote all fields
# csv.QUOTE_NONNUMERIC – quote all non-number fields
# csv.QUOTE_NONE – never quote (escape delimiters with escapechar)
# Reading back quoted fields
with open(“quoted.csv”, “r”, encoding=”utf-8″) as f:
reader = csv.reader(f)
for row in reader:
print(row)
🕯️ Magic Note
Python’s csv module handles quoting automatically. You rarely need to worry about commas or quotes inside your data. The module will quote fields appropriately.
Python
import csv
from pathlib import Path
def process_large_csv(filename, chunk_size=1000):
“””Process a large CSV file in chunks.”””
with open(filename, “r”, encoding=”utf-8″) as f:
reader = csv.DictReader(f)
batch = []
for row in reader:
batch.append(row)
if len(batch) >= chunk_size:
process_batch(batch)
batch = []
# Process remaining rows
if batch:
process_batch(batch)
def process_batch(batch):
“””Process a batch of rows (e.g., insert into database).”””
print(f”Processing {len(batch)} records…”)
# Do something with the batch
# Memory-efficient: reads one row at a time
with open(“large_file.csv”, “r”, encoding=”utf-8″) as f:
reader = csv.DictReader(f)
total = 0
for row in reader:
total += int(row[“amount”])
print(f”Total: {total}”)
Python
# Note: pandas requires installation: pip install pandas
import pandas as pd
# Read CSV into DataFrame
df = pd.read_csv(“data.csv”)
print(df.head()) # First 5 rows
print(df.describe()) # Statistical summary
print(df[df[“age”] > 25]) # Filter rows
# Write DataFrame to CSV
df.to_csv(“output.csv”, index=False)
🕯️ Magic Note
Pandas is the industry standard for data analysis in Python. It reads CSV files faster than the csv module and provides powerful data manipulation tools. However, it uses more memory. For simple CSV tasks, the built-in csv module is sufficient.
Python
import csv
from collections import defaultdict
from datetime import datetime
def analyze_sales(csv_file):
“””Analyze sales data from a CSV file.”””
# Sample CSV structure:
# date,product,quantity,price
# 2025-01-15,Product A,2,19.99
# 2025-01-16,Product B,1,29.99
# 2025-01-16,Product A,3,19.99
product_sales = defaultdict(int)
monthly_sales = defaultdict(float)
total_revenue = 0.0
with open(csv_file, “r”, encoding=”utf-8″) as f:
reader = csv.DictReader(f)
for row in reader:
product = row[“product”]
quantity = int(row[“quantity”])
price = float(row[“price”])
revenue = quantity * price
product_sales[product] += quantity
total_revenue += revenue
# Group by month
date = datetime.strptime(row[“date”], “%Y-%m-%d”)
month_key = date.strftime(“%Y-%m”)
monthly_sales[month_key] += revenue
# Print report
print(“=” * 50)
print(“SALES REPORT”)
print(“=” * 50)
print(f”\nTotal Revenue: ${total_revenue:,.2f}”)
print(“\n— Product Sales —“)
for product, qty in sorted(product_sales.items(), key=lambda x: x[1], reverse=True):
print(f” {product}: {qty} units sold”)
print(“\n— Monthly Revenue —“)
for month, revenue in sorted(monthly_sales.items()):
print(f” {month}: ${revenue:,.2f}”)
return {
“total_revenue”: total_revenue,
“product_sales”: dict(product_sales),
“monthly_sales”: dict(monthly_sales)
}
# analyze_sales(“sales.csv”)
- Forgetting newline=”” when writing CSV files (extra blank lines on Windows)
- Assuming all values are strings (numbers are stored as strings)
- Not handling encoding issues (use encoding=”utf-8″)
- Closing the file after reading (use with statement)
- Loading entire large CSV into memory (process row by row)
- Assuming the delimiter is always comma (check file format)
- Write code to read a CSV file and print each row as a dictionary.
- How do you write a list of dictionaries to a CSV file with headers?
- What is the difference between csv.reader and csv.DictReader?
- Why do you need newline=”” when writing CSV files?
- How do you read a tab-delimited file?
- Write a function that calculates the average of a numeric column in a CSV file.
⚡ Whisper
CSV files are the universal language of data. Every tool speaks CSV. Excel exports it. Databases import it. APIs return it. Python’s csv module is your translator. It reads messy files with different delimiters. It writes clean files that any program can read. It handles quotes, escapes, and newlines. Use DictReader for clarity. Use DictWriter for convenience. Process large files row by row to save memory. Convert numbers manually. Handle encoding explicitly. CSV is simple but powerful. Master it, and you can exchange data with any system. Spreadsheets become data. Reports become automated. Data flows. The csv module is the bridge. Build it.
