Practical data workflows
Clean and validate CSV rows
Convert CSV strings and report a bad row.
Lesson 60 of 63
Reading: about 2 min
Lesson + practice: about 17 min
Difficulty: Intermediate
On this page5 sections
csv.DictReader returns strings for field values. Convert minutes explicitly and attach the row number to errors so a user can find the source.
For file practice, download the Python practice files v1.0.0 bundle. The examples write their own files, so the samples stay unchanged. Work on a copy if you edit a sample.
Worked example
import csv
from io import StringIO
source = StringIO("subject,minutes\nPython,20\nSQL,25\n")
rows = []
reader = csv.DictReader(source)
if reader.fieldnames != ["subject", "minutes"]:
raise ValueError("unexpected CSV columns")
for row in reader:
line_number = reader.line_num
if None in row or not isinstance(row["subject"], str) or not row["subject"].strip():
raise ValueError(f"invalid row on line {line_number}")
subject = row["subject"].strip()
try:
value = int(row["minutes"])
except (TypeError, ValueError) as error:
raise ValueError(f"bad minutes on line {line_number}") from error
if value <= 0:
raise ValueError(f"invalid row on line {line_number}")
rows.append((subject, value))
print(rows)[('Python', 20), ('SQL', 25)]Why this works
The header is line 1, so the first data row ends on line 2. reader.line_num reports the physical line where the current CSV record ends, including a record with a quoted newline. strip() normalises surrounding subject whitespace, and int() makes numeric operations possible.
Another case
import csv
from io import StringIO
print(next(csv.DictReader(StringIO("subject,minutes\nPython,20\n")))["minutes"])20
Practice
Try a row with minutes 0. What should the validation report? Then run the same checks on the sample sessions.csv from the practice bundle by opening the file instead of using StringIO; you should get [('Python', 20), ('SQL', 25), ('Python', 30)].
Check the practice result
Reveal answer
It raises ValueError for an invalid row at that line number.
Try another case
In the same CSV, a third line SQL,ten replaces SQL,25. What error appears?
Check your answer
Reveal answer
ValueError("bad minutes on line 3"), raised from the failed int() conversion.
Your progress
Reveal the check answer or reach the end to complete this lesson.