Python code quality in the Gen AI era (#dev #python #ai #codequality #testing)

Honest question: how many lines of Python did an agent write for you this week? How many of those are pure microslop held together by wishful thinking and spit? And how many of those did you actually read, line by line? Yeah, that's what I thought. Code got cheap, reviewing didn't, and "the model seemed confident" is not at all a quality gate. In this post we'll set up the tools that help keep a Python codebase in shape when half the commits come from your LLM, and the other half from your coworkers "secretly" using Gen AI to create code.

To be clear, I'm not here to tell you to stop using agents, on the contrary: Use it to your hearts' content! The idea here is to show (or remind) you about tools that will help you keep your codebase in shape, hopefully reduce the impact of hallucinations, and avoid things like "you asked AI to fix X, and it did. But now Y stopped working".

Why bother? The AI writes the code now

That's exactly why.

When you typed every line yourself, code was generated just as fast as your fingers could press the keys. Quality tooling was nice to have, but you could sort of keep the codebase in your head because you had personally suffered through writing it. Now an agent produces a whole module, docstrings included, plus a cheerful summary of how production-ready it is, in the time it takes you to refill your coffee.

Like it or not, the bottleneck moved from writing code to judging it.

You could add "always write clean, idiomatic Python, following PEP8 guidelines" to your agent instructions. I'm sure it will be taken very seriously, right next to "do not hallucinate".

Even in the best of scenarios, a prompt is a suggestion; but a linter is a gate.

Deterministic tools give the same verdict for the same code every single run, and they close the loop: an agent that gets back B006 Do not use mutable data structures for argument defaults fixes it on the first try. An agent that gets back "please make it cleaner" hands you the same code with three new adjectives in the docstrings, and a set of comments in the code full of em-dashes.

There's a second win that's easy to miss: every tool below is configured in pyproject.toml, inside the repo. Humans, CI, and whatever agent opens the folder next week all inherit the same rules. Nobody has to remember anything, which is good, because nobody will anyway.

So, let's start from the basics...

uv: stop babysitting virtual environments

You know the ritual: create the venv, activate it, install stuff, forget to activate it next month and install stuff into the system Python, then spend twenty minutes playing "why is this import failing". And the requirements.txt you swore you'd keep updated? (You didn't.)

uv is my favorite package and project manager for Python, made by Astral (the Ruff folks) and written in Rust. It replaces pip, virtualenv, pip-tools, and pipx with one binary, and the docs claim a 10-100x speedup over pip. In practice, the environment step just stops being something you notice.

Install it once, globally:

1# Linux / macOS
2curl -LsSf https://astral.sh/uv/install.sh | sh
3
4# Windows
5powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"

A new project looks like this:

1uv init demo-api
2cd demo-api
3uv add httpx
4uv add --dev pytest ruff mypy black
5uv run python main.py

Notice what's missing: no python -m venv, no activate, no deactivate, no pip freeze. The environment exists (a regular .venv, created on demand), you just never manage it by hand.

Two features carry the whole thing:

• uv.lock. Every uv add or uv remove updates a lockfile with the exact resolved version of every dependency, transitive ones included. Commit it. On any other machine, in CI, or in an agent sandbox, uv sync rebuilds exactly that environment. Versions only move when you ask (uv lock --upgrade), which kills two classics at once: "works on my machine" and "the agent bumped a library to fix a bug and nobody noticed for three weeks". (Think of package-lock.json on a ReactJS project.)
• uv run. It runs any command inside the project environment, syncing it first if the lockfile changed. uv run pytest, uv run mypy ., uv run python script.py. There's no activation state to get wrong, which matters a lot when the one typing commands is an agent: "forgot to activate the venv" becomes an entire category of failure that stops existing.

Bonus points: uv python pin 3.13 pins the interpreter version for the project (uv downloads it if the machine doesn't have it), and uvx <tool> runs one-off tools in a throwaway environment, pipx-style.

With just uv, you're already in a better place than just using pip + requirements.txt, but without too much effort, we can improve even more.

Ruff

Ruff is a Python linter, also from Astral, also written in Rust (so you know it's fast). It reimplements the rules from Flake8, isort, pyupgrade, flake8-bugbear, and a long list of other plugins in a single tool, and it lints a real codebase in milliseconds.

That speed changes how you use it: checks this cheap run after every agent iteration, instead of once, grudgingly, right before the PR.

In case you're curious about the tools that Ruff covers:

• isort automatically sorts, groups, and formats Python imports.
• pyupgrade automatically rewrites Python code to use newer, more concise language syntax based on the project’s minimum Python version.
• Flake8 is an extensible Python linter that checks code for syntax errors, likely mistakes, style violations, and complexity issues.
• flake8-bugbear is a Flake8 plugin providing additional opinionated checks for likely bugs and questionable design patterns.

It's worth noting that none of those tools are dead, but (IMO) Ruff displaced them for many cases when people start new Python projects.

Before I digress too much, let's get back to business: Ruff configuration goes in pyproject.toml.

 1[tool.ruff]
 2line-length = 120
 3target-version = "py313"
 4
 5[tool.ruff.lint]
 6select = [
 7    "E", "W",  # pycodestyle
 8    "F",       # pyflakes (undefined names, unused imports...)
 9    "I",       # isort (import ordering)
10    "B",       # flake8-bugbear (likely bugs, like mutable default args)
11    "UP",      # pyupgrade (modern syntax)
12    "SIM",     # flake8-simplify (needlessly convoluted code)
13]

Out of the box Ruff enables only a small subset (pyflakes plus a few pycodestyle groups), so it won't yell at anyone until you tell it to. The selection above is (IMO) a solid starting point that catches real bugs without drowning you in pedantry. Tweak from there.

Day to day, you need exactly two commands:

1uv run ruff check          # lint everything
2uv run ruff check --fix    # lint and auto-fix what's safe to fix

The output format is made for tight feedback loops: file, line, rule code, message. Let the agent run the command itself, and the fix usually lands correctly on the first pass because there's nothing to interpret. B006 doesn't have a "well, actually" mode.

Mypy

I particularly like the type annotations in Python, but I also know some pretty good developers that find them annoying. Mypy kind of helps with that, because it does a type checking both by inference and by annotations.

Mypy reads your annotations and verifies that what flows through the code matches what the signatures promise, without running anything.

This one earns its keep in the Gen AI era, because signatures are exactly where models slip: a keyword argument that existed in an older version of the library (or never), a str handed to something expecting a Path, a branch that returns None while the annotation says it can't.

1[tool.mypy]
2python_version = "3.13"
3strict = true
4
5[[tool.mypy.overrides]]
6module = ["some_untyped_lib.*"]
7ignore_missing_imports = true

Run it with:

1uv run mypy .

On a brand-new project, turn strict = true on from day one; it costs nothing while there are zero errors. On an existing codebase, strict mode will greet you with a wall of errors, so start with the defaults and tighten module by module using [[tool.mypy.overrides]]. Less dramatic, same destination.

The strict mode means that Mypy will try to enforce explicit type annotations on the signature of your functions.

Side note: annotations pay twice with agents. They're promises Mypy checks, and they're context the model reads. A well-typed signature tells the agent what a function accepts without it having to guess from usage examples.

Black

Black calls itself "the uncompromising code formatter", and that's the whole pitch: it has almost no options, so there's nothing to argue about. You trade your formatting opinions for never having a formatting discussion again. You'll survive. I did...(I think...)

1[tool.black]
2line-length = 120
3target-version = ["py313"]

1uv run black .          # format the project
2uv run black --check .  # CI mode: fail if anything would be reformatted

Formatting sounds cosmetic until you're reviewing agent output at volume. With a formatter in place, whitespace is settled before the diff exists, so the review attention you have left goes to the parts that can actually break. It also normalizes the model's creative layout choices, and there will be creative layout choices.

Side note: Ruff also ships a formatter (uv run ruff format), designed as a drop-in replacement for Black, same style. If you like the idea of one binary doing lint plus format, use it and skip Black entirely.

Tip: Pick exactly one formatter per repo! two formatters with different opinions will happily reformat each other's output forever. I'm covering Black here because it's the incumbent, and odds are your existing projects already use it.

pytest

Last but definitely not least: The testing framework.

Everything above checks code without running it. Tests check what the code actually does, and in a codebase where agents commit, the suite picks up a second job: it's the feedback loop that tells the agent (and you) whether a change broke something. This is the one section I'll ask you to take seriously even if you skim the rest.

pytest is the de-facto standard: plain assert statements, readable failure output, fixtures, parametrization, and a plugin for everything else.

1uv add --dev pytest

1[tool.pytest.ini_options]
2testpaths = ["tests"]

 1# tests/test_pricing.py
 2import pytest
 3
 4from demo_api.pricing import apply_discount
 5
 6
 7def test_applies_percentage_discount():
 8    assert apply_discount(100.0, 0.1) == 90.0
 9
10
11def test_rejects_negative_discount():
12    with pytest.raises(ValueError):
13        apply_discount(100.0, -0.5)

1uv run pytest

Quick tip/Best practice for usage with AI agents: Keep an eye out on the test files. LLMs have the bad habit of breaking the code and changing the tests so the broke code won't generate test failures. So if you see big changes on test files, pushback and make sure the agent is not breaking the code and changing the rules of the game.

Making agents actually run it

Agents are surprisingly good at running tests... when told to. So tell them, in writing, in the repo. Whatever instruction file your tool reads (CLAUDE.md, AGENTS.md, CONTRIBUTING.md, a rules file), put the contract there: run uv run pytest and uv run ruff check before declaring anything done, and a red suite means not done (mentioning the failures casually in the summary doesn't count).

Then enforce the same commands in CI, because agents open pull requests now, and CI is the reviewer that doesn't get tired.

A suite that's green before the change and green after is what makes agent iteration safe-ish. Without one, the agent's "this should work now" is just a vibe session that (probably) is just adding more microslop to the world.

The agent generated the code, but the responsibility is yours

Now the trap: the agent writes the tests too, and agent-written tests have their own failure modes. The ones I keep running into:

• Tests that can't fail. assert result is not None on a function that always returns something. Green forever, worth nothing.
• Mock everything, test the mock. Patch enough of the world and the test verifies your patches, not your code.
• Tests welded to implementation details. Rename a private helper, and the suite goes red; introduce an actual bug, and it stays green.
• The green-at-any-cost move. A failing test gets "fixed" by weakening the assertion or by deleting the test. This is the dangerous one (I mentioned before) because the suite keeps looking healthy while it slowly stops meaning anything.

So here's the rule I'd tattoo on every agent-assisted project: the human signs off on the tests. Review them like production code, because they are. Two questions do most of the work: if I broke this behavior on purpose, would a test fail? And would the suite survive a refactor that keeps behavior intact? When an agent wants to delete or weaken a test, that's not its call. It explains why, and you decide.

And don't chase the coverage percentage. Coverage tells you which lines ran. It says nothing about whether the assertions around those lines mean anything, and 90% coverage made of hollow assertions is worse than 60% made of real ones, because the 90% feels safe.

Wrapping up

None of this is exotic: one binary for the project, one pyproject.toml carrying the config for everything, and a test suite with an owner. On a new project this is minutes of setup. On an existing one, maybe an afternoon. After that, every line of Python that enters the repo, typed by you or generated between two coffee refills, has to clear the same bar.

Hope it helps! :)

References:

• https://docs.astral.sh/uv/
• https://docs.astral.sh/ruff/
• https://docs.astral.sh/ruff/formatter/
• https://mypy.readthedocs.io/en/stable/config_file.html
• https://black.readthedocs.io/en/stable/
• https://docs.pytest.org/en/stable/