venv & pip — project hygiene

Coming from Java you are used to a pom.xml or build.gradle that belongs to one project and resolves its own dependencies. Python's default is the opposite: pip install writes into one shared folder for the whole machine, and every project you have ever touched reads from it. That works beautifully until your second project, and the failure mode is not an error message — it is a project that used to run and now doesn't.

venvpiprequirements.txt pip freezedependency drift

One shared folder, many projects

Below, each project pins the version of each library it was written against — perfectly ordinary, nobody has done anything wrong. They all share a single global site-packages, so there is only one version of each library on the machine, and the last pip install wins. Add projects and watch how many still run:

How many of your projects still work?

projects sharing the machine measured over 400 random project sets at each count

Notice this is not a rare edge case — it is the default outcome. Two projects already have worse than even odds. And the breakage is silent: pip install for the new project cheerfully upgrades a library the old project depended on, prints nothing alarming, and the old project fails weeks later when you next open it.

A virtual environment removes the problem rather than managing it. python -m venv .venv creates a directory holding its own site-packages; activating it puts that directory first on the import path, so pip install writes there and imports read there. Every project gets its own, and the share-working column becomes 100% by construction — there is nothing left to collide.

python -m venv .venv          # create — do this once per project
source .venv/bin/activate     # macOS/Linux  (.venv\Scripts\activate on Windows)
pip install -r requirements.txt
deactivate                    # when you are done

Add .venv/ to .gitignore. The environment is a build artifact, not source — what you commit is the list of what to install, which is the next problem.

The same requirements file, six months later

A requirements.txt is a shopping list, and pip fills it with whatever is newest at the time unless you say otherwise. There are three levels of care, and the middle one is where most projects sit — along with most of the surprises. Nine packages here: three you asked for and six pulled in behind them:

How much moved since you wrote the file?

months since you wrote it packages keep releasing; a fresh install picks up whatever is current

The middle row is the trap worth internalising. You pinned pandas==2.1.4, so pandas is exactly what you asked for — and numpy, which pandas pulls in, is not pinned by anybody, so it moves. Your build is reproducible in the packages you thought about and unreproducible in the ones you didn't, which is exactly the shape of bug that takes a day to find.

pip freeze > requirements.txt   # every installed package, exact version — direct AND transitive

That is the cheapest lock file there is, and it turns the bottom row on. Its weakness is that it does not record why anything is there, so the file grows and nobody dares delete a line — which is the problem uv, Poetry and PDM exist to solve: they keep the short list you maintain separate from the exact lock file they generate. For Phase 0, plain venv + pip freeze is entirely enough; know that the tools exist and why.

⚠️ Traps & honesty: the collision numbers come from a simulation where each project needs an exact major version of three libraries out of six — real projects specify ranges, which collide less often but also fail more confusingly when they do · the release calendar here is invented; the point is the shape of drift, not any package's real cadence · a venv is not isolation from the operating system — it shares the Python interpreter and anything installed at system level, so a C library mismatch still bites · conda solves a different, larger problem (non-Python binaries), which is why data-science stacks often use it instead · pinning everything forever has its own cost: you stop getting security fixes, so lock deliberately and update on purpose.

Takeaways: pip install writes to one shared folder by default, and with a single machine-wide site-packages only a minority of projects keep working — two projects are already at 56% · python -m venv .venv gives each project its own, which removes the collision instead of managing it; .gitignore the folder, commit the list · an unpinned requirements file drifts on every package within a year; pinning only your direct dependencies still leaves 6 of 9 free to move · pip freeze is the cheap lock file — exact versions for direct and transitive alike · uv/Poetry exist to keep the list you maintain separate from the lock they generate. Next: dataclasses.

Second opinion (taught here — these corroborate): venv docs · Python Packaging User Guide · uv.