Comment by hamishwhc
2 months ago
The author’s point about “not caring about pip vs poetry vs uv” is missing that uv directly supports this use case, including PyPI dependencies, and all you need is uv and your preferred Python version installed: https://docs.astral.sh/uv/guides/scripts/#using-a-shebang-to...
Actually you can go one better:
Then you don't even need python installed. uv will install the version of python you specified and run the command.
alternatively, uv lets you do this:
The /// script block is actually specified in PEP 723 and supported by several other tools apart from uv.
4 replies →
I’ve started migrating all of my ~15 years of one-off python scripts to have this front matter. Right now, I just update when/if I use them. I keep thinking if were handier with grep/sed/regex etc, I’d try to programmatically update .pys system-wide. But, many aren’t git tracked/version controlled, just laying in whatever dir they service(d). I’ve several times started a “python script dashboard” or “hacky tools coordinator” but stop when I remember most of these are unrelated (to each-other) and un/rarely used. I keep watching the chatter and thinking this is probably an easy task for codex, or some other agent but these pys are “mine” (and I knew^ how they worked when I wrote^ them) and also, they’re scattered and there’s no way I’m turning an agent loose on my file system.
^mostly, some defs might have StackOverflow copy/pasta
1 reply →
This is an awesome features for quick development.
I'm sure the documentation of this featureset highlights what I'm about to say but if you're attracted to the simplicity of writing Python projects who are initialized using this method, do not use this code in staging/prod.
If you don't see why this is not production friendly it's for the simple a good.reaaon that creating deployable artifacts packaging a project or a dependency of a project this uses this method, creating reproducible builds becomes impossible.
This will also lead to builds that pass your CI but fail to run in their destination environment and vice versa due to the fact that they download heir dependencies on the fly.
There may be workarounds and I know nothing of this feature so investigate yourself if you must.
My two cents.
This isn't really "alternatively"; it's pointing out that in addition to the shebang you can add a PEP 723 dependency specification that `uv run` (like pipx, and some other tools) can take into account.
I'm actually a bit annoyed that uv won. I found pdm to be a really nice solution that fixed a lot of the issues poetry had without the hard ideological stance behind it, while fixing most of its problems. But maybe that ideology was partly what drove it's adoption.
Rust is getting this feature too, it's great for one off scripts
Yeah, but you need `uv`. If we are reaching out for tools that might not be around, then you can also depend on nix-shell,
Yeah, but you need Nix. If we are reaching out for tools that might not be around, then you can also depend on `curl | sudo bash` to install Nix when not present.
(this is a joke btw)
4 replies →
As shared in a sibling comment, you can get away with just curl+shell: https://paulw.tokyo/standalone-python-script-with-uv/
The issue I have with `nix-shell` is that the evaluation time is long, so if you need to run the script repeatedly it may take a long time. `nix shell` at least fix this issue by caching evaluations, but I think uv is still faster.
This comes with the added benefit that your environment is reverted as soon as you exit the Nix shell.
3 replies →
That shebang will work on GNU link based systems, but might not work elsewhere. I know that’s the most popular target, but not working on macOS, BSDs, or even busybox.
I just tried the one you are replying to and it worked great on macOS. I frequently use a variant of this on my Mac.
2 replies →
And with some small shebang trick, you don't even need to have uv installed [1], just curl and a posix shell
[1] https://paulw.tokyo/standalone-python-script-with-uv/
> Then you don't even need python installed. uv will install the version of python you specified and run the command
What you meant was, "you don't need python pre-installed". This does not solve the problem of not wanting to have (or limited from having) python installed.
I thought that too, but I think the tricky bit is if you're a non-python user, this isn't yet obvious.
If you've never used Clojure and start a Clojure project, you will almost definitely find advice telling you to use Leiningen.
For Python, if you search online you might find someone saying to use uv, but also potentially venv, poetry or hatch. I definitely think uv is taking over, but its not yet ubiquitous.
Ironically, I actually had a similar thing installing Go the other day. I'd never used Go before, and installed it using apt only to find that version was too old and I'd done it wrong.
Although in that case, it was a much quicker resolution than I think anyone fighting with virtual environments would have.
That's my experience. I'm not a Python developer, and installing Python programs has been a mess for decades, so I'd rather stay away from the language than try another new tool.
Over the years, I've used setup.py, pip, pipenv (which kept crashing though it was an official recommendation), manual venv+pip (or virtualenv? I vaguely remember there were 2 similar tools and none was part of a minimal Python install). Does uv work in all of these cases? The uv doc pointed out by the GP is vague about legacy projects, though I've just skimmed through the long page.
IIRC, Python tools didn't share their data across projects, so they could build the same heavy dependencies multiple times. I've also seen projects with incomplete dependencies (installed through Conda, IIRC) which were a major pain to get working. For many years, the only simple and sane way to run some Python code was in a Docker image, which has its own drawbacks.
> Does uv work in all of these cases?
Yes. The goal of uv is to defuck the python ecosystem and they're doing a very good job at it so far.
26 replies →
> IIRC, Python tools didn't share their data across projects, so they could build the same heavy dependencies multiple times.
One of the neatest features of uv is that it uses clever symlinking tricks so if you have a dozen different Python environments all with the same dependency there's only one copy of that dependency on disk.
1 reply →
I would not be putting up with Python if not for uv. It’s that good.
Before uv came along I was starting to write stuff in Go that I’d normally write in Python.
2 replies →
That's partly because python has a very large installed base, and ease of entry (including distribution). This leads to people running into issues quicker, and many alternative solutions.
Unlike something like Rust, which has much fewer users (though growing) and requires PhDs in Compiler Imprecation and Lexical Exegetics.
Or C++ which has a much larger installed base but also no standard distribution method at all, and an honorary degree in Dorsal Artillery.
uv solved it, it’s safe to come back now.
There's definitely a philosophical shift that you can observe happening over the last 12-15 years or so, where at the start you have the interpreter as the centre of the world and at the end there's an ecosystem management tool that you use to give yourself an interpreter (and virtual environments, and so on) per project.
I think this properly kicked off with RVM, which needed to come into existence because you had this situation where the Ruby interpreter was going through incompatible changes, the versions on popular distributions were lagging, and Rails, the main reason people were turning to Ruby, was relatively militant about which interpreter versions it would support. Also, building the interpreter such that it would successfully run Rails wasn't trivial. Not that hard, but enough that a convenience wrapper mattered. So you had a whole generation of web devs coming up in an environment where the core language wasn't the first touchpoint, and there wasn't an assumption that you could (or should) rely on what you could apt-get install on the base OS.
This is broadly an extremely good thing.
But the critical thing that RVM did was that it broke the circular dependency at the core of the problem: it didn't itself depend on having a working ruby interpreter. Prior to that you could observe a sort of sniffiness about tools for a language which weren't implemented in that language, but RVM solved enough of the pain that it barged straight past that.
Then you had similar tools popping up in other languages - nvm and leiningen are the first that spring to mind, but I'd also throw (for instance) asdf into the mix here - where the executable that you call to set up your environment has a '#!/bin/bash' shebang line.
Go has sidestepped most of this because of three things: 1) rigorous backwards compatibility; 2) the simplest possible installation onramp; 3) being timed with the above timeline so that having a pre-existing `go` binary provided by your OS is unlikely unless you install it yourself. And none of those are true of Python. The backwards compatibility breaks in this period are legendary, you almost always do have a pre-existing Python to confuse things, and installing a new python without breaking that pre-existing Python, which your OS itself depends on, is a risk. Add to that the sniffiness I mentioned (which you can still see today on `uv` threads) and you've got a situation where Python is catching up to what other languages managed a decade ago.
Again.
It is sort of funny, if we squint just the wrong way, “ecosystem management tool first, then think about interpreters” starts to look a lot like… a package manager, haha.
> you might find someone saying to use uv, but also potentially venv, poetry or hatch.
This is sort of like saying "You might find someone saying to drive a Ford, but also potentially internal combustion engine, Nissan or Hyundai".
Only to those already steeped in Python. To an outsider they're all equally arbitrary non-descriptive words and there's not even obvious proper noun capitalization to tell apart a component from a tool brand.
6 replies →
I imagine by this they meant `python -m venv` specifically, using that interface directly, rather than through another wrapper CLI tool.
2 replies →
Do you think a non-python user would piece it together if the shebang line reveals what tool to use?
I think yes if that line was UV. But otherwise, of its just python, you have the issue that you need two tools, one for running scripts and one for managing dependencies and environments.
> If you've never used Clojure and start a Clojure project, you will almost definitely find advice telling you to use Leiningen.
I thought the current best practice for Clojure was to use the shiny new built-in tooling? deps.edn or something like that?
Clojure CLI (aka deps.edn) came out in 2018 and in the survey "how do you manage your dependencies?" question crossed 50% usage in early 2020. So for 6-8 years now.
1 reply →
deps.edn is becoming the default choice, yes. I interpreted the parent comment as saying "you will see advice to use leiningen (even though newer solutions exist, simply because it _was_ the default choice when the articles were written)"
1 reply →
uv has been around for less than two years. It’s on track to become the default choice, it’s just a matter of time.
I solved this in 2019 with PyFlow, but no one used it, so I lost interest. It's an OSS tool written in rust that automatically and transparently manages python versions and venvs. You just setup a `pyproject.toml`, run `pyflow main.py` etc, and it just works. Installs and locks dependencies like Cargo, installs and runs the correct Python version for the project etc.
At the time, Poetry and Pipenv were the popular tools, but I found they were not sufficient; they did a good job abstracting dependencies, but not venvs and Python version.
sounds awesome. Just out of interest, why do you think pyflow didn't catch on, but UV did?
My best guess: I'm bad at marketing, and gave up too soon. The feedback I received was generally "Why would I use this when Pip, Pipenv and Poetry work fine?". To me they didn't; they were a hassle due to not handling venvs and Py versions, but I didn't find many people to also have had the same problem.
1 reply →
Polish and that uv gets you entire python interpreters automatically without having to compile or manually install them.
That in retrospective was what made rye temporarily attractive and popular.
I've moved over mostly to uv too, using `uv pip` when needed but mostly sticking with `uv add`. But as soon as you start using `uv pip` you end up with all the drawbacks of `uv pip`, namely that whatever you pass after can affect earlier dependency resolutions too. Running `uv pip install dep-a` and then `... dep-b` isn't the same as `... dep-b` first and then `... dep-a`, or the same as `uv pip install dep-a dep-b` which coming from an environment that does proper dependency resolution and have workspaces, can be really confusing.
This is more of a pip issue than uv though, and `uv pip` is still preferable in my mind, but seems Python package management will forever be a mess, not even the bandaid uv can fix things like these.
Ive been away from python for awhile now, I was under the impression uv was somehow solving this dependency hell. Whats the benefit of using uv/pip together? Speed?
As far as I can tell, `pip` by itself still doesn't even do something basic as resolving the dependency tree first, then download all the packages in parallel, as an basic example. The `uv pip` shim does.
And regardless if you use only uv, or pip-via-uv, or straight up pip, dependencies you install later steps over dependencies you installed earlier, and no tool so far seems to try to solve this, which leads me to conclude it's a Python problem, not a package manager problem.
`uv pip` is still uv, it's just uv's compatibility layer for pip.
i found uv frustrating. i dont know what problem is it trying to solve. it's not a tool for managing virtualenvs, but it does them as well. i guess it's a tool for dependency management. the "uv tool" stuff. kinda weird. i gave it an honest try but i was working around it with shell functions all the time.
in the end i went back to good old virtualenvwrapper.sh and setting PYTHONPATH. full control over what goes into the venv and how. i guess people like writing new tools. i can understand that.
Maybe I "entered" the Python ecosystem at a different time, but I never used virtualenvwrapper.sh nor sat PYTHONPATH manually ever. When I first came into contact with Python, I think doing `virtuelenv venv && source venv/bin/activate` was what was recommended to me at the time. Eventually I used `python -m venv` but always also with `pip` and a `requirements.txt`. I pretty much stuck with that until maybe 1 year ago I started playing around with `uv`, and for me, I just use `uv venv|pip|init|add` from uv, and nothing else from any other tools, and generally do pretty basic stuff.
Maybe for more complex projects and use cases it's harder, but it's a lot faster than just pip and pyproject.toml is a lot nicer to manage than `requirements.txt`, so that's two easy enough wins for me to move over.
uv solves many problems, but one textbook case is the problem of running some arbitrary Python-based command-line tool, where 1/you don’t have any Python interpreter installed, 2/your OS-provided Python interpreter isn’t compatible with the tool, or 3/you want to run single or multiple tools from any arbitrary folder where your data already is, as opposed to adapting your workflow to fit the virtualenv or running the risk that two tools have conflicting dependencies that would make the virtualenv not work well.
2 replies →
you don't even need you prefered python version, uv will download it.
There are really so many things about this point that I don't get.
First off, in my mind the kinds of things that are "scripts" don't have dependencies outside the standard library, or if they do are highly specific to my own needs on my own system. (It's also notable that one of the advantages the author cites for Go in this niche is a standard library that avoids the need for dependencies in quick scripts! Is this not one of Python's major selling points since day 1?)
Second, even if you have dependencies you don't have to learn differences between these tools. You can pick one and use it.
Third, virtual environments are literally just a place on disk for those dependencies to be installed, that contains a config file and some stubs that are automatically set up by a one-liner provided by the standard library. You don't need to go into them and inspect anything if you don't want to. You don't need to use the activation script; you can just specify the venv's executable instead if you prefer. None of it is conceptually difficult.
Fourth, sharing an environment for these quick scripts actually just works fine an awful lot of the time. I got away with it for years before proper organization became second nature, and I would usually still be fine with it (except that having an isolated environment for the current project is the easiest way to be sure that I've correctly listed its dependencies). In my experience it's just not a thing for your quick throwaway scripts to be dependent on incompatible Numpy versions or whatever.
... And really, to avoid ever having to think about the dependencies you provide dynamically, you're going to switch to a compiled language? If it were such a good idea, nobody would have thought of making languages like Python in the first place.
And uh...
> As long as the receiving end has the latest version of go, the script will run on any OS for tens of years in the future. Anyone who's ever tried to get python working on different systems knows what a steep annoying curve it is.
The pseudo-shebang trick here isn't going to work on Windows any more than a conventional one is. And no, when I switched from Windows to Linux, getting my Python stuff to work was not a "steep annoying curve" at all. It came more or less automatically with acclimating to Linux in general.
(I guess referring to ".pyproject" instead of the actually-meaningful `pyproject.toml` is just part of the trolling.)
> Third, virtual environments are literally just a place on disk for those dependencies
I had a recent conversation with a colleague. I said how nice it is using uv now. They said they were glad because they hated messing with virtualenvs so much that preferred TypeScript now. I asked them what node_modules is, they paused for a moment, and replied “point taken”.
Uv still uses venvs because it’s the official way Python stores all the project packages in one place. Node/npm, Go/go, and Rust/cargo all do similar things, but I only really here people grousing about Python’s version, which as you say, you can totally ignore and never ever look at.
From my experience, it seems like a lot of the grousing is from people who don't like the "activation script" workflow and mistakenly think it's mandatory. Though I've also seen aesthetic objections to the environment actually having internal structure rather than just being another `site-packages` folder (okay; and what are the rules for telling Python to use it?)
The very long discussion (https://discuss.python.org/t/pep-582-python-local-packages-d...) of PEP 582 (https://peps.python.org/pep-0582/ ; the "__pypackages__" folder proposal) seems relevant here.
3 replies →
Won't those dependencies then be global? With potential conflicts as a result?
uv uses a global cache but hardlinks the dependencies for your script into a temp venv that is only for your script, so its still pretty fast.
Nope! uv takes care of that. uv is a work of art.
Then I should seriously take a look at it. I figured it was just another package manager.
....but you have to be able to get UV and on some platforms (e.g. a raspberry pi) it won't build because the version of rust is too old. So I wrote a script called "pv" in python which works a bit like uv - just enough to get my program to work. It made me laugh a bit, but it works anywhere, well enough for my usecase. All I had to do was embed a primitive AI generated TOML parser in it.
> All I had to do was embed a primitive AI generated TOML parser in it.
The standard recommendation for this is `tomli`, which became the basis of the standard library `tomllib` in 3.11.