Comment by suspended_state
3 months ago
I am not well versed in python programming, this is just my opinion as an outsider.
For anyone interested in using these tools, I suggest reading the following:
https://www.reddit.com/r/Python/comments/10zdidm/why_type_hi...
That post should probably be taken lightly, but I think that the goal there is to understand that even with the best typing tools, you will have troubles, unless you start by establishing good practices.
For example, Django is large code base, and if you look at it, you will observe that the code is consistent in which features of python are used and how; this project passes the stricter type checking test without troubles. Likewise, Meta certainly has a very large code base (why develop a type checker otherwise?), and they must have figured out that they cannot let their programmers write code however they like; I guess their type checker is the stricter one for that reason.
Python, AFAIK, has many features, a very permissive runtime, and perhaps (not unlike C++) only some limited subset should be used at any time to ensure that the code is manageable. Unfortunately, that subset is probably different depending on who you ask, and what you aim to do.
(Interestingly, the Reddit post somehow reminded me of the hurdles Rust people have getting the Linux kernel guys to accept their practice: C has a much simpler and carefree type system, but Rust being much more strict rubs those C guys the wrong way).
The top comment in that post shuts down the whole nonsense pretty quickly and firmly:
> If you have a super-generic function like that and type hinting enforced, you just use Any and don't care about it.
It's a stupid example, but even within the context of a `slow_add` function in a library: maybe the author originally never even thought people would pass in non-numeric values, so in the next version update instead of a hardcoded `time.sleep(0.1)` they decide to `time.sleep(a / b)`. Oops, now it crashes for users who passed in strings or tuples! If only there were a way to declare that the function is only intended to work with numeric values, instead of forcing yourself to provide backwards compatibility for users who used that function in unexpected ways that happened to work.
IMO: for Python meant to run non-interactively with any sort of uptime guarantees, type checking is a no-brainer. You're actively making a mistake if you choose to not add type checking.
As the author of that post, I'd like to point out the example was meant to be stupid.
The purpose was to show different ideologies and expectations on the same code don't work, such as strict backwards compatibilities, duck typing, and strictly following linting or type hinting rules (due to some arbitrary enforcement). Although re-reading it now I wish I'd spent more than an evening working on it, it's full of issues and not very polished.
> If you have a super-generic function like that and type hinting enforced, you just use Any and don't care about it.
Following the general stupidness of the post: they are now unable to do that because a security consultant said they have to enable and can not break RUFF rule ANN401: https://docs.astral.sh/ruff/rules/any-type/
> Following the general stupidness of the post: they are now unable to do that because a security consultant said they have to enable and can not break RUFF rule ANN401: https://docs.astral.sh/ruff/rules/any-type/
Okay, then your function which is extremely generic and needs to support 25 different use cases needs to have an insane type definition which covers all 25 use cases.
This isn't an indictment of the type system, this is an indictment of bad code. Don't write functions that support hundreds of input data types, most of which are unintended. Type systems help you avoid this, by the way.
6 replies →
Stupid is okay. _Nonsense_ is not. Your example was nonsense, it was absurd. The moment I saw the first example I was like, this should be add_ints and should only take ints.
Imagine I say "the human body is dumb! Here's an example: if I stab myself, it bleeds!" Like is that stupid or absurd?
1 reply →
But there was a conceivable way (maybe not in Python) to make a `slow_add` function very generic, yet only be defined over structures where any conceivable `+` operation is defined.
You just have to say the type implements Semigroup.
Yes, this would work if the arguments are lists, or integers, or strings. And it won't pass the typecheck for arguments that are not Semigroups.
It may not work with Python, but only because it's designers weren't initially interested in typechecking.
3 replies →
One thing that post does do though is very clearly highlight the difference between Python's type system and say ... TypeScript's.
TypeScript's goal is to take a language with an unhinged duck type system that allows people to do terrible things and then allow you to codify and lock in all of those behaviours exactly as they're used.
Mypy (and since it was written by GVM and codified in the stdlib by extension Python and all other typecheckers)'s goal is to take a language with an unhinged duck type system that allows people to do terrible things and then pretend that isn't the case and enforce strict academic rules and behaviours that don't particularly care about how real people write code and interact with libraries.
If you include type hints from the very beginning than you are forced to use the very limited subset of behaviours that mypy allow you to codify and everything will be "fine".
If you try to add type hints to a mature project, you will scream with frustration as you discover how many parts of the codebase literally cannot be represented in the extremely limited type system.
At this point I'm fairly convinced that the effort one would spend trying to typecheck a python program is better spent migrating away from python into a language that has a proper type system, then using interop so you can still have the bits/people that need python be in python.
Obviously that isn't always possible but you can spend far too long trying to make python work.
I think you're forgetting how easy type annotation is.
I occasionally spend like 2h working on some old python code. I will spend say 15 minutes of that time adding type annotations (sometimes requires some trivial refactoring). This has an enormous ROI, the cost is so low and the benefit is so immediate.
In these cases migrating code to a proper language and figuring out interop is not on my radar, it would be insane. So having the option to get some best-effort type safety is absolutely fantastic.
I can definitely see your point, it's a useful analysis for projects under heavy development. But if you have a big Python codebase that basically just works and only sees incremental changes, adding type annotations is a great strategy.
If you're supposedly good at software and you spent too long trying to make python work consider the possibility that you're not good at software?
Python has flaws and big ones at that, but there's a reason it's popular. Especially with tools like pydantic and fastapi and uv (and streamlit) you can do insane things in hours what would take weeks and months before. Not to mention how good AI is at generating code in these frameworks. I especially like typing using pydantic, any method is now able to dump and load data from files and dbs and you get extremely terse validated code. Modern IDEs also make quick work of extracting value even from partially typed code. I'd suggest you just open your mind up to imperfect things and give them a shot.
Six month into learning to build a modern python app, with linters, type systems, tests, venvs, package managers, etc… I realized that the supposed difficulty of rust is drastically less than coming to speed and then keeping up with the python “at scale” ecosystem.
My strong suspicion is that such a story depends a great deal upon the personalities of the developers much more than any {tool chaos + type chaos} --- {new syntax + lifespan annotations} spectrum
I don't understand this point at all. I've worked on Django codebases which have a huge set of typing problems... and while it's not 100% I get a lot of value out of type checking.
You annotate enough functions and you get a really good linter out of it!
Unfortunately with us being in the middle of the AI hype cycle, everyone and their dog is currently busy migrating to python.
I don't see why AI hype means more Python code.
State of the art AI models are all closed source and accessible through an API anyways. APIs that any language can easily access.
AAa for AI model development in itself, yes it's mostly Pyython, but niche.
4 replies →
I’d be surprised if _anyone_ is migrating server code to Python because of AI.
If you do that you need to compile, which means you can't just distribute a text file with your python program. You need a build infrastructure for every python version, every architecture and every OS.
Have fun with that!
> Python, AFAIK, has many features, a very permissive runtime, and perhaps (not unlike C++) only some limited subset should be used at any time to ensure that the code is manageable. Unfortunately, that subset is probably different depending on who you ask, and what you aim to do.
I'll get started on the subset of Python that I personally do not wish to use in my own codebase: meta classes, descriptors, callable objects using __call__, object.__new__(cls), names trigger the name mangling rules, self.__dict__. In my opinion, all of the above features involve too much magic and hinder code comprehension.
There's a time and a place for each of them:
* Meta classes: You're writing Pydantic or an ORM.
* Descriptors: You're writing Pydantic or an ORM.
* Callable objects: I've used these for things like making validators you initialize with their parameters in one place, then pass them around so other functions can call them. I'd probably just use closures if at all possible now.
* object.__new__: You're writing Pydantic or an ORM.
* Name mangling: I'm fine with using _foo and __bar where appropriate. Those are nice. Don't ever, ever try to de-mangle them or I'll throw a stick at you.
* self.__dict__: You're writing Pydantic or an ORM, although if you use this as shorthand for "doing things that need introspection", that's a useful skill and not deep wizardry.
Basically, you won't need those things 99.99% of the time. If you think you do, you probably don't. If you're absolutely certain you do, you might. It's still good and important to understand what they are, though. Even if you never write them yourself, at some point you're going to want to figure out why some dependency isn't working the way you expected, and you'll need to read and know what it's doing.
I never understood why pydantic reimplemented attrs, but doing it much slower, instead of just using attrs.
> Basically, you won't need those things 99.99% of the time
That's kind of my point. If you don't need a language feature 99.99% of the time perhaps it is better to cut it out from your language altogether. Well unless your language is striving to have the same reputation as C++. In Python's case here's a compromise: such features can only be used in a Python extension in C code, signifying their magic nature.
3 replies →
You should try Go!
It should be banned by the geneva convention.
Can you share a little bit about what makes you form opinions when you are not even using the language? I think its fascinating how especially discussions about typing makes people shake their fists against a language they don't even use - and like your post make up some contrived example.
>I think that the goal there is to understand that even with the best typing tools, you will have troubles, unless you start by establishing good practices.
Like - what makes you think that python developers doesn't understand stuff about Python, when they are actively using the language as opposed to you?
Indeed, I'm not a regular Python practitioner. I had to use it from time to time because it's the language chosen by the tools I happened to use at that time, like Blender, or Django. In the former case, it wasn't very enjoyable (which says a lot about my skills in that area, or rather lack thereof), while in the latter case I found it quite likeable. So that's my background as far as python goes.
I must admit that I largely prefer static typing, which is why I got interested in that article. It's true that trying to shoehorn this feature in the Python ecosystem is an uphill battle: there's a lot of good engineering skill spent on this.
Perhaps there's a connection to make between this situation and an old theorem about incompleteness?
https://copilot.microsoft.com/shares/2LpT2HFBa3m6jYxUhk9fW
(was generated in quick mode, so you might want to double check).
As someone who has been writing python for years the worst mistake I have ever seen people make is not add type hints and not using a type checker.
Also not creating custom, expressive Pydantic types and using nested dicts in places. Nested dicts suck, you never know what you're getting, and it's well worth the time converting them to classes.
TypedDicts or data classes are both a good idea.
To try to tl;dr that rather long post:
> When you add type hints to your library's arguments, you're going to be bitten by Hyrum's Law and you are not prepared to accurately type your full universe of users
That's understandable. But they're making breaking changes, and those are just breaking change pains - it's almost exactly the same if they had instead done this:
but anyone looking at that would say "well yeah, that's a breaking change, of course people are going to complain".
The only real difference here is that it's a developer-breaking change, not a runtime-breaking one, because Python does not enforce type hints at runtime. Existing code will run, but existing tools looking at the code will fail. That offers an easier workaround (just ignore it), but is otherwise just as interruptive to developers because the same code needs to change in the same ways.
---
In contrast! Libraries can very frequently add types to their return values and it's immediately useful to their users. You're restricting your output to only the values that you already output - essentially by definition, only incorrect code will fail when you do this.