Comment by perrygeo
3 days ago
I agree. But the data science/engineering space is enamored with Python. Makes good sense when doing interactive work. Makes no sense in production. But by that point, most developers consider it a sunk cost and just keep their Python apps limping along rather than rewriting.
That's why polars is a great option. Start prototyping in python, then a relatively easy port to a Rust app when the need hits.
You can still have extensive "compile-time" checking with linters and static type checkers. This makes Polars even better in typed Python, as it type-checks much better than the dynamic mess that is Pandas.
Polars is much better than pandas in this regard. But using it via Python still carries the risk of runtime errors.
Python's linters leave many holes unfilled - it catches some stuff but still leaves everything else to the runtime. Literally every Python application that I've worked on in the last 3 years a) is 100% type checked yet b) still hits massive numbers of novel runtime errors on the regular. Python's anemic type system is in no way a substitute for an advanced compiler.
When I deal with datascience, I rarely have type problems in my programs that python typecheckers can't catch. I do have type and dirty data problems of the dataframes/csvs/parquet I'm reading.
1 reply →
If it’s 100% type checked and you’re still hitting massive numbers of type related runtime errors, it probably isn’t 100% checked.
1 reply →