Comment by duskdozer
3 days ago
I guess I am a casual pandas user only. Reading a guide on migrating/differences, it's hard to see why polars would be obviously better.
3 days ago
I guess I am a casual pandas user only. Reading a guide on migrating/differences, it's hard to see why polars would be obviously better.
Here are a couple reasons:
- much faster, multithreaded by default. Read in a big csv with it and see how it feels.
- no index/MultiIndex. Pandas special treatment of index always felt like more trouble than it was worth, so no need to reset_index() everywhere.
- expressions are very portable. At first using pl.col everywhere feels like a bit much, but you can define them anywhere and then apply them to a dataframe whenever you want.
- once internalized, the syntax makes much more sense and is far more consistent compared to pandas.
Of course all depends on what your use cases are. If performance is important then I'd strongly recommend trying it out. If you just use it to have a look at the odd dataframe, maybe not worth your time as much
End of the day, they both get the job done.
Pandas is more ergonomic in that some ideas can be more tersely represented. The downside is that this results in more dynamism which can change if the underlying data gets updated. Polars is more strict in that it will not silently flip a data type on you. However this strictness does come at the cost of being a bit slower to type and some data idioms not having a good Polars equivalent.
People like to note the speed improvements, but that is the least interesting thing about the library. Rarely have I ever had a problem where I was bottlenecked by Pandas throughout.
Polars is very much a Pandas 2.0 with a bunch of lessons learned. I do not think it is earth shattering changes, but it is worth migrating when you can.