Comment by mugul
2 days ago
Looks like a pretty cool community building great tools with care. Using a functional language for data transforms sounds like a sane idea, haven't played around with it yet but it's definitely on my list now.
However they claim using Haskell for data science is "fast", which doesn't really mean anything until you have numbers to show. A little benchmark with pandas and polars wouldn't hurt I guess.
This is underway: https://github.com/duckdblabs/db-benchmark/pull/180
FWIW, https://duckdblabs.github.io/db-benchmark/ linked to from the readme, every result for "Haskell" shows OOM or undefined error. (Even on the large instance, even with the small input.
Cool cool cool!
> Using a functional language for data transforms sounds like a sane idea, haven't played around with it yet but it's definitely on my list now.
It's been one of the plays that the F# community has also been trying to make (somewhat with a modicum of help from Microsoft's marketing arm, but not enough help from what I've seen) pitching F# as a language close enough to Python to feel familiar and useful to data science but with the performance help of rich ML types and the modern .NET performance ecosystem.
(To my experience: getting "fast" compared to Python seems easy for most functional languages. Getting data science out of Python seems hard for a lot of sociology reasons more than technical ones.)
Haskell tends to be C-fast.
That's not what I see reported, they say Haskell tends to have bad memory layout generally and takes a 5x or so hit to performance.
Depends what you're doing, but it really can have C-comparable speed. [0]
Unoptimised/naive Haskell might be that slow. But that's true of a lot of languages and isn't particularly interesting to me. Java is slow if you do everything the naive way, too.
[0] https://entropicthoughts.com/on-competing-with-c-using-haske...
2 replies →
The phrase “tends to” is doing a lot of work here. Inexperienced Haskellers tend to use the singly linked list for everything and that data structure just doesn’t perform in any language.
In many cases it's possible to write Haskell that runs as fast as a given C implementation. However the result will usually not be idiomatic Haskell.