Comment by nutate

11 years ago

F# is a real breath of fresh air in comparison to something like Scala. It's direct ML heritage really shows, also just diving in with an IDE (like Xamarin, or I suppose Visual Studio) is super easy.

I see it as the future of pop-functional programming. For example look at the way it handles type inference w/ JSON parsing. Compare that to what you have to do to parse JSON in Scala. It's subtle, but a major usability win.

44 comments

nutate

Reply
narrator  11 years ago

Scala is getting really interesting lately though with projects like Spark and SparkSQL. Spark takes a Scala AST tree and computes a plan on it in order to distribute it to a cluster of possibly thousands of nodes and then uses transformations on that plan, similar to how an SQL query optimizer works, to make the plan more efficient. The cluster compute benchmarks on it are crazy good.

The only problem I have with Scala is that it has so many features that it tends to have a steeper learning curve than most languages. However, this is mitigated somewhat in that it lets one switch between object and functional depending on what's more convenient or performant, thus making it easier for programmers who are not highly skilled in functional programming to pick up.

  • nutate  11 years ago

    Yeah, I did a presentation at the DC Area Apache Spark meetup... slides: http://www.slideshare.net/RichardSeymour3/2015-0224-washingt... associated blog post: https://www.endgame.com/blog/streaming-data-processing-pyspa...

    I've done a bit of scala spark as well, and my initial thought was prototype in pyspark and then rewrite in scala if necessary. Just this week DataBricks announced they are working on changing some of the data structures behind RDDs to save on unnecessary java object creation hubbub https://databricks.com/blog/2015/04/28/project-tungsten-brin...

    That and the SQL compiler thing seems pretty darn awesome. Spark has the nice benefit of being plug and play (w/ a joyful time of compiling and deploying) with legacy HDFS/Hadoop systems. That alone will keep it in toolboxes for a long time to come.

  • iskander  11 years ago

    >Spark takes a Scala AST tree and computes a plan on it in order to distribute it to a cluster of possibly thousands of nodes

    Unless Spark has changed dramatically in the year since I used it, that's not really how it works. You lazily construct an expression graph of RDD operators but the actual Scala code (the functional payload of e.g. flatMap) doesn't get analyzed. Are you talking specifically about Spark SQL's code generator?

    >The cluster compute benchmarks on it are crazy good.

    ...and also carefully chosen to show off the niche cases where Spark shines. More commonly, you'll encounter crippling garbage collector stalls leading to mysterious and undebuggable job failures.

    Spark has a very clean API but the implementation (at least for Spark versions 0.8-1.0) is still prototype quality.

    • devonkim  11 years ago

      So basically you're talking about the design and limitations of https://databricks.com/blog/2015/04/13/deep-dive-into-spark-... I take it?

      The implication of this work I thought is that it could be further expanded to other languages and DSLs. However, Spark's SQL generator still being very JVM-dependent and this optimizer generating bytecode kind of makes it pretty specific to JVM-supporting languages only. This would probably leave out Haskell and Erlang / Elixir in the short term, which is probably where I'd expect to see a different perspective on the whole data analytics front. We have Datomic, sure, (and I guess Clojure is enough for a lot of folks) but it'd be nice to have something other than "we want to make Hadoop... BUT BETTER" as a lot of the motivations.

cannam  11 years ago

I did a toy comparison of ML-derived languages running on Linux the other day (http://thebreakfastpost.com/2015/04/22/four-mls-and-a-python...) and found F# already simple to get along with and fast to run, at this beginner level. It could become a really interesting option.

typish  11 years ago

Also interesting is their F* language: http://fstar-lang.org/

"F* is a new higher order, effectful programming language (like ML) designed with program verification in mind. Its type system is based on a core that resembles System Fω (hence the name), but is extended with dependent types, refined monadic effects, refinement types, and higher kinds. Together, these features allow expressing precise and compact specifications for programs, including functional correctness properties. The F* type-checker aims to prove that programs meet their specifications using an automated theorem prover (usually Z3) behind the scenes to discharge proof obligations. Programs written in F* can be translated to OCaml, F#, or JavaScript for execution. "

It's open source too: https://github.com/FStarLang/FStar

vezzy-fnord  11 years ago

You would think something like OCaml would fill in that niche with its wider availability.

  • cannam  11 years ago

    They do seem like very similar languages. I imagine in most cases your choice would have effectively already been made for you by the context -- e.g. whether you wanted the option of fast native binaries (OCaml) or deep .NET integration (F#).

  • seanmcdirmid  11 years ago

    OCaml doesn't come with the ecosystem that F# does. It isn't about availability, but APIs to play with.

    • s_kilk  11 years ago

      I'm not sure I buy the ecosystem argument. We've recently seem other languages come from out of nowhere, with no ecosystem, to become major languages (Golang, for example).

      I don't think there's really anything stopping OCaml from being more widely adopted, except for lack of attention from trend-setters.

      4 replies →

d4rkph1b3r  11 years ago

I don't agree, it's 'direct ML heritage' is quite superficial, as it omits ML's most powerful feature, functor modules.

Scala however has all kinds of super powered functional programming features that F# is missing, implicit parameters for type class constraints, ADTs, higher kinded types, etc.

ericmo  11 years ago

I know it doesn't say much, but having never coded in ML and having tried F# briefly (I code mostly C#, and it'd be easy to have mixed F#/C#), I think Scala has much clearer syntax, or perhaps it's just more familiar for someone who's coming from imperative languages like C++ or C#.

justthistime_  11 years ago 

  case class Fruit(name: String, variants: Set[String])

  val fruitJson =
    Json.parse(
        """{
             "name": "apple",
             "variants": ["cox", "braeburn"]
           }""")

  val fruit = fruitJson.as[Fruit]

Doesn't seem to be too painful to me.

I agree that F# is syntactically closer to ML (though Scala is closer to ML with its module system which F# gave up completely).

  • bunderbunder  11 years ago

    In this base case, no, it isn't too painful. But once you get to larger and more complicated structures, F#'s type provider system can be a tangible timesaver. Especially if you're doing exploratory programming.

    TBH, I think the JSON example might underrate things a bit anyway. Where type providers really start to feel impressive is when you get to tricks like being able to noodle around inside undocumented (or poorly documented) COM interfaces with the help of CodeSense, or getting red squigglies in your editor (and compile time errors) when there's a problem with an SQL query.

    F# isn't all roses, of course. I really wish it had typeclasses, and discriminated unions are less powerful than Scala's case classes in some tangible ways. But from a purely pragmatic perspective, my (almost certainly biased) sense is that I can usually Get Things Done™ with less ceremony in F#, and type providers are a shining example of that pragmatism.

    • justthistime_  11 years ago

      Well, then I think JSON is a petty bad example, because pretty much no service provides some schema-like specification of their protocol in JSON.

      If there is one thing I learned using type providers it is that maintaining your own "description"–be it as sample data or a schema file–loses most benefits of using type providers in the first case.

      On the other hand, want a type provider for X in Scala? Just write it.

      Stuff is still moving around because Scala devs want to get the meta APIs right first, but if adapting things from time to time is not an issue, then go ahead.

  • nutate  11 years ago

    Tell me what happens with that technique when your json has > 22 keys. ;)

    (Edit) this is apparently working in the latest scala 2.11 But it killed my adoption of scala for real work early on. https://issues.scala-lang.org/browse/SI-7099

    • justthistime_  11 years ago

      It works without issues.

      I never have seen a use-case where this would have been an issue. Sure, it "looks" unfortunate, but there is only so much you can do when running on a completely uncooperative runtime.

      If devs need a typed representation of unlimited length, they provide a HList and move on.

      Such a representation will replace the length-limited tuples in a future version of Scala.