Comment by pg
15 years ago
Occam's Razor suggests that the reason there is no dialect of Lisp as popular as less expressive languages is that no one happens to have created one yet. What languages are has changed. Languages used to be specs. Now they're open source projects. Open source projects seem to succeed based more on the energy of their initial organizers than anything else. There are only a few big language-project organizers, and none of them happens to have chosen to implement a language that is a dialect of Lisp. That's the sort of thing that's true until it's false. E.g. Clojure could be the counterexample.
Maybe there's more to it than a small sample size, but that seems the likeliest explanation. The second most likely explanation is almost as mundane: that the reason new dialects of Lisp have trouble attracting adherents is that mainstream programmers are put off by s-expressions.
I agree with Peter Van Roy on this - mainstream programmers tend to have an unclear picture of the relationship between syntax and semantics. This goes for both C-style and whitespace based mainstream languages. Every syntax brings along hard-wired semantics like a ball and chain. But the ball is not particularly heavy, in fact sometimes it's light enough to not be much of a bother. This actually explains quite a few things - people tend to be satisfied with whatever language they've become comfortable with (in fact they get angry if somebody actually points out the ball and chain). The few people that actually take the time to learn several dramatically different languages discover that ball and chains come in all different kinds of shapes and sizes.
But I believe there is a sea change going on and Lisp will only benefit from it. That sea change is Functional Programming. Functional Programming is very much concerned with optimal semantics. The more people understand that semantics and syntax are inextricably intertwined the sooner they'll see that Lisp has been and continues to be one of the best ways to rapidly prototype and build optimal semantics.
But this also means Lispers need to constantly look at other languages - the optimal semantics are constantly evolving and changing (Racket and Clojure are doing a good job of adopting better semantics).
On that note, I'd like to say that you've done an incredible job at pointing out that the ball and chain is there to far more mainstream programmers than anyone else I know. Thanks for that.
What languages are has changed.
In fact, when it comes to choosing languages, I think things have changed even more. At justin.tv, we've chosen nearly all of the languages we use because of a particularly good library that each one has (Python, because of Twisted; Ruby because of Rails, etc). The actual features of the languages have been almost irrelevant. We would definitely jump on a Lisp dialect if it happened to have a great library we wanted to use.
I think this is particularly insightful. What you've stated is a raw and pragmatic consideration - what is it that I want to do and what library lets me get that done the fastest?
Coming from justin.tv, this insight may be more start-up perspective oriented than historical reasons for programming language choice.
But I also believe that programming language choice is driven by market or behavioral forces that don't necessarily act in a rational way or lend itself to overly rational analysis.
Another thing that's changed is who is creating the dominant programming languages.
In the programming-language-as-a-spec days, people versed in programming language design created the dominant programming languages. In the programming-language-as-an-open-source-project world, languages tend to be created by people that are relatively clueless about language design at the start (this holds for at least Perl, Python and Ruby). I don't know what the root of such is, but there seems to be some negative correlation between the design integrity of a given language and its popularity (for web hackers at least – Java and C# adhere to the old model).
In contrast, Lisp seems to be fairly popular among folks that are programming languages enthusiasts. So for whatever reason, there's a gap between the folks that are creating new popular languages, and the sort of people that tend to be Lisp fans.
The other bit is that, well, Lisp looks funny. This is something like someone looking at a language that uses Greek or Cyrillic or Arabic alphabets and assuming that they're by default harder languages because they use different characters. In fact, a reasonably clever person can learn a new alphabet in a day. It's a totally artificial barrier for someone who's serious about learning the language since the investment to learn the new alphabet is dwarfed by the work of actually learning the language, but it tends to make it scarier if upon casual inspection. I suspect Lisp pushes some people away via a similar artificial barrier.
My understanding is that prior to creating Perl, Larry Wall had studied (natural language) linguistics, with an aim towards perhaps designing a writing system for some natural language.
I believe he's explicitly rejected the ideas (common in the Lisp world) that extreme conciseness and conceptual orthogonality should be the ultimate goals of programming language design.
Calling him "relatively clueless about language design" seems a little odd.
I think the inverse correlation you've found is easily explained. There are folks who design languages from the perspective of academia and there are folks who design languages from the perspective of practicality, and there is not much overlap in those communities. Perl and PHP are imminently practical languages, but they never would have been put forth by academia, because they are not pure, they aren't interesting in a research sense.
It's difficult to say that with complete conviction. Racket, for example, aims to be very practical. However, most of the research in PL isn't necessarily on entire new language systems as a whole, but maybe a specific feature or category.
Disclaimer: I guess by association I'm affiliated with this group[1].
[1] http://www.racket-lang.org/people.html
Matz was a researcher working on compilers and programming languages when he created Ruby, so it doesn't seem obvious to me that he was "relatively clueless about language design" at the start.
He was definitely clueless about the implementation of Ruby. MRI started as a half-assed Scheme implementation, which is easy to see when you read the MRI source code. Scheme has lexical scoping, while Ruby has...python/perl-style scoping.
Scheme has first class anonymous functions, while Ruby has blocks, which are a hack around MRI's slow function calls.
2 replies →
>In the programming-language-as-a-spec days, people versed in programming language design created the dominant programming languages.
Nope.
The creators of C were witness or party to some earlier languages, but they weren't well versed by any means.
Prior to Python, Guido had worked on ABC - he was an experienced language designer by that point.
Alternatively, free market theory suggests that the reason there is no dialect of Lisp as popular as less expressive languages is that expressiveness is not a trait that increases fitness, or even might decrease it.
(for the record, I like Lisp)
That is true to a degree. I agree with what abstractbill says, that libraries are the dominant factor in most people's language choices (after familiarity of course).
Kawa Scheme makes it easy to call all the Java libraries, yet that hasn't made it huge yet.
1 reply →
When you look at lisp code, all you have to go by are the names of functions. The code could do anything, and you won't find out what until you dig. By contrast, other languages have hints like types and standard, well-understood libraries and restrictions on what might happen.
This suggests lisp code is less approachable for second authors, precisely because of its expressiveness.
I agree. Even Stuart Halloway said they use Rails and JRuby for a lot of the projects they do at relevance. He said the reason was because of the community and the lack of equivalently good Clojure web frameworks. Its hard to think of someone who would be more sold on Clojure than him. And yet, he still thinks it makes sense to use Ruby instead of Clojure.
Its very hard to argue against it. If it takes you less time to write a Rails app than to write a Compojure app, which one would you choose? It seems like Rails is the correct answer, unfortunately.
I'm interested to know why you ended that last statement with "unfortunately."
I'm a web developer who currently uses Ruby for my projects, and I've been looking at Clojure. I've looked at Ring, Compojure, and other things, and it all feels like "Hey, Clojure can also do web! Isn't that cool?
I don't think it's unfortunate at all that people use Rails for making web sites. It's what it's for. Want to make a multithreaded concurrent backend queue? Probably not a good fit for Ruby, but sounds to me like a great thing for Clojure.
But I'm interested in your "unfortunately" statement because wonder what I'm missing. What would using Clojure give me, as a web developer?
I meant that unfortunately Clojure will not be gaining very much popularity in web development.
I agree that Rails is a much better web framework than Compojure/Ring. If you are doing web development, it makes sense to use Rails.
You can use Compojure or Enlive for straight web development if you want. But, at this point it is inferior to rails.
So I guess what I meant is, I would like Clojure to be more popular. But, it doesn't look like it will be getting any major attention in web development any time soon.
6 replies →
> Open source projects seem to succeed based more on the energy of their initial organizers
But whomever starts a Lisp language is going to have the usual Lisp suspects looking over his/her shoulder. That's going to dampen most open source organizers' energies, I guess.
So making Lisp more popular would be a matter of getting some hackers on the ground floor of a dialect who are excited about their project. I can't think of a counter-example among the popular languages. If you're right, then it blows away my thesis, but I can't think of a way to test this (aside from watching a Lisp dialect get as popular as Java).