Comment by throwawaylsp
14 days ago
There's one thing I've never understood. Lisp is 65 years old. It's older than any mainstream programming language apart from FORTRAN. It has a bevy of vocal fans in every generation. So...why hasn't it gone mainstream? Or at least, why has it failed to remain there?
Lisp became incredibly popular at one point. It spawned so much productivity that it created huge programs that used lots of memory and demanded expensive hardware.
This peaked at a time when microcomputers had not reached the right affordability and power parameters.
People who were developing in Lisp turned their eyes to the microcomputer market and the business to be had there, if the stuff would only run. So there was some activity of rewriting Lisp stuff in languages like Bliss and C.
The transition from powerful workstations (where we can count Lisp machines) to microcomputers basically destroyed everything which couldn't make the jump nimbly.
The new crop of programmers who cut their teeth on micros simply had no knowledge or experience with anything that didn't run on micros.
Poof, just like that, a chunk of the computing sphere consisting of new people suddenly had amnesia about Cobol, Fortran, Snobol, PL/I, operating systems like TOPS/20 and VMS and whatnot.
Only Unix pulled through, pretty much --- and that's because Unix started on relatively weak hardware, and was kept small. Unix started getting complicated approximately in step with micro hardware getting more complicated and powerful. E.g. a Unix kernel was around 50 kilobytes in 1980. not a good fit for some Apple II or Commodore Pet, but not far off from the resources the IBM PC would have.
By the time micros were powerful enough to the huge Lisp stuff with 20 megabyte images, we were into the 90s, and starting to be overrun with crap dynamic languages.
Now Lisp people could have buckled down and worked on promoting excellent Lisps for microcomputers. There were a few fledgling efforts like that that were not promoted well.
It seems that what Lisp programmers there were, they were mostly wrapped up working on bigger problems on larger hardware, and ignored microcomputers.
It's very hard to promote anything today that most of your Generation X (now management class) didn't get to play with in the 80s and 90s.
Because Lispers sneered at the microprocessors of the time while everybody else piggybacked on them. And, by the time that attitude changed in the early 90s, there were already socially entrenched languages that had the "superpower" of garbage collection (Tcl for ECAD, Perl for sysadmin).
Nobody in the Lisp world ever took the time to implement stuff that people wanted on those tiny machines. Or to demonstrate to people the cool stuff it could do.
You can see this in Dr. Dobbs Journal. People are doing things like drawing graphics, writing spell checkers and controlling modems. Assembly and BASIC are normal but C and Forth are mentioned regularly. Turbo Pascal pops up in 1984. Some of the names are famous enough that you recognize them even now, decades later.
Lisp just ... gets barely mentioned in passing sometimes. And nobody of note writes anything about it. Somebody could have built a word processor, a spell checker, a chess game, a reversi game, ANYTHING ... but nobody did.
For the record, Tcl doesn't use garbage collection, rather objects are reference-counted and freed when ref count is <= 0.
But interestingly, Tcl is a good example of languages that took a piece of Lisp's domain. In fact, Tcl is quite Lisp-like with a syntax distinct from Lisp. Tcl has been successful and with the recent release of v. 9.0, it stands to gain traction among programmers.
OTOH CL and Scheme remain underused though current implementations are generally well-equipped to handle contemporary requirements. I've used Scheme to build website generators and other tools but there's a dearth of large-scale, visible Lisp/Scheme projects out there to attract developers.
Starting such a project is a big commitment, probably programmers are hoping somebody else will pick up the ball and run with it.
Ref counting is garbage collection (cf https://courses.cs.washington.edu/courses/cse590p/05au/p50-b...). It's just not tracing garbage collection.
I'm a decent fan of both Tcl and CL, but Tcl has the big problem of being "almost" homoiconic and lacking good meta-programming tools like quasi-quoting. I say almost because comments break homoiconicity, whereas in CL they are discarded at read-time, never appearing in the parsed tree.
2 replies →
Related question: Why is welding pretty mainstream while blacksmithing is a much more niche craft? Blacksmithing is a more overarching skill: After all every blacksmith knows how to weld but relatively few welders can forge effectively.
Possible answers:
1. Blacksmiths enjoy making custom tools for each domain while welders just want to get on with solving their domain problem.
2. Blacksmithing is harder to learn. Welding using modern techniques is easy to learn. (Caveat: Welding well is quite difficult. But learning to weld good enough to repair a broken hitch on your tractor is easy.)
3. Welding can solve a very large chunk of metalwork problems. Not all of them--and not always with elegance--but it gets the job done quickly. Blacksmithing can solve a larger set of metalwork problems with more elegance but it also takes more time and skill.
you can very reasonably do welding in your garage. aside from a welder (as little as $150 for a barely-usable mig), all you need is an angle grinder to cut and finish the welds. commercially you can get a mid-range mig and a couple more smallish tools and you can start selling custom fencework and mounting brackets and such.
blacksmithing you need a forge, which immediately takes up more space and is somewhat more likely to start a fire. an anvil, and tongs, and hammers. its also a lot more physically demanding, even if you use a power hammer.
your #2 and #3 are pretty key. most importantly most fabrication jobs are much happier to get quick work with reasonable precision using stock shapes. once you start talking about real free-form hot shaping you're immediately going up at least 10x in price/time. welded table base - $500. handcrafted wrought table base - $10,000.
really its that metalwork is mostly functional (fences, stairs, railings, walkways, enclosures, stainless for commercial kitchens, pipefitting, etc). its very difficult to stay in business as a actual craftsman making well-designed objects. architectural metal is probably the easiest in (wall coverings, nice looking railing and stairs, lamps, and other decorative elements). and there its still dominated by fabrication processes (machining and welding of stock shapes), although nicer materials like bronze start to have their place.
edit: you know I left this thinking I was missing something and I realized what it is. welding you make shapes out of like-shapes. like making drawings in figma. I don't think a lot of people have what it takes to learn to be a really good freehand artist. and even if you have the skill, being able to design those kind of organic arbitrary shapes so that they are emotive and attractive is another step up. do you want a piece of art which is a direct expression of the concept held by the artist? or do you want a 3x5' 32" inch high workbench for 1/20 the cost?
Also, if you live in the city (including suburbs), your neighbors are likely to get pretty annoyed by the sound of hammering metal. Welding makes noise too, but a lot less of it. That's the main reason why I haven't gotten into blacksmithing even though I think I would really enjoy it. I just don't think it'd go over well with the neighbors.
1 reply →
> you can very reasonably do welding in your garage
Let me just clarify one thing: you can reasonably do _arc_ welding in your garage, not torch welding. Source: my house burned down once due to the guy next door torch welding in his garage.
Because with welding, you can build useful frame structures out of tubes and rods.
Would you ride a bike frame forged by a blacksmith? Haha.
I don't understand your point. Do you think blacksmiths cannot forge tubes? Who do you think made the first guns? Or bicycles for that matter?
A bike frame forged by a blacksmith would be incredibly strong but it would also be an enormous amount of work.
Disclaimer: this mostly happened before, or at best shortly after, I was born, so this isn't drawn from personal recollection but rather attempting to synthesize from others' recollections, often from people who have some bias.
One of the major trends in computing in the 80's and 90's is that high-end systems lost out to the growth in capabilities of low-end systems, and this happens in pretty much every level in the computing stack. Several people responded to this trend by writing articles sniffling that their high-end systems lost to mass market garbage, often by focusing on the garbage of the mass market garbage and conveniently avoiding analysis as to why the high-end systems failed to be competitive in the mass market. The wonders of Lisp is one of the major topics of this genre.
Most famously, Lisp was tarred by its association with AI during the concomitant collapse of AI that led to the AI Winter, though it's less often explored why AI failed. In short, it didn't work. But more than just AI at the time, people also felt that the future of programming in general was based around the concept of something like rules-based systems: you have a set of rules that correspond to all of the necessary business logic, and a framework of program logic that's making those rules actually take effect--you can see how a language like Lisp works very well in such a world. But programming doesn't have a clean separation between business logic and program logic in practice, and attempts to make that separation cleaner have largely failed.
So Lisp has a strong competitive advantage in a feature that hasn't proven to actually be compelling (separating business from program logic). Outside of that feature, most of its other features are rather less unique and have seeped into most mainstream programming languages. Functional paradigms, REPLs, smart debuggers, garbage collection--these are all pretty widespread nowadays. Where Lisp had good ideas, they've been extensively borrowed. Where those ideas haven't pulled their weight... they've languished, and most of the people wistfully wishing for a return to Lisp haven't acknowledged that the limitations of these features.
It very nearly did. Then the AI Winter happened.
https://en.wikipedia.org/wiki/AI_winter
The language was married to and sold with a hardware architecture that didn’t achieve massive commercial success compared to the other workstations at the time and later microcomputers.
And it pretty much had to be married to that because off-the-shelf microcomputers weren't powerful enough for it.
The Lisp Curse: http://winestockwebdesign.com/Essays/Lisp_Curse.html
That's just a mind projection piece written by someone with next to no Lisp experience, let alone in a setting with multiple developers.
Exactly. A post full of bollocks.
"Lisp is so powerful that problems which are technical issues in other programming languages are social issues in Lisp."
So true. Lisp was designed to give individual programmers tremendous power. That means Lisp programmers sometimes prefer to reinvent solutions to problems rather than learn to use some existing solution. This tendency can be an absolute nightmare on a software engineering team.
Not that using Lisp on a software engineering team cannot be done, but it requires very strong discipline and leadership. The absence of strong discipline and leadership on a Lisp SWE team can lead to enormous amounts of wheel reinvention and technical debt.
Obviously discipline and leadership are necessary for any SWE team but languages like C don't encourage reinvention nearly as much as Lisp does, and Lisp programmers in general tend to be very resistant to the imposed discipline that SWE requires. (I say this as a diehard Lisp programmer, so I'm talking about myself.)
> So true. Lisp was designed to give individual programmers tremendous power. That means Lisp programmers sometimes prefer to reinvent solutions to problems rather than learn to use some existing solution. This tendency can be an absolute nightmare on a software engineering team.
I've found that there's a world of difference between my tendency to wheel-reinvent when I'm messing around on my own vs. my tendency in an industrial setting. When I'm messing around on my own, Lisp gives me so much more reach it's incredible, and yeah, I kinda do want to reinvent the application server, or the TCP/IP stack, or something sometimes.
But when I'm getting paid and there are milestones and deadlines? Fuck it, I'll just use what's available to build what's needed. The difference is that in a Lisp codebase, some really smart people have come before and built some really cool abstractions. Like a test framework that makes automated testing so much simpler it's ridiculous. Like, two lines of code and you have a test for a new feature. You get access to tools and techniques that let you close the gap between "ticket lands in your lap" and "done" much faster than you would in Java.
1 reply →
But where is it now? If not mainstream, where? Is it not used at all, or only by hobbyists, or also by successful companies, today? If it isn't mainstream, is it important, if not, what's the cursor?
elements to not judge in the void https://github.com/azzamsa/awesome-lisp-companies/ (some are hiring) (that's just the companies we know, nothing official)
https://github.com/CodyReichert/awesome-cl/
Programming language adoption is more than just about syntax and semantics; there are other factors. For example, JavaScript is often criticized for its design, yet this hasn’t stopped tens of millions of developers from learning the language, since if you want to do client-side Web programming (the most widely deployed platform in the world), you need to use JavaScript, period. It also helps if a language has/had a major corporate backer at a crucial time in its life. Java has Sun/Oracle, C# has Microsoft, Go has Google, and C and C++ had AT&T (Bell Labs).
Lisp’s most successful commercial period was during the 1980s during an AI boom. Companies such as Symbolics, Texas Instruments, and Xerox sold workstations known as Lisp machines that were architecturally designed for running Lisp programs. They had corporate and institutional customers who were interested in AI applications developed under Lisp, including the United States government. Lisp was also standardized during this time period (Common Lisp). Lisp even caught the attention of Apple; Apple had some interesting Lisp and Lisp-related projects during its “interregnum” period when Steve Jobs was absent, most notably Macintosh Common Lisp, the original Newton OS (before C++ advocates won approval from CEO John Sculley), Dylan, and SK8.
However, the AI Winter of the late 1980s and early 1990s, combined with advances in the Unix workstation market where cheaper Sun and DEC machines were outperforming expensive Lisp machines at Lisp programs, severely hurt Lisp in the marketplace. AI would boom again in the 2010s, but this current AI boom is based not on the symbolic AI that Lisp excelled at, but on machine learning, which relies on numerical computing libraries that have C, C++, and even Fortran implementations and Python wrappers. Apple in the 1990s could have been a leading advocate of Lisp for desktop computing, but Apple was an unfocused beacon of creativity; many interesting projects, but no solid execution for replacing the classic Mac OS with an OS that could fully meet the demands for 1990s and 2000s computing. It took Apple to purchase NeXT to make this happen, and under Steve Jobs’ leadership Apple was a focused beacon of creativity with sharp execution. Of course, we ended up with Smalltalk-inspired Objective-C, not Common Lisp or Dylan, as Apple’s official language before Swift was released after the end of Jobs’ second reign.
Some other factors: 1. Lisp was truly unique in the 60s, 70s, and 80s, but it required expensive hardware to run. It would be hard to conceive of a Lisp running well on a 6502 or an 8086. Something like my NeXT Cube with a 68040 would do a much better job, but those machines cost roughly $6500 in 1989 dollars, out of reach for many developers.
2. By the time hardware capable of running Lisp acceptably became affordable, other languages started offering certain features that used to be unique to Lisp. Wanted garbage collection? In 1995 Java became available. Want object-oriented programming? You didn’t even have to wait until 1995 for that due to C++. Want anonymous functions and map()? Python’s popularity took off in the 2000s. Yes, Lisp still offers features that are not easily found in other languages (such as extensive metaprogramming), but the gap between Lisp and competing popular languages has been narrowing with each successive decade.
> Lisp was truly unique in the 60s, 70s, and 80s, but it required expensive hardware to run. It would be hard to conceive of a Lisp running well on a 6502 or an 8086.
You'd be surprised. https://retrocomputing.stackexchange.com/questions/11192/wha... Of course something like FORTH was perhaps more suited to these smaller machines, but LISP implementations were around. Many users of 6502-based microcomputers were familiar with LOGO, which is just a LISP with different syntax.
Thanks, that was very interesting and informative!