Steel Bank Common Lisp version 2.6.7

6 hours ago (sbcl.org)

Fun fact I learned from a Func Prog podcast: the name Steel Bank is a play on it's origin as Carnegie-Mellon Common Lisp (Carnegie made his fortune in Steel, while Mellon did so with Banks):

https://www.sbcl.org/history.html

  • There's another wordplay in the name.

    Namely, that "SB" stands for "Sanely Bootstrappable" - something that the CMU Common Lisp was unable to do

    • In my first job, grumpy Martin had as one of his many indispensable tasks, to build cmucl. (Itasoftware, one of the main lisp shops of the age)

    * the SB-SIMD contrib now supports ARM64. (Thanks to Sylvia Harrington)
    * AVX512 instructions are now supported on X86-64. (Thanks to Robert Smith and Arthur Miller)
    * additional support for SIMD instructions on ARM64 and X86-64. (Thanks to Arthur Miller)

These seem like pretty awesome additions. Does anyone know how SIMD works in SBCL? Is this at the codegen layer? i.e. can it auto-vectorize or anything like that? Or are these intrinsics you have to explicitly ask for?

I wonder sometimes how the world would look like if lisp won and the unit of deployment was a lisp machine image, if that makes sense. How would a kubernetes optimized for lisp look like? How would AWS EC2 look like? etc.

  • We wouldn't have need such barbaric things in a lisp world. :p

    Personally I think/hope they would have earlier discoveries / more broad usage of the deterministic systems like nix etc, which are built upon functional principles of immutability etc.

    Maybe the world would be guix/Hurd!

    • Common Lisp is pretty much the opposite of what people think of as functional nowadays. You're constantly changing the image and everything is mutable.

      5 replies →

    • > Personally I think/hope they would have earlier discoveries / more broad usage of the deterministic systems like nix etc, which are built upon functional principles of immutability etc.

      Yeah, that, uh, doesn't sound like Lisp

  • That (deployment is via a customized image) is the thing I find most awkward.

    Somewhere, I have a copy of a commercial LISP for Windows which would compile to an executable --- apparently this sort of thing is still possible, but it's not widely known/used, and sadly Jean-Marie Hullot's "SOS Interface" for the Mac was co-opted to NeXTstep:

    https://denninginstitute.com/itcore/userinterface/GUIHistory...

    I'd dearly love to see a RAD (Rapid Application Development) tool using LISP w/ a nifty UI for working up a GUI which would compile to something easily deployed (maybe HTML5 Canvas and JavaScript) as a stand-alone/single-file web application?

    • sbcl can compile to a single executable (the image is embedded in the binary). From a deployment standpoint, the only real issue is if you depend on a dynamically linked C library like OpenSSL.

      2 replies →

    • Docker deploys is a bit like that I think. Overall, I think many languages and programming systems pick up pieces from each other nowadays. PHP almost by accident can be quite stateless. Yet it's looking more like Java. Many things which could be deployed as an executable, are deployed as VM images. I'm sure we can come up with more things...

  • > How would a kubernetes

    We have a few services built in Clojure and we expose nrepl port on our pods in our SDEs, it's enormously helpful to test and debug things on the fly, without having to redeploy or restart anything. Without having to deal with state, caching, etc.

  • I've often thought that s-expression diff-viewers would be very nice tooling. I've also often wondered how easy it would be to just sling s-expressions across the wire to execute in another environment. What would modern infrastructure look like if you could just have lambdas running lambdas?

    • Not sexps, but better.

      Quite a while back I have read about a system based on Scheme , called termite (I don't remember which scheme that was) that can actually sling live running code/closures across network and execute them remotely..

      Boggles my mind even today.

      1 reply →

    • in at least Common Lisp world slinging s-expressions is considered to be a hack, at best something you do in a development environment, like swank/slime wire protocol. one of the reasons is that a readtable is both powerful, user extendable, and has all kinds of default ways in which a malicious input would be detrimental. you can remove all kinds of reader macros like #.(xyzzy), but to make a truly bulletproof s-expression reader you'd have to build a json like subset from first principles. even things like colons in symbol name package:symbol will trip you up. it's been understood since long time ago, that common internet standards, like RFCs, are the preferred method. in which case the fact that it's an s-expression versus a json is pretty much irrelevant. it's a kind of reader/writer anyway.

  • Not that different.

    AWS already has lots of AMI and docker are essentially image based.

    We would version whole images in something like git, and that's all.

  • Lisp's been around for 70 years at this point and hordes of developers tried it and said: lovely, but I ain't using it at work.

    I'm strongly convinced, having used Scheme, CL, Racket, Clojure that Lisp is doomed to be (mostly) a hobby language.

    The very power of Lisp, macros, dynamic programming, reflection lead to a mess of an ecosystem where every single developer reinvents the wheel and nobody can understand yet another DSL invented by the next developer next to them.

    Racket's theme of being a "programming language for building programming languages" is just the poster child of this naivety: I don't want even more friction.

    What scales and works is simple and boring.

    That's by the way why also Haskell has struggled forever. Beyond its dreadful DX, poor tooling and unacceptable compilation times, the language is just plagued by compiler extensions and every single developer reinventing its own abstractions.

    There was the simple haskell movement to just standardize around a set of extensions and conventions, but nope, these languages unavoidably attract people that want to stay in the ivory tower.

    Really, I love both lisps and haskell, but I would take a dreadful php over them every single day at work. No contest.

    • The problem is that the type of developer you're talking about is the kind who invents a thousand excuses about anything. It doesn't begin and end with different languages. Using a different subset of their favorite language? A different toolchain? A radically different build process? You'll hear plenty of whining about it. Even just moving them to a different codebase will make them miserable. You put a Lisp or a Prolog in front of them, and their attention flows upwards to the most general principles to whine about. The entire thing is that if you stick them into an unfamiliar environment, they will vocalize their discomfort through rationalization. I think most people who have led teams will know exactly what I'm talking about.

      Not that these people are necessarily bad at their jobs, but they're definitely a personality type you should learn to identify so you can manage them properly.

      1 reply →

    • I disagree with this profoundly. There is no argument that Lisp in general can get you in a terrible mess. So can C, or pretty much any language (I well remember a case of 11-way multiple inheritance in C++ game code). If I were running a Lisp-based project, there would be project standards (like no exported macros without signoff, and a list of standard library dependencies), documentation requirements, and code review.

      The fact that Lisp is the “programmable programming language” doesn't mean that every engineer should be inventing weird versions of while loops. What it does mean is that a skilled Lisp programmer can build a domain-specific language that substantially helps in development.

      One good example of this is the Crash Bandicoot games, along with the same studio's Jax and Daxter, which were built with dialects of Lisp (yes, they had to compile the code, and take care with storage allocation, just like any other game code). Cisco hired Kent Dybvig, principal author of the Chez Scheme system, and open-sourced that software; I have no clue what they use it for, to be honest, but I assume that they had some reason for doing this. Companies using various Lisp languages have been documented in areas from fintech to quantum computing.

    • I was fortunate enough to use lisp professionally for nearly 20 years. The only reason that gig ended was the unfortunate demise of the majority owner and founder, and his heirs selling the shop to a competitor that only wanted the customer database imported. Lisp was never the problem, nor particularily messy.

      On the contrary, we considered it a major reason we were able to build a quite successful stock broker and bank from scratch with a team of 4-6 people, who doubled as datacenter ops (we ran on-prem), internal tech support for the rest of the firm (~20 people) and 2.line customer support.

      We quite naturally converged on a set of internal libraries for various tasks. Understanding and fixing each other’s code was not a big deal.

      Coincidentally, our frontend was php. That was messier, and we conciously kept that layer as lean as possible.

    • >> said: lovely, but I ain't using it at work.

      Says who, you? Wrong. I use Common Lisp at work.

      edit: and not just use as in a side toy, design and writing software in CL is my primary responsibility. FWIW, at a FAANG!

      3 replies →

    • +1 upvote

      I don’t agree with you but your comment is very well thought out and doesn’t deserve downvotes.

    • > The very power of Lisp, macros, dynamic programming, reflection lead to a mess of an ecosystem where every single developer reinvents the wheel and nobody can understand yet another DSL invented by the next developer next to them.

      Have you like looked at anything in the Clojure ecosystem because this description makes no sense of any kind. Clojure has macros but less powerful than common lisp and Clojure code in the ecosystem tends to be small and understandable.

      6 replies →

    • I have a complete and utterly opposite experience. I have used and shipped all sorts of solutions in dozens of different languages before getting to know Lisp. My only regret is that I have not tried learning Lisp sooner - it feels like such a walloping waste of my life.

      Lisp dialects turned out to be extremely practical - I just can't even imagine doing the things I do today with Emacs in anything else. Achieving similar effects with any other tool would be enormously more time consuming and far more frustrating. Sure, it really took me years to get to that point, but even though I had already knew all sorts of other tools which I could've picked to achieve similar results - from bash/zsh, awk, sed and tcl to python, golang and a bunch of others, the way Lisp-driven Emacs lets me get there is beyond meaningful comparison. Nothing even comes close and I'm enormously proficient in vim, I spent almost a decade in IntelliJ and used tons of different IDEs before - from Delphi and Eclipse to Visual Studio and VSCode.

      Clojure, at which I scoffed at some point, turned out to be an immensely pragmatic tool for dealing with data. Any kind of data - big or small. It effectively replaced jq in my toolbelt, all my API interrogations happen in a Clojure REPL today. Fetching data from psql, mysql, sqlite and sorting, slicing, dicing, transforming that data interactively, directly from my editor is an absolute delight.

      Building simple web-scraping scripts with nbb driving Playwright is so much faster than using any other stack, even javascript is no longer "an obvious choice" for me, even though I spent decades learning its quirks.

      Babashka has replaced any kind of bash-scripting - anything longer than three lines almost has no reason to be made in bash/zsh/fish anymore.

      Anything Lua-driven is now done with Fennel. It's not even funny how a dozen-lines boilerplate of Lua can be compacted into a simple, reasonable three-liner Fennel macro. Or the way how I can connect to the Hammerspoon REPL and poke through visual elements of any app on Mac, interactively and with ease - is complete and total bananas.

      Era of LLMs incidentally highlighted another enormous advantage of Lisp - when you give an LLM a true Lisp REPL, agents stop guessing and start empirically analyzing current state of things and produces working solution faster, costing far less tokens. And you get to watch it solve things interactively, e.g. I often let AI poke through our UI (via Playwright-driven Clojurescript REPL), while monitoring situation in k8s - through nrepl port, connected to Clojure REPL. It literally interactively walks the DOM, finds the selectors, clicks buttons, etc. All without restarts, complex state management and all.

      "Well", you may say: "these examples exactly what I'd consider a 'hobby language' territory".

      Alas, I have worked in teams where Clojure was used for shipping commercial software and I have seen truly complex projects - Cisco's infosec infrastructure and FindingCircle's complex Kafka topologies. I have met and talked to people working at Netflix, Apple, Amazon, Nubank, CircleCI, etc., and I can confidently say: your self-conviction is absolutely backwards.

      Lisp-world has never been more cornucopian than today; we see the proliferation of different tools and new Lisp dialects popping almost every passing week - Jank, Jolt, ClojureDart, Squint, Coalton, Clojerl, LFE, uLisp, etc. It is frankly inconceivable to find today any platform where you can't really run a Lisp. My advice to any programmer who's aspired to become a hacker - do learn Lisp. It comes in handy. For real.

If any SBCL dev is here, please add documentation for how to use the memory arena feature. The only doc is an very old proposal document.

  • Yeah I'm not sure why it's not in the manual, as it's had arena allocation support since at least 2.4.x, but basically:

    - use SB-VM:NEW-ARENA to make a new arena

    - use SB-VM:WITH-ARENA to redirect ordinary allocation into an existing arena like you would use WITH-OPEN-FILE or similar macros

    The only real doc is this internals note, and it doesn't even cover NEW-ARENA which I guess is left as an exercise to the reader: https://github.com/sbcl/sbcl/blob/master/doc/internals-notes...

If memory serves, traditionally, CCL (Clozure Common Lisp [0]) had better support for Windows but SBCL was unbeatable for speed. Is that still the case?

[0] https://ccl.clozure.com/

  • SBCL works fine on Windows. It used to display a warning about fragility at startup, but that was removed in version 2.0.3, 6 years ago, after already being obsolete for quite a while.

    CCL isn't very actively maintained and currently doesn't have an ARM64 port, but otherwise continues to work fine. I believe one reason people use it is that it compiles a bit faster than SBCL, at least in part by doing less optimization.

  • Part of why SBCL took longer is, IIRC, that SBCL went with embedding proper SEH support into generated code (important among other things to get page fault information), whereas CCL used VEH (which is simpler to use but only available from XP onwards)

Great to see the project is still going strong, I kinda want to try CL but I always feel like I don't have a great use-case.

  • An extensible LLM agent (such as https://pi.dev/ or maybe hermes) written in common lisp could be interesting. Conditions and restarts and general debugging and repair of the live system, fast startup, native execution speeds, ability to add or replace or modify core functionality on the fly, solid multi-threading support, dynamic introspection including documentation, CLOS and multiple dispatch, saved images.

  • CL can be used pretty much anywhere nowadays, even on the web with ECL wasm compilation.

    • Anywhere is where exactly? I've been thinking of where I could use it for like 10 minutes, and I couldn't come up with anything. Maybe a game engine or web services. It wouldn't make much sense for any kind of desktop software or command line utilities that are supposed to start really fast and have minimal runtime.

      3 replies →

I'm going pretty deep on Lisp on accident - can anybody give me more context on this?