Ruby and Its Neighbors: Smalltalk

5 hours ago (noelrappin.com)

What this blog omits to mention and I think the most impressive part of Smalltalk ecosystem is the structure of the image - the application's state that you can freeze and distribute and resume elsewhere (makes supporting client side applications so much easier). The Smalltalk image is a very interesting beast. Look at it as a kind of immortality. Many current Smalltalk systems, Pharo, Squeak, VisualWorks among them, share a common ancestor, that is, a Smalltalk image from Xerox PARC. This common ancestor however is not some remote thing, but actually still alive in those modern systems. The modern variants were produced by sending messages to the objects in that image. Some of those messages actually morphed the current objects. Classes are full-blown objects, and creating new classes is done by sending messages to class objects. Some of the objects in a Smalltalk image may date back to 1972, when the first Smalltalk image bootstrapped! Smalltalk images never die, they just fade into something potentially fundamentally different. You should view your application building as not fundamentally different from creating a new Smalltalk version.

  • The image concept, in my opinion, is what really limited Smalltalk's appeal and distribution.

    The image meant you basically got whatever state the developer ended up with, frozen in time, with no indication really of how they got there.

    Think of today's modern systems and open source, with so many libraries easily downloadable and able to be incorporated in your system in a very reproducible way. Smalltalk folks derided this as a low tech, lowest-common-denominator approach. But in fact it gave us reusable components from disparate vendors and sources.

    The image concept was a huge strength of Smalltalk but, really in the end in my opinion, one of the major areas that held it back.

    Java in particular surged right past Smalltalk despite many shortcomings compared to it, partially because of this. The other part of course was being free at many levels. The other half of Smalltalk issues beyond the image one, was the cost of both developer licenses ($$$$!) and runtime licenses (ugh!).

    • I agree that the image concept was a problem, but I think that you're focused on the wrong detail.

      The problem with an image based ecosystem that I see is that you are inevitably pushed towards using tools that live within that image. Now granted, those tools are able to be very powerful because they leverage and interact with the image itself. But the community contributing to that ecosystem is far smaller than the communities contributing to filesystem based tools.

      The result is that people who are considering coming into the system, have to start with abandoning their familiar toolchain. And for all of the technical advantages of the new toolchain, the much smaller contributor base creates a worse is better situation. While the file-based system has fundamental technical limitations, the size of the ecosystem results in faster overall development, and eventually a superior system.

      1 reply →

    • Agreed with that; this is why I think it should unite both the "scripting" as well as image approach, at the same time.

    • It wasn't the image concept. You use it every day in Docker containers for everything else.

      But saving the image has some drawbacks. Mutability always requires special care.

      4 replies →

    • Isn't this kinda where AI is now?

      We have things like LLM's and it's impossible to separate the "reasoning" from data it has stored to learn that reasoning.

    • For this very reason I'm working on a development platform that makes all changes part of a cheaply stored crdt log. The log is part of the application, there are some types of simulations inside of this that we can only timestamp and replay but we can always derive the starting position with 100% accuracy.

    • the entire philosophy of Smalltalk was to think of software artifacts as living entities. You can just find yourself in a piece of software, fully inspect everything and engage with it by way of a software archaeology. To do away with the distinction between interacting, running and writing software.

      They wanted to get away from syntax and files, like an inert recipe you have to rerun every time so I think if you do away with the image you do away with the core aspect of it.

      Computing just in general didn't go the direction they wanted it to go in many ways I think it was too ambitious of an idea for the time. Personally I've always hoped it comes back.

      1 reply →

  • That part is also cool. I'd like this in ruby too, e. g. where we could have tons of objects reflect on that state, and just resume work there. Everything is an object. Squeak everywhere, but to also retain the "scripting" (light) commandline approach. Nobody in smalltalk really understood why "scripting" is important. Python showed why.

  • > makes supporting client side applications so much easier

    I was thinking that supporting a Smalltalk application must be a nightmare because it is so malleable. Users can inspect and modify the entire system, no?

    • I used to think so, then watched as javascript in the browser rose to be the premium application platform - where the user has access to a console/repl, developer tools etc...

      1 reply →

    • > Users can inspect and modify the entire system, no?

      That should make the Smalltalk family popular with free software proponents. That makes me curious why that is not the case in history. The efforts of FSF on Smalltalk pale in comparison with those on C, Lisp and other languages.

  • Adding to what you said. Squeak was a clean open source reimplementation (by the devs who did the original Smalltalk-80), so it's real history starts from there (ie: the 90s, not the 70s)

    One thing to keep in mind is that smalltalks all have the same ability to save & load code to & from disk, just as any other programming environment. But, they also have the option of just using the image to persist, and iterate on that.

    Squeak overdid that aspect of it, such that over time, it became hard to prune older side projects & and it just became increasingly bloated. Both Pharo & Cuis forked from squeak at about the same time.

    Pharo images are fully bootstrapped from a seed.

    Cuis is not quite there yet, but cuis from its inception went on a ruthless simplification drive (the number of classes in the system was reduced by about 500% !), so that it's base is effectively a "seed", and the rest of a cuis image is built up by importing projects (from disk & git) on demand.

    But yeah, curating a set of images over time is remarkably enticing & friction free. Even in cuis, I find I have to force myself to keep flushing changes to my own packages.

    Its not that the tools to use files are limited. In cuis, they're not. You can work on multiple different things WITHIN THE SAME IMAGE (changes to some builtins, a couple of your own projects, etc), and the system will keep track of what belongs where. So a couple of mouse clucks will fileout the relevant code to the relevant changesets & packages.

    And yet - just banging on the same image is just ... fun, easy, enticing.

Since Smalltalk is effectively an OS unto itself, when we hear about the days of commercial applications being written in Smalltalk, how did deployment to end-users work?

Were the users running, say, Windows and then the Smalltalk "OS" would be running on top of that, but in a sort of "kiosk mode" where its full OS-ness was suppressed and it was dedicated to showing a single interface?

  • I used a "modern" app written in Cincom Smalltalk this year (still maintain and sold a high price), and you don't see the underlying system. Like you don't see the source code when opening any other app, in fact. The app resets the state at startup, so no lingering eternal session.

    It's funny because in the past I got the chance to test izware Mirai, which is written in Lisp — when the app got into a problematic state (which was often on my machine) you were sent to the REPL where you could inspect the memory and so on. It was alien to me at the time. Today I dream of having that.

  • I worked on a Smalltalk system which ran on Visual Smalltalk Enterprise, and in that system the image opened its windows as native Windows GDI windows, which made the application quite seamless in the OS (except this was in 2016-2018 and VSE was last updated in ‘99, so the look and feel was a bit dated :D).

  • Easy, you make use of tree-shaking, which is actually older concept than minifying JavaScript, and add glue it together with an executable header that boots the image.

    • I don't feel like this really answers OP's question, which is more about the user experience than the technical approach.

      When, as a dev, I use Smalltalk, it opens up what's effectively a virtual machine on my desktop. The whole Smalltalk GUI runs inside its own frame, none of the controls are native, etc. And it's a development environment - I have access to a class browser, a debugger, a REPL, and so on. I can drill down and read/modify the source code of everything. Which is great as a dev, but may be intimidating for an end user.

      Is that what the end user experience is like as well? I think that's what OP is asking. I've never used a Smalltalk application as an end user to my knowledge, so I can't say myself.

      2 replies →

Smalltalk is among the top most ergonomic ways to model concepts in a computer (capacity to model essential complexity) because the CPU got abstracted and compacted in the VM.

But destiny made CPUs win and now we're using AI to chew their accidental complexity for us.

thank you for this great post. my nostalgia for the late 90's ParcPlace IDE is heavy.

> You’ve likely used powerful coding editors and terminals. Smalltalk is just different. You are in the running environment.

I've experienced this a few different times: with Microsoft BASIC-80 (and GW-BASIC), with SBCL and SLIME, with LOGO, with GForth, with OpenFirmware, with MS-DOS DEBUG.COM, with Jupyter, and of course with Squeak. It really is nice.

It used to be the normal way of using computers; before memory protection, it was sort of the only way of using computers. There wasn't another memory space for the monitor to run in, and the monitor was what you used to do things like load programs and debug them. This approach continued as the default into many early timesharing systems like RT-11 and TENEX: there might be one virtual machine (memory space) per user, but the virtual machine you typed system commands into was the same one that ran your application. TENEX offered the alternative of running DDT (the debugger) in a different memory space so bugs in the application couldn't corrupt it, and that was the approach taken in ITS as well, where DDT was your normal shell user interface instead of an enhanced one.

All this seems very weird from the Unix/VMS/Win32 perspective where obviously the shell is a different process from your text editor, and it's designed for launching black-box programs rather than inspecting their internal memory state, but evolutionarily it was sort of the natural progression from a computer operator single-stepping a computer (with no memory protection) through their program with a toggle switch as they attempted to figure out why it wasn't working.

One of the nicest things about this way of working is halt-and-continue. Current versions of Microsoft Visual Studio offer sometimes halt and continue. In MBASIC you could always halt and continue. ^C halted the program, at which point you could examine variables, make arbitrary changes to the program, GOTO a line number, or just CONT to continue where you'd interrupted it. Smalltalk, SLIME, or ITS allows you to program in this way; if you like, you can refrain from defining each method (or function or subroutine) until the program tries to execute it, at which point it halts in the debugger, and you can write the code for the method and continue.

This is an extremely machine-efficient approach; you never waste cycles on restarting the program from the beginning unless you're going to debug program initialization. And in Smalltalk there isn't really a beginning at all, or rather, the beginning was something like 50 years ago.

Myself, though, I feel that the hard part of programming is debugging, which requires the experimental method. And the hard part of the experimental method is reproducibility. So I'm much more enthusiastic about making my program's execution reproducible so that I can debug faster, which conflicts with "you're in the running environment". (As Rappin says, "Code could depend on the state of the image in ways that were hard to replicate in deploys." I experience this today in Jupyter. It's annoying to spend a bunch of time trying to track down a bug that doesn't exist when you restart from scratch; worse is when the program works fine until you restart it from scratch.) So I'm much more excited about things like Hypothesis (https://news.ycombinator.com/item?id=45818562) than I am about edit-and-continue.

Paul Graham wrote somewhere (I can't find it now) about how in Viaweb's early days he would often fix a bug while still on the phone with the customer who was experiencing it, because he could just tweak the running CLisp process. But you can do the same thing in PHP or with CGI without sacrificing much reproducibility—your system's durable data lives in MariaDB or SQLite, which is much more inspectable and snapshottable than a soup of Smalltalk objects pointing to each other. (#CoddWasRight!) Especially since the broad adoption of the Rails model of building your database schema out of a sequence of "migrations".

  • > But you can do the same thing in PHP or with CGI without sacrificing much reproducibility

    PHP is similar, but not the same. You can't (or at least I can't) stop a request in progress and change its code; but you can rapidly change the code for the next request. Make a change in the editor, hit reload in the browser is a productive short loop, but stop at a breakpoint, inspect the state and change the code is a more powerful loop. Stopping at a breakpoint is challenging in systems with communication though, and I've learned to live without it for the most part.

    • Yes, I agree. I meant that you can edit code on the production web server so that the customer you're on the phone with sees their application start working. PG had the advantage that he could tweak the Lisp process for that specific user, reducing the risk of breaking the site for other concurrent users.

      Database transactions bridge some of the gap between "change the code for the next request" and "stop at a breakpoint and change the code": as long as your request handler code keeps failing, it will abort the transaction, so the database is unchanged, so you can restart the transaction as many times as you want to get to the same point in execution, at least if your programming language is deterministic. By providing a snapshot you can deterministically replay from, it allows you to add log entries before the point where the problem occurred, which can be very useful.

      Stopping at a breakpoint can be more productive, especially with edit-and-continue, but often it isn't. A breakpoint is a voltmeter, which you can use to see one value at every node in your circuit; logs are a digital storage oscilloscope with a spectrum analyzer, where you can analyze the history of millions or billions of values at a single node in your circuit.

  • I found the Smalltalk way of working in the running environment to be very programmer efficient too, and that it was by far the smoothest development experience I’ve had, even in a pretty dated and clunky Smalltalk at that point. And debugging wasn’t really a problem in my experience, but we stored application state outside of the image in an SQL database (initially Sybase, then MSSQL), which probably removes some «haha, the image has some weird data saved in the dark and dusty corners» issues.

thanks for this article. as a history of languages aficionado myself, I appreciate both the content and the style. thanks again for the good reading

I kind of like Smalltalk. Unfortunately syntax-wise Smalltalk is a huge set back and step back compared to Ruby. But Alan Kay's ideas are still epic today. I still don't think any language came close to his vision either; smalltalk definitely not. Ruby got closer but also no dice.

Elixir kind of got close too (prettier "erlang") - to have fault-tolerant mini-CPUs ("objects", aka similar to biological cells). The problem is that even people with grand ideas, such as Alan Kay, are not automatically great language designers. Matz is a better language designer than Alan Kay, for instance, whereas Alan Kay has the better ideas. It's a trade-off.

Note: I myself would not know how a language should look like that would follow Alan Kay's vision closer. It is definitely not smalltalk; it is also not ruby, though ruby is very good. I know I would suck as language designer, but as an improvement I would have a language similar to ruby (no, not crystal - crystal is a worse "ruby"), with a stronger focus on speed (perhaps have a second language that can be as fast as C here and be similar) and with a much stronger focus on what erlang brought to the table; that would get us close to, say, around 85% or 90%. And then add the rest that could fulfil Alan Kay's vision. Of course we need a specification too, because just saying "90% of his vision" is also pointless. I would perhaps start from erlang, retain the good bits, make it a modern variant of OOP (and OOP is also defined differently in different programming languages, so we need to define that term too, in the specification).

Rants like this about languages which have little/no code snippets are the worst because they often sell you an idea w/o actually showing how it works in practice (because often times it doesn't work well in practice). talk is cheap, show me the fucking code, else don't waste my time.

  • > Rants like this

    What about this makes you think it's a rant? Is the author making an impassioned plea for people to use Smalltalk? Is he going off on a tirade about something?

  • This article is more about the development environment than it is about the programming language, so it would be misleading to focus heavily on example code. A UI screencast video would be much more helpful.

  • The post is about the conceptual underpinnings of the system, not about the specifics of the code. It would be like judging the Linux kernel by the UI running atop it: it's largely immaterial to the discussion at hand.