← Back to context

Comment by susam

5 days ago

A few years ago, I decided to migrate my personal website to a Common Lisp (CL) based static site generator that I wrote myself. In hindsight, it is one of the best decisions I have made for my website. It started out at around 850 lines of code and has gradually grown to roughly 1500 lines. It handles statically rendering blog posts, arbitrary pages, a guestbook, comment pages, tag listings, per tag RSS feeds, a consolidated RSS feed, directory listing pages and so on.

I have found it an absolute joy to maintain this piece of little 'machinery' for my website. The best part is that I understand every line of code in it. Every line of code, including all the HTML and CSS, is handcrafted. This gives me two benefits. It helps me maintain my sense of aesthetics in every byte that makes up the website. Further, adding a new feature or section to the site is usually quite quick.

I built the generator as a set of layered, reusable functions, so most new features amount to writing a tiny higher level function that calls the existing ones. For example, last month I wanted to add a 'backlinks' page listing other pages on the web that link to my posts and it took me only about 40 lines of new CL code and less than 15 minutes from wishing for it to publishing it.

Over the years this little hobby project has become quite stable and no longer needs much tinkering. It mostly stays out of the way and lets me focus on writing, which I think is what really matters.

Only problem I find with self-hosted blogs and certain personalities like mine is that I spend more time tinkering with the blog engine than actually blogging.

I ended up migrating back to a hosted solution explicitly because it doesn't allow me such control, so the only thing I can do is write instead of endlessly tinkering with the site.

  • I ended up separating out a "plumbing" blog, from the "real" blogs, with no discussion of the tinkering allowed on the real ones - so the plumbing blog grew in details but didn't "count" for the non-meta blogging I was trying to accomplish. A little bit of sleight-of-hand but it worked for me...

    • In my case it was less about the discussion of the tinkering and more the tinkering itself. I'd spend all my blogging time tinkering with the site, to the point where it's never ready and never actually deployed. As of right now in my projects folder I have an (actually finished and usable) Ghost theme and a handful of Wagtail blog projects in various states of functionality. Neither have actually been deployed. (at least I learnt enough Wagtail to be dangerous so I guess that's a win)

      I ended up subscribing to Bear Blog and calling it a day. In fact I need to delete those half-baked attempts so I am never tempted to get back to them.

  • > I spend more time tinkering with the blog engine than actually blogging.

    You should write a blog about it, like geerlingguy did.

  • > Only problem I find with self-hosted blogs and certain personalities like mine is that I spend more time tinkering with the blog engine than actually blogging.

    I don't see the problem with that ;)

  • Honestly this is so true. I have a few blogs for various reasons, and the hosted ones are where I post most because it’s so effortless to do. There’s so much less inertia. You can go even further and post by email (I use Pagecord) which removes virtually all barriers to posting.

    That said, building your own static site and faffing with all the tech is generally an enjoyable distraction for most techies

I'm also happy with the freedom and stability of a single-purpose static site generator. My previous project, Tclssg, was public and reusable from the start. This had big upsides: I learned to work with users and was compelled to implement features I wouldn't have. I actually wrote documentation. Seeing others use it was one of the best parts of the work. However, it also put constraints on what I could do. I couldn't easily throw away or radically change features, like how templates are rendered by default. With an SSG that's only for my site, I can.

If I were maintaining multiple large sites or working with many collaborators, I'd rely on something standard or extract and publish my SSG. For a personal site, I believe custom is often better.

The current generator is around 900 SLOC of Python and 700 of Pandoc Lua. The biggest threats to stability have been my own rewrites and experimentation, like porting from Clojure to Python. I have documented its history on my site: https://dbohdan.com/about#technical-history.

I did the same thing, but implemented my site generator in Go. My site has grown by a lot over the years, but I can still build it from scratch (from MD files, HTML snippets and static files) in less than one second!

Also have a RSS feed generator and it can highlight code in most programming languages, which is important to me as I write posts on many languages.

I did try Hugo before I went on to implement my own, and I got a few things from Hugo into mine, but Hugo just looked like far too overengineered for what I wanted (essentially, easy templating with markdown as the main language but able to include content from other files either in raw HTML or also markdown, with each file being able to define variables that can be used in the templating language which has support for the usual "expression language" constructs). I used the Go built-in parser for the expression language so it was super easy to implement it!

Used this for code syntax higlighting: https://github.com/alecthomas/chroma And this for markdown: https://github.com/russross/blackfriday

The rest I implemented myself in simple to read Go code.

Ha. Well, https://taoofmac.com was ported to Hy (https://github.com/rcarmo/sushy) in a week, then I eventually rewrote that in plain Python to do the current static site generator —- so I completely get it.

I am now slowly rebuilding it in TypeScript/Bun and still finding a lot of LISP-isms, so it’s been a fun exercise and a reminder that we still don’t have a nice, fast, batteries-included LISP able to do HTML/XML transforms neatly (I tried Fennel, Julia, etc., and even added Markdown support to Joker over the years, but none of them felt quite right, and Babashka carries too much baggage).

If anyone knows about a good lightweight LISP/Scheme dialect that has baked in SQLite and HTML parsing support, can compile to native code and isn’t on https://taoofmac.com/space/dev/lisp, I’d love to know.

  • Babashka now has a built-in markdown library. Babashka now also has a built-in HTML parsing library: Jsoup.

    Why does bb carry too much baggage? Because it has useful libraries like the above?

How do you handle comments in a "static blog"?

  • > How do you handle comments in a "static blog"?

    The comment form is implemented as a server-side program using Common Lisp and Hunchentoot. So this is the only part of the website that is not static. The server-side program accepts each comment and writes to a text file for manual review. Then I review the comments and add them to my blog.

    In the end, the comments live like normal content files in my source code directory just like the other blog posts and HTML pages do. My static site generator renders the comment pages along with the rest of the website. So in effect, my static site generator also has a static comment pages generator within it.

  • Not the poster, but what I did was to have a CGI script which would receive incoming comments and write them to "/srv/data/blog/comments/XXX/TIMESTAMP.txt" or similar.

    The next time I rebuilt the blogs the page "XXX" would render a loop of all the comments, ordered by timestamp, if anything were present.

    The CGI would send a "thanks for your comment" reply to the submitter and an email to myself. If the comment were spam I'd just delete the static file.

  • You could have a _somewhat_ static blog and incorporate something like Webmentions[0] for comments or replies. For example, Molly White's microblog[1] shows the following text below the post:

      Have you responded to this post on your own site? Send a webmention[0]! Note: Webmentions are moderated for anti-spam purposes, so they will not appear immediately.
    

    I find this method to be a sweet spot between generating content on your own pace, while allowing other people to "post" to your website, but not relying on a third-party service like Disqus.

    [0] https://indieweb.org/Webmention

    [1] https://www.mollywhite.net/micro/entry/202511101848

  • The way I’ve seen it done is that you render a script tag that embeds a third party comment system (like Disqus).

  • On mine, I don't. Any interactivity is too much hassle for me to worry about wrt moderation etc. I also don't particularly care what random people have to say. If my friends like what I wrote, they can tell me on Signal or comment on the Bluesky post when I share the link.

    • I have my email barebones on the about page. I get five spam emails a week and that’s enough of a price to pay to have one way to be reached.

> I decided to migrate my website to a Common Lisp based static site generator that I wrote myself.

Many programmers' first impulse when they start[0] to blog is to write their own blog engine. Props to you for not falling into that particular rabbit hole and actually using - as opposed to just tinkering on - that engine.

[0] you said you migrated it, implying you already had the habit of blogging, but still,

I started blogging with emacs and an org-based solution, and it was horrid.

I had a vision of what I wanted the site to look like, but the org exporter had a default style it wanted. I spent more time ripping out all the cruft that the default org-html exporter insisted on adding than it would have taken to just write a new blog engine from scratch and I wish I had.

There's a way to set a custom export template, but I couldn't figure it out from the docs. I found and still do find the emacs/org docs to be poorly written for someone who doesn't already understand the emacs internals, and I wasn't willing to spend the time to become an emacs internals expert just to write a blog.

So I lived with a half-baked org->pandoc->html solution for a while but now I'm on Jekyll and much happier with my blogging experience.

Your wife’s Python version is quite impressive. It wouldn’t have occurred to me to do the simple thing and just do some string-replacement targeted at a narrow use-case instead of using a complicated templating engine.

  • > just do some string-replacement targeted at a narrow use-case instead of using a complicated templating engine.

    A neat little in-between "string replacements" and "flown blown templating" is doing something like what hiccup introduced, basically using built-in data structures as the template. Hiccup looks something like this:

        (h/html [:span {:class "foo"} "bar"])
    

    And you get both the power of templates, something easier than "templating engine" and with the extra benefit of being able to use your normal programming language functions to build the "templates".

    I also implemented something similar myself (called niccup) that also does the whole "data to html" shebang but with Nix and only built-in Nix types. So for my own website/blog, I basically do things like this:

        nav = [ "nav"
          [ "h2" "Posts" ]
          [ "ul" (map (p: [ "li" [ "a" { href = "${p.slug}.html"; } p.title ] ]) posts) ]
        ]
    

    And it's all "native" Nix, but "compiles" to HTML at build-time, great for static websites :)

  • > Your wife’s Python version is quite impressive.

    Thank you. That was, in fact, the inspiration behind writing my own in CL.

Similar to my "Go 101" books website, about 1000 line of Go code (started from 500 lines at 9 years ago). The whole website can be built into a single Go binary.

Writing blog generator is not only fun but also grants ultimate control, such as static syntax highlighting, equation rendering and custom build steps. Highly recommend!

The posts looks pretty smooth to read.

You list all the links to posts on the landing page: what if you have 1000, 2000 posts ? Have you thought of paginating them ?

  • Thank you. My current home page has about 70 entries. The HTML size is about 7 kB and the compressed transfer size is about 3 kB.

    I created a test page with 2000 randomly generated entries here: <https://susam.net/code/test/2k.html>. Its actual size is about 240 kB and the compressed transfer size is about 140 kB.

    It doesn't seem too bad, so I'll likely not introduce pagination, even in the unlikely event that I manage to write over a thousand posts. One benefit of having everything listed on the same page is that I can easily do a string search to find my old posts and visit them.

  • Raises an interesting point.

    How large does the canvas need to get before pagination makes sense?

    Modern websites are enormous in terms of how much needs to be loaded into memory- sure, not all of it is part of the rendered document, but is there a limit to the canvas size?

    I'm thinking you could probably get 100,000+ entries and still be able to use CTRL+F on the site in a responsive way since even at 100,000+ entries you're still only about 10% of Facebooks "wall" application page. (Without additional "infinite scroll" entries)