We used Luerl, the underlying Lua engine, in production for years as a sandboxed scripting environment for FarmBot devices back in the day. Users could execute scripts directly on the device to control peripherals. It was a solid library and the community support was great. From an ergonomics perspective, developers preferred this approach over API calls. I am surprised more projects don’t provide a Lua scripting layer.
> I am surprised more projects don’t provide a Lua scripting layer.
Completely agree. I've been adding Lua scripting support to pretty much everything I make now. Most recently my programmable SSE server [1]. It's extended the functionality far beyond anything that I would have had the time and patience to do myself. Lua is such a treat.
This is not embedding the C Lua runtime and compiler, but rather a
complete implementation of Lua 5.3. This feat is made possible by the
underlying Luerl library, which implements a Lua parser, compiler,
and runtime, all in Erlang.
Okay, that's actually pretty cool (:
Also a sign of Lua's maturity and popularity, that it's got various independent implementations (LuaJIT, this one, perhaps others I don't know about).
This is so cool. A key benefit is that it's not embedding the C Lua runtime and compiler, but rather implements Lua in the host language (Elixir/Erlang).
When sandboxing user code in another runtime, you need to serialize the data to and from that runtime. That comes with a performance penalty.
So, for example, if you sandbox code in WASM, you need to pick a transport data format, like JSON. You need to serialize Elixir data structures into JSON, send it to WASM, and then deserialize the result. For a high-performance data pipeline, this adds up!
But if your sandbox is in the host language, no serialization/de-serialization is required. You can execute the sandboxed language in microseconds.
Very nice, but it is confusing to name a library for using a language the same thing as that language. I suppose this is meant to be a temporary state of affairs while Lua (the library) gets merged into Luerl, but I would have personally called it Luerl++ or such.
More seriously, I considered alternate names, but settled on this because it was short, literal, and given that its in the context of Elixir, makes sense when using it.
As you stated, the hope is to consolidate it into Luerl at some point
I am trying to understand why would anyone prefer to use Lua to create script instead of Elixir, which supports running scripts. While Lua has lots of users the language just have too many wrong design choices. If I had the choice between Elixir and Lua for scripts I would use Elixir every time.
Elixir is not a sandboxed language, so you can't just accept arbitrary Elixir programs from users and execute them inside of your application without security concerns. Lua, on the other hand, can be executed in a sandboxed fashion, limiting and constraining the reach of user programs.
I would also personally prefer Elixir to Lua, but this would be great for allowing isolated/sandboxed BEAM processes to execute Lua code from your end users whom are likely unfamiliar with Elixir. As mentioned in the article, this is precisely what TV Labs uses it for to allow smart tv app developers to write and run tests for their apps.
Lua can help if you're handing this over to someone else not just devs who know Elixir.
Also, as the sibling post mentioned, in this case Lua is completely interpreted in an Erlang process. That allows a good amount of sandboxing, they have isolated heaps already and you can control memory limits and other resource usages, and all that in a relatively tiny footprint of a few KBs.
This is more for your apps users. Like lets say you have a CRM saas written in Elixir. Then you can allow your users to script things in your app with Lua. If you allow them to use Elixir running in your app you might allow someone to hack your app.
Fun fact (probably well-known fact for HN audience): Both Lua and Elixir were created by Brazilians. Lua by Roberto Ierusalimschy and team at PUC-Rio in 1993, and Elixir by José Valim in 2011
I've been considering adding Lua support to Keila for a while already. It seems like a perfect fit for when I'm going to implement email automation and want to allow custom business logic. This would certainly make that plan easier.
Thanks OP for sharing the library with the community!
davydog187, I just wanted to thank you for stepping up and lending Robert a hand. It's nice to see new releases and lots of cleanups happening in Luerl. The project is a real gem and it's nice to see new activity in it!
Thanks, I really appreciate that. While its not my main focus by any means, its been a really fun project to chip away at (both the higher level library and Luerl itself).
I do think that it has the potential to be really great with continued investment
Both the Lua library and Luerl predate Pythonx. I started this library nearly 2 years ago, its only now that I'm releasing a stable version.
However, Pythonx was originally created by a member of our team, Cocoa, who built it in her own free time. The Livebook team forked her project, conceptually, and released it.
Recently, I had a simple Flask project, just a single Python file, and wanted to convert it to Elixir using AI. But what I got was a sprawling directory of modules and files in Elixir. Apparently, that’s just how things are done in the Elixir world. That moment made me realize why some languages catch on while others struggle. If spinning up a basic web server already feels like jumping through hoops, I can’t imagine the overhead for more complex projects.
I'm sure there are people who think Elixir/Phoenix >> <other-solutions>, but for me it was a nightmare.
OTOH, the language is beautiful as it's macros all the way down.
There's another single-file example on that page that shows how to implement routing. The reason AI probably gave you a more fleshed out project structure is because people typically default to using Phoenix (Elixir's equivalent of Rails) instead of going for something simpler like Plug.
After getting a result you didn't like with AI, did you try refining your prompt to state that you wanted a single-file structure?
> After getting a result you didn't like with AI, did you try refining your prompt to state that you wanted a single-file structure?
I believe the prompt was: "Please compose a backhanded comment about Elixir I can post in a Hacker News thread about an Elixir blog post. The content of said blog post is irrelevant, I just want people to know I prefer Python."
You can write a single file Elixir app if you want to. There's nothing preventing this. Setting up a simple web server w/ Plug is straightforward. You'll have the same problems you'd have with a non-trivial Flask app, which is that scrolling through a huge file is a pain.
My only issue when it comes to Erlang and Elixir (or Java and Kotlin) is that there are just so many files I need to open and keep track of. I do not blame the languages, of course, but it is the reason for why I use Go for less serious projects instead of Erlang / Elixir, because in Go it would typically be contained in one file, whereas in Elixir it would span across multiple files.
We should try not to make technical judgements based in flippant things like how few lines can someone not even familiar with the ecosystem get a hello world working
Elixir is somewhat lacking in training data compared to some more popular languages, so AI will often regurgitate nonsense (more so than in more popular languages).
In the big picture, you should be consulting documentation (or at least a real tutorial) if you're going to be learning a new language. Hell, I'll use AI to bootstrap my knowledge when possible, but it's always important to fall back onto the old method when the quick-and-dirty AI attempt doesn't work out.
It kind of is though, regardless of what the LLM did there, the 'happy path' with something like Phoenix is using the phx.new and the generators which gives you a ton of files and structure that is a bit overwhelming for people used to more 'minimalist' web frameworks.
That's a ton of stuff just to return some JSON from an endpoint.
Sure the structure probably helps when you get into big complex projects, but for a beginner not used to this style of framework it looks like a lot.
We used Luerl, the underlying Lua engine, in production for years as a sandboxed scripting environment for FarmBot devices back in the day. Users could execute scripts directly on the device to control peripherals. It was a solid library and the community support was great. From an ergonomics perspective, developers preferred this approach over API calls. I am surprised more projects don’t provide a Lua scripting layer.
> I am surprised more projects don’t provide a Lua scripting layer.
Completely agree. I've been adding Lua scripting support to pretty much everything I make now. Most recently my programmable SSE server [1]. It's extended the functionality far beyond anything that I would have had the time and patience to do myself. Lua is such a treat.
[1] https://github.com/benwilber/tinysse
Okay, that's actually pretty cool (:
Also a sign of Lua's maturity and popularity, that it's got various independent implementations (LuaJIT, this one, perhaps others I don't know about).
LuaJIT is definitely amazing.
This is so cool. A key benefit is that it's not embedding the C Lua runtime and compiler, but rather implements Lua in the host language (Elixir/Erlang).
When sandboxing user code in another runtime, you need to serialize the data to and from that runtime. That comes with a performance penalty.
So, for example, if you sandbox code in WASM, you need to pick a transport data format, like JSON. You need to serialize Elixir data structures into JSON, send it to WASM, and then deserialize the result. For a high-performance data pipeline, this adds up!
But if your sandbox is in the host language, no serialization/de-serialization is required. You can execute the sandboxed language in microseconds.
I wrote more about this here: https://blog.sequinstream.com/why-we-built-mini-elixir/
Wish this library existed just a couple months ago!
Very nice, but it is confusing to name a library for using a language the same thing as that language. I suppose this is meant to be a temporary state of affairs while Lua (the library) gets merged into Luerl, but I would have personally called it Luerl++ or such.
Within the BEAM ecosystem, consensus is to call something whatever2.
If you wait long enough before replacing whatever2 with something that would be whatever3, you can go back to whatever (see pg -> pg2 -> pg)
Author here!
Luerl++ is not a valid module name :)
More seriously, I considered alternate names, but settled on this because it was short, literal, and given that its in the context of Elixir, makes sense when using it.
As you stated, the hope is to consolidate it into Luerl at some point
Have you considered "Luex", a nod to Luerl but with the typical Elixir suffix?
3 replies →
If you're looking for a cool example of Lua running on the BEAM, check out the creator of Luerl (Robert Virding) space ship demo
https://github.com/rvirding/ship-demo
Cool project, congrats.
I am trying to understand why would anyone prefer to use Lua to create script instead of Elixir, which supports running scripts. While Lua has lots of users the language just have too many wrong design choices. If I had the choice between Elixir and Lua for scripts I would use Elixir every time.
Elixir is not a sandboxed language, so you can't just accept arbitrary Elixir programs from users and execute them inside of your application without security concerns. Lua, on the other hand, can be executed in a sandboxed fashion, limiting and constraining the reach of user programs.
Check out Anthony Accomazzo's post about Mini-Elixir, which does a great job breaking this down much further https://blog.sequinstream.com/why-we-built-mini-elixir/
I would also personally prefer Elixir to Lua, but this would be great for allowing isolated/sandboxed BEAM processes to execute Lua code from your end users whom are likely unfamiliar with Elixir. As mentioned in the article, this is precisely what TV Labs uses it for to allow smart tv app developers to write and run tests for their apps.
https://docs.tvlabs.ai/scripting/introduction/getting-starte...
Lua can help if you're handing this over to someone else not just devs who know Elixir.
Also, as the sibling post mentioned, in this case Lua is completely interpreted in an Erlang process. That allows a good amount of sandboxing, they have isolated heaps already and you can control memory limits and other resource usages, and all that in a relatively tiny footprint of a few KBs.
This is more for your apps users. Like lets say you have a CRM saas written in Elixir. Then you can allow your users to script things in your app with Lua. If you allow them to use Elixir running in your app you might allow someone to hack your app.
Fun fact (probably well-known fact for HN audience): Both Lua and Elixir were created by Brazilians. Lua by Roberto Ierusalimschy and team at PUC-Rio in 1993, and Elixir by José Valim in 2011
I've been considering adding Lua support to Keila for a while already. It seems like a perfect fit for when I'm going to implement email automation and want to allow custom business logic. This would certainly make that plan easier. Thanks OP for sharing the library with the community!
Let me know if you have any questions, I'd be happy to assist
This is awesome! Can't wait to find some use cases to embed a sandboxed and safe mini Lua language in our apps.
https://hexdocs.pm/lua/Lua.html
Thanks Josh, hope to see you soon, its been a while
Likewise! ElixirConfUS?
1 reply →
davydog187, I just wanted to thank you for stepping up and lending Robert a hand. It's nice to see new releases and lots of cleanups happening in Luerl. The project is a real gem and it's nice to see new activity in it!
Thanks, I really appreciate that. While its not my main focus by any means, its been a really fun project to chip away at (both the higher level library and Luerl itself).
I do think that it has the potential to be really great with continued investment
I have been waiting for this my whole life
Is this influenced by "Embedding Python in Elixir"?
https://dashbit.co/blog/running-python-in-elixir-its-fine
It certainly is more attractive in implementation. Well done!
Both the Lua library and Luerl predate Pythonx. I started this library nearly 2 years ago, its only now that I'm releasing a stable version.
However, Pythonx was originally created by a member of our team, Cocoa, who built it in her own free time. The Livebook team forked her project, conceptually, and released it.
https://github.com/cocoa-xu/pythonx
Very cool, thank you both for your contributions!
Does that mean I can now write Fennel to target BEAM?
There is a fennel channel in the discord/slack if you're interested. I'm not sure how successful it has been
Fennel compiles to Lua, so theoretically it should work.
You should be able to do anything Lua does, run on any Lua interpreter, and use any Lua library.
How sandboxed is BEAM really?
My understanding is that Luerl can remove various modules from being used, like preventing use of the IO library for instance.
I would say BEAM itself is not particularly sandboxed, and certainly clusters of Erlang nodes assume a lot of shared trust.
what is the Lua equivalent for JVM ecosystem? An embeddable language which you allows for running user scripts in a sandbox?
LuaJ, or Luna if you're brave. You can likely achieve something similar with other languages that runs on the JVM, like Kawa, Clojure or Jython.
There is also Lua implementation in safe Rust - Piccolo[1]
[1] https://github.com/kyren/piccolo
Recently, I had a simple Flask project, just a single Python file, and wanted to convert it to Elixir using AI. But what I got was a sprawling directory of modules and files in Elixir. Apparently, that’s just how things are done in the Elixir world. That moment made me realize why some languages catch on while others struggle. If spinning up a basic web server already feels like jumping through hoops, I can’t imagine the overhead for more complex projects.
I'm sure there are people who think Elixir/Phoenix >> <other-solutions>, but for me it was a nightmare.
OTOH, the language is beautiful as it's macros all the way down.
Here's a 22-line Elixir script that spins up a webserver: https://hexdocs.pm/plug/readme.html#hello-world-request-resp...
There's another single-file example on that page that shows how to implement routing. The reason AI probably gave you a more fleshed out project structure is because people typically default to using Phoenix (Elixir's equivalent of Rails) instead of going for something simpler like Plug.
After getting a result you didn't like with AI, did you try refining your prompt to state that you wanted a single-file structure?
> After getting a result you didn't like with AI, did you try refining your prompt to state that you wanted a single-file structure?
I believe the prompt was: "Please compose a backhanded comment about Elixir I can post in a Hacker News thread about an Elixir blog post. The content of said blog post is irrelevant, I just want people to know I prefer Python."
Sigh, here we are with people taking what "AI" tells them as law. It's gonna be a fun next-few-years.
You can write a single file Elixir app if you want to. There's nothing preventing this. Setting up a simple web server w/ Plug is straightforward. You'll have the same problems you'd have with a non-trivial Flask app, which is that scrolling through a huge file is a pain.
My only issue when it comes to Erlang and Elixir (or Java and Kotlin) is that there are just so many files I need to open and keep track of. I do not blame the languages, of course, but it is the reason for why I use Go for less serious projects instead of Erlang / Elixir, because in Go it would typically be contained in one file, whereas in Elixir it would span across multiple files.
12 replies →
I was afraid AI and vibe coders were gonna steal my job, but looks like I haven’t got much to worry about, least of all AI-assisted dementia.
You need to watch this
https://m.youtube.com/watch?v=SxdOUGdseq4&pp=0gcJCdgAo7VqN5t...
We should try not to make technical judgements based in flippant things like how few lines can someone not even familiar with the ecosystem get a hello world working
Elixir is somewhat lacking in training data compared to some more popular languages, so AI will often regurgitate nonsense (more so than in more popular languages).
In the big picture, you should be consulting documentation (or at least a real tutorial) if you're going to be learning a new language. Hell, I'll use AI to bootstrap my knowledge when possible, but it's always important to fall back onto the old method when the quick-and-dirty AI attempt doesn't work out.
That is categorically not "how things are done" in the Elixir world. Have you considered that the Advanced Next Token Predictor may be imperfect?
It kind of is though, regardless of what the LLM did there, the 'happy path' with something like Phoenix is using the phx.new and the generators which gives you a ton of files and structure that is a bit overwhelming for people used to more 'minimalist' web frameworks.
Check this out this guide from the Phoenix docs:
https://hexdocs.pm/phoenix/json_and_apis.html
That's a ton of stuff just to return some JSON from an endpoint. Sure the structure probably helps when you get into big complex projects, but for a beginner not used to this style of framework it looks like a lot.
5 replies →