Comment by corysama
9 months ago
At the indie studio I used to work at, we had some folks with engine experience. So, we rolled our own 3D engine and asset pipeline from scratch. But, we didn't have the budget for an editor. Instead we set up the asset pipeline to hot-reload everything. Meshes, scenes, materials and animations from Maya. Textures from Photoshop. Audio as a pile of WAVs. Scripting in Lua. UI layout in XML. Changing the asset files would change the game live.
Then we added "State machines as Lua exported from Excel". Rows are states, columns are events, cells are code to execute given a state+event combo. I've done this a few times. It makes huge state machines manageable.
Our games were very stats-heavy and our designers liked Excel. So, a new thing we added as "Dynamic data sources as grids of Lua exported from Excel". So, fill out Excel sheets like normal. But, instead of Excel script, every cell is evaluable Lua code. Strings, numbers, bools, functions are all values in Lua. So, a cell might contain a number. Or, it might contain a function checking the contents of two other cells and optionally triggering an event on some other object.
We shipped multiple games on a single executable using this system. Artists could lay out 3D scenes and 2D UIs with hooks for the Lua to control it. And, the designers could populate scenes and UIs from Excel dynamically according to the state of the game. The programmers mostly worked on game-agnostic features in C++, and the heavier side of scripting in Lua for game-specific features.
Aside: Lua is being so dynamically typed makes it not great for large-scale software engineering. But, https://teal-language.org/ might be a good TypeScript-For-Lua. I haven't tried it. Also, are there any Lua debuggers newer than the ancient https://github.com/unknownworlds/decoda that require zero integration? Decoda just need a pdb of your executable and it can automatically debug any scripts passing through the Lua library.
Eventually, we switched to Unity for corporate reasons. As a primary implementer of our custom engine, I think the switch was overall a good thing. Our games had started to outgrow what our little engine team could deliver. The artists reported they felt slightly less productive working in Unity's editor. I switched roles to finding and working around the undocumented bugs in Unity that we ran into. Then later finding which bugs had been fixed without being mentioned in the release notes so I could delete my work-arounds. It was a very boring couple of years until the company burned to the ground because of the same corporate reasons that lead us to switch to Unity :P
> But, we didn't have the budget for an editor. Instead we set up the asset pipeline to hot-reload everything. Meshes, scenes, materials and animations from Maya.
I was the tools lead for "Superman Returns: The Videogame" (the PS2/X630 Superman game, no the N64 one). We did the same thing. All of Metropolis was essentially "modeled" in Maya with plug-ins handling importing and exporting between that and the in-game format.
It was an open world streaming game and it would have killed an artist's machine to try to load the entire city into Maya, so the plug-in would let an artist pick which chunks of the city to load, load them in, let them make changes, and save the results back out.
It worked out fairly well.
Have you tried Luau - Roblox's open source compiled Lua with Types? Someone made a debugger for it that plugs into Visual Studio Code.
https://github.com/luau-lang/luau/ https://github.com/sssooonnnggg/luau-debugger
I'm working on an engine based in C++, Luau, and OpenGL - started almost 2 months ago. I aim for it to me MIT license open source, but it's too early for sharing. When it is, I do plan to post a show HN with the Github link.
I have not, but it looks great!
The usage of Excel is so hilariously cursed I can't stop thinking about it. Its just raw memory address look ups with extra steps. How did you merge the Excel file? Did you actually have source control on this?
Ok maybe a sheet is fine for a state machine if you don't care about visualizing transitions... Why reference cells when you could have named functions in a source file that could actually parse the syntax? That file would be trivially merged. You want to be loose on types so I guess its fine but you could actually have type safety and linting all the other language features and tooling if you just used a file full of named globals.
The mind boggles
> visualizing transitions
In systems where we used this it was common to have 100+ states x 100+ events. Visualizing that would have been fun, but not useful.
One implementation had layerable sheets. So, you could define a base state sheet and multiple overlay sheet that the game could enable dynamically on major changes.
Within each event column there would be long, broken up runs of "do the same thing" for runs of states. So, you could define a chunk of code in a cell below the main table and reference it in the runs and it would show in the main table.
Almost all functions were trivial. 1 or 2 statements. "Modify a value. TransitionToState X". There would be hundreds of these little snippets in a sheet. Naming each one would double the complexity for no gain. Let alone having function signatures. They'd all be `void randomName(event)`. And, Lua's a dynamic language. You aren't going to get much error checking in an IDE. Better to just keep everything local to the sheet rather than flipping between Excel and VSCode.
And, in the implementation discussed above, most of the cells were full of numeric/string values or references to assets.
I guess the main disadvantage with a custom engine vs Unity, is that you must train new people in it all, vs hiring experts in the tools.
Your custom engine sounds interesting, though. I can recognize some thoughts ..
This is so cool and clever!
Scott Bilas also documents doing something similar for Gabriel Knight 3, one of the “grandfathers” of data-driven engines and bit of an influence on ECS.
Instead of excel however it was comma-separated text files that shipped with the game. During development could edit the files locally and reload the data without recompiling.
Since we’re in a web heavy forum: An sql database and a web editor would also allow for a distributed workflow.
There's the Local Lua Debugger [0]. I don't know how it compares to decoda, but I experimented a bit with this debugger last weekend and it seems nice enough - well integrated into VS Code.
---
[0]: https://github.com/tomblind/local-lua-debugger-vscode
The magic of Decoda is that you can download the debugger and immediately start debugging your existing program with no modifications. No libraries. No sockets. No changes at all.
Decoda uses your PDB to locate the Lua C library within your .exe and the Win32 API CreateRemoteThread to inject the code into your process to hook the Lua API.