Comment by DonHopkins
3 years ago
Thank you, that's an interesting response to his quotes, too! Like the proverbial plate of shrimp, this stuff has been coming up a lot recently, so I've been researching and recombining ideas from Will's old talks about Spore and with Brian Eno, David MacKay's Dasher text input system, and I've also dug up some even older unpublished thoughts from Ed Fredkin and John Cocke about a Theory of Dreams, and how they relate to LLMs and compression.
Being able to fit a program that generates lots of content on a small floppy disk or cdrom is one goal, less important today, but the important part is that thinking of procedural content generation as decompression of noise (random, user generated, contextual, or environmental) is a useful technique even with today's virtually unlimited storage and high speed delivery.
Will speaks about compression in information theoretic and arithmetic coding terms, not just referring to standard compression algorithms like "jpeg" or "mp3", but to information encoding and decoding theory, using compression techniques for procedural generation, like LLMs.
Here's a great video of Will Wright and Brian Eno discussing generative systems and demonstrating cellular automata with Mirek's Cellebration to Brian Eno's generative music, at a talk at the Long Now Foundation,:
Will Wright and Brian Eno - Generative Systems (excepts from talk):
http://www.gameaipro.com/GameAIPro2/GameAIPro2_Chapter40_Pro...
>One of the first examples of PCG was in the game Elite [Braben 84], where entire galaxies were generated by the computer so that there could be an expansive universe for players to explore without running afoul of memory requirements. However, unlike most modern games that incorporate PCG, Elite’s content generation was entirely deterministic, allowing the designers to have complete control over the resulting experience. In other words, Elite is really a game where PCG is used as a form of data compression. This tradition is continued in demoscenes, such as .kkrieger [.theprodukkt 04], which have the goal of maximizing the complexity of interactive scenes with a minimal code footprint. However, this is no longer the major goal for PCG systems.
>Regardless of whether creation is deterministic, one of the main tensions when creating a game with PCG is retaining some amount of control over the final product. It can be tempting to use PCG in a game because of a desire to reduce the authoring burden or make up for missing expertise—for example, a small indie team wanting to make a game with a massive world may choose to use PCG to avoid needing to painstakingly hand-author that world. But while it is relatively simple to create a system that can generate highly varied content, the challenge comes in ensuring the content’s quality and ability to meet the needs of the game.
I want to add this one line that tattooed itself on to my brain an early age, when I read a book about the screenwriting trade by William Goldman, of Princess Bride fame.
He cited it as the line that likewise instigated an irreversible tectonic shift in his own approach to the craft of writing. It was simply this: the realization that "poetry is compression."
I think this insight has stood up to the rise of technological leverage and may generalize extraordinarily well into some sort of philosophical ground truth, on the suspicion that given the mismatch between the scale of the universe and the two-odd kilogram lump of tissue in our skull, all of human knowledge is essentially an exercise in curated compression.