For anyone interested/thinks this stuff is cool, the author of Elevated has made a website for experimenting with real time pixel shaders in WebGL: https://www.shadertoy.com/
Some pretty incredible things have been done there.
Holy crap how have I not heard of this? This already scratches an itch I have for no good vertex shader editors. I'm looking at the "composed" graph editor thing, a little confusing but it looks really powerful. Thanks for sharing.
The practical answer is no. There is an unimaginable amount of possible 3:30 minute vidoes—far more than the number of possible 4kb or even 40kb files.
To be fair, most of those possible vidoes are just noise. We don't have to be able to compress those because people don't care if one video of noise is different from another. We also don't have to reconstruct the video perfectly: as long as it looks more or less the same, the audience is happy. (This is called "lossy compression".)
But even with these caveats, there is no realistic method for compressing realistic 3:30 minute videos that well on a computer. We likely can't do all that much more than current compression algorithms without a different set of tradeoffs. (Like being better at some videos but worse at others.)
That said, a big part of how compression works is by relying on information already present when decompressing. This demo relies on having a particular kind of chip with certain capabilities (ie a CPU and a GPU) and presumably some standard library functions... etc.
How well could we "compress" videos if we had more information available when decompressing? Here's a fun thought experiment: what if we had a model of a human mind? We could then feed in a pretty sparse description and have the model fill in the details in a natural intuitive way. It would be very lossy, but the results would be compelling.
And you know what? That's a decent mental model of how speech works! If you just look at information content, spoken words are not very dense. But if I describe a scene you can imagine it almost as if you're seeing a video. This works because we both have the same sort of brain as well as shared experiences and intentions.
You can think of speech as incredibly effective—but also rather lossy—compression.
It could be very useful to deliberately pursue SUPER lossy compression. As long as no one can really tell based on the end result, it doesn't really matter.
For example, if you can only tell something was lossy by directly comparing two instances of the same video during playback, then that's probably good enough in most situations.
It occurred to me that we could compress the hell out of written works by translating them into some super dense language, and ultimately only retain the basics of the meaning/concepts/some of the writing style. Then can re-translate that back to whatever language we want to read it in.
For compressing pictures or videos, there could be some similar translation to a much more compact representation. Would probably rely on ML heavily though.
4K of English text is a couple of pages of a novel, enough to describe a character and a situation, maybe an interaction. A good writer can conjure up a whole world in 4K... but probably not a description of an arbitrary 3 and a half minutes of activity.
Nice insight you brought with the CPU and the standard libraries being a relevant factor, hadn't thought of that.
Your thought experiment sounds more like a "codec" than a procedural generation. I guess it is an arbitrary line given that we are using CPU, etc. But the bigger the decompressing "model" the further away from true 4k compression we are.
The Kolmogorov Complexity of a video (or any other data) is the size of the shortest program which outputs that video then halts. This 4k executable is similar in spirit, but also follows strict rules about efficiency: Kolmogorov complexity places no time limits on that shortest program, whereas this program must output pixels fast enough to make the video realtime.
Sorry, I thought it was obvious, but the question is:
Could procedural generation be used to achieve amazing compression rates given a currently impossible to code algorithm?
Well, it's not compressed, it's generated.
You could generate an endless video with less code, but it would most likely be uninteresting. Scene demos are interesting because it's art and direction and music generated from algorithms rather than creating those things and compressing them efficiently.
But, yes, at some level there is an idea of a dna seed and a process to create something much more profound, we as humanity haven't come close to cracking that, though.
I suspect that if at all possible to have an algorithm that can generate the seeds plus the process to expand them, then that algorithm would take orders of magnitude longer to run then there would be practical in any meaningful time scale.
Not visuals, but along a similar vein, random number generators with high dimensionality and equidistribution can be coerced into generating very specific output, given enough exploration of the output space.
For example, and output of all zeros, or the source for a a random number generator itself, or a zipped archive of a work of Shakespeare.
I know a RasPi doesn't have near the specs needed to run this but I'd love to gut an old flat screen monitor and put it in a frame with a RasPi running something like this, generating random "art" and hang it on a wall somewhere...
That's awesome! Thank you for that link, a little pricy but the screen would be nicer than what I have laying around and it's more plug-and-play, bookmarked!
This was one of my favorite demos back in the day. I still have a copy of it (along with 10~ other favorites) sitting in a "Demoscene" folder somewhere. Many of them don't work on today's hardware/software, sadly (including this one).
It's great that it's open source now! That means if someone's really motivated, they can update it to run on modern environments (by no longer keeping it 4 KB), even OS X, etc.
And that indeed is what we have to look forward to in online forums over the next 5+ years, as machine learning cuts its teeth on places like reddit, HN & disqus, and spammers try to find new irritating ways of getting eyeballs.
That's unreal. On what kind of graphics hardware, though? Seems like it probably offloads most of the work on GPU whereas we'd have had to do most of it in software on HW weak enough that 4KB size actually mattered. And probably not achieve this demo.
Edit: I'm wrong about the two triangles. From the .nfo-file:
for those wondering, this a (too) low density flat mesh displaced with
a procedural vertex shader. there arent any texturemaps for texturing,
instead texturing (and shading) is defferred and computed procedurally
in a full screen quad. this means there is zero overdraw for the quite
expensive material at the cost of a single geometry pass. then another
second full screen quad computes the motion blur. camera movements are
computed by a shader too and not in the cpu, as only the gpu knows the
procedural definition of the landscape.
Thanks for detailed response. I figured it mostly did GPU stuff. So, real computing necessary here is a massively-parallel chip with generic and custom hardware with a bunch of memory plus a regular core using 4KB on other end. I think a more interesting challenge would be to force use of a subset of GPU functions or memory plus tiny memory on CPU side. I don't follow demoscene close enough to know if they subset GPU's like that. Idea being making them run closer to the old Voodoo or pre-GeForce GPU's to see just how much 2D or 3D performance once could squeeze out of it.
Tricks could have long-term benefit given any emerging FOSS GPU is more likely to be like one of the older ones given complexity of new ones. I'd clone one like SGI's Octane ones they used to do movies on with mere 200MHz processors. Meanwhile, similar tricks might let one squeeze more out of the existing, embedded GPU's in use. Maybe subset a PC GPU in demoscenes like one of the smartphone GPU's. Yeah, that's got some interesting potential.
Back in the day demos were more impressive imo. A lot of them now use direct x or opengl. For the most part the stuff you see isn't written by hand anymore AFAIK, they just have programs to generate the actual demo. Basically they use modeling programs.
There are all kinds of demos: some of them use models, some of them do not, some are technical feats, and some are artforms. Many are both.
Models are just serialized polygon meshes. We've been using models for demos for way longer than DX/OGL have existed. They're just another tool in the box which you can use (if you want to).
Using DirectX or OpenGL nowadays is like using the CPU: it's just part of the stack. They are probably lower level than you think: using DX/OGL isn't just doing something like drawModel(model, x, y, z), it's way lower level than that.
This demo in particular is not very different from old school demos. Back in the day we had interruptions, now we have API calls, but in the end shaders are just code. Elevated uses D3D to execute them in the GPU but that's all. And the synthesizer is apparently coded in ASM.
Even if you really miss the old school platforms there are still demos produced for them, often pushing the limits of what can be done.
Ah, so yes I appreciate demos which pushes hardware to the limit or create some new effect which hasn't been seen before. I feel most demos on older hardware falls within this category then newer demos.
Additionally, please correct me if I'm wrong, older demos didn't have nice graphics apis to call. They had to create and store what's given for free by Apis these days into their binary. I think the Amiga did have some 3d stuff?
With opengl with a handful of lines you could get a spinning cube with lighting. So much more work had to be put into older demos to get to the same point. On top of that, the demo writers really had to know the hardware well, and diving into undocumented behavior. A lot more was being calculated on the cpu back then too.
I haven't been on pouet.net in a while but I will look at newer demos.
Lots of groups use demotools, including conspiracy for their 64k prods. Check out the moleman documentaries for lots of interesting scene info and lore! http://vakondok.com/index_en.html
Cool idea, too bad it's not free software. It's less free than any software I've seen (explicitly saying that you can't use it for "settings where security is critical" -- something that doesn't even make sense from a software license perspective). It's like the "Good not Evil" line in the JSON license.
Release info: http://www.pouet.net/prod.php?which=52938
Binary: https://files.scene.org/view/parties/2009/breakpoint09/in4k/...
Video: https://www.youtube.com/watch?v=jB0vBmiTr6o
Everything you see and hear is procedurally generated by the 4096 byte executable, in real time. It still blows my mind 7 years after release...
Here is a presentation put together by one of the guys behind this release that explains a lot of about how it was made.
I assume you meant to link this: http://iquilezles.org/www/material/function2009/function2009...
3 replies →
The 4kb genre in the scene is incredibly vibrant the last few years. I think this demo really helped kickstart that.
4KB using directx/opengl, very different from old time writing stuff directly in the video card.
I bet they'd still prefer to write directly to the video card, but it's kinda hard nowadays to access GPU resources without an API.
new constraints though... you got a fit a couple of shaders in there too :)
Exactly, 4kb on top of the hundreds of megabytes of libraries and drivers.
8 replies →
Even 4KB demo in browsers is a thing these days.
the part I loved best was how those 4kb uncompressed to over 250MB of GPU instructions
For anyone interested/thinks this stuff is cool, the author of Elevated has made a website for experimenting with real time pixel shaders in WebGL: https://www.shadertoy.com/
Some pretty incredible things have been done there.
A friend of mine also has this project, http://shaderfrog.com/
Let's you experiment with and compose different shaders.
This is brilliant, I use shadertoy a bit to quickly test out where I've gone wrong in my fragment shaders (which is usually the whole thing).
This actually seems like it'll be much better for the job.
I'd love to see ISF export added too: https://www.interactiveshaderformat.com/spec
Holy crap how have I not heard of this? This already scratches an itch I have for no good vertex shader editors. I'm looking at the "composed" graph editor thing, a little confusing but it looks really powerful. Thanks for sharing.
He has a version of Elevated which runs on shadertoy:
https://www.shadertoy.com/view/MdX3Rr
(iq is him.)
Alas, it doesn't run on this Chromebook...
After viewing LFT's work in using an ATMEL microcontroller as a demoscene platform: http://www.linusakesson.net/scene/craft/index.php
...I had the thought a possible frontier in demoscene is make your own hardware out of discrete components to run your demo.
The MOnSter 6502 would count - http://monster6502.com/
Just because that wasn't enough, lft has also written a demo which created a custom CPU/GPU on an FPGA to run a demo:
http://www.linusakesson.net/scene/parallelogram/
Most programming jobs must feel terribly boring for anyone who managed to write that.
Here's what he's been up to lately: http://mygtc.gputechconf.com/quicklink/esLu6kr
Will an online recording be available of that talk? I'd love to watch it.
2 replies →
iq of rgba worked at pixar.
In year 2000 when I was 16 years old 64k fr-08 by farbrausch told me that I know nothing about programming ;)
[1] https://www.youtube.com/watch?v=Y3n3c_8Nn2Y
Indeed, and for me, before that, Future Crew's Second Reality.
Second Reality was the best. First demo that I saw was Spaceballs' State of the art https://www.youtube.com/watch?v=c5kuYfTCGLg and I vividly remember of thinking 'wtf is that, a video or what?' and then Second Reality a year later or so - https://www.youtube.com/watch?v=rFv7mHTf0nA But then (I look all and every demo out there by then at the age of 13) I saw Mars demo https://www.youtube.com/watch?v=_zSjpIyMt0k which hooked me forever on the path of graphics programming.
1 reply →
Although they generally didn't compete in the 4kb genre, Farbrausch open-sourced many of their demos and demo-making tools a few years ago [1].
fr-041: Debris [2], and kkrieger [3] (the 96kb FPS) are particularly worth checking out for the uninitiated.
[1] https://github.com/farbrausch/fr_public
[2] https://youtu.be/wqu_IpkOYBg
[3] https://youtu.be/2NBG-sKFaB0
Does this mean that lots of 3:30 minutes 1080p videos could be compressed into 4kb?
EDIT: We can be generous and say 40kb for sake of adding more colours / etc.
This is actually an insightful question.
The practical answer is no. There is an unimaginable amount of possible 3:30 minute vidoes—far more than the number of possible 4kb or even 40kb files.
To be fair, most of those possible vidoes are just noise. We don't have to be able to compress those because people don't care if one video of noise is different from another. We also don't have to reconstruct the video perfectly: as long as it looks more or less the same, the audience is happy. (This is called "lossy compression".)
But even with these caveats, there is no realistic method for compressing realistic 3:30 minute videos that well on a computer. We likely can't do all that much more than current compression algorithms without a different set of tradeoffs. (Like being better at some videos but worse at others.)
That said, a big part of how compression works is by relying on information already present when decompressing. This demo relies on having a particular kind of chip with certain capabilities (ie a CPU and a GPU) and presumably some standard library functions... etc.
How well could we "compress" videos if we had more information available when decompressing? Here's a fun thought experiment: what if we had a model of a human mind? We could then feed in a pretty sparse description and have the model fill in the details in a natural intuitive way. It would be very lossy, but the results would be compelling.
And you know what? That's a decent mental model of how speech works! If you just look at information content, spoken words are not very dense. But if I describe a scene you can imagine it almost as if you're seeing a video. This works because we both have the same sort of brain as well as shared experiences and intentions.
You can think of speech as incredibly effective—but also rather lossy—compression.
It could be very useful to deliberately pursue SUPER lossy compression. As long as no one can really tell based on the end result, it doesn't really matter.
For example, if you can only tell something was lossy by directly comparing two instances of the same video during playback, then that's probably good enough in most situations.
It occurred to me that we could compress the hell out of written works by translating them into some super dense language, and ultimately only retain the basics of the meaning/concepts/some of the writing style. Then can re-translate that back to whatever language we want to read it in.
For compressing pictures or videos, there could be some similar translation to a much more compact representation. Would probably rely on ML heavily though.
4K of English text is a couple of pages of a novel, enough to describe a character and a situation, maybe an interaction. A good writer can conjure up a whole world in 4K... but probably not a description of an arbitrary 3 and a half minutes of activity.
1 reply →
Nice insight you brought with the CPU and the standard libraries being a relevant factor, hadn't thought of that.
Your thought experiment sounds more like a "codec" than a procedural generation. I guess it is an arbitrary line given that we are using CPU, etc. But the bigger the decompressing "model" the further away from true 4k compression we are.
Take a look at https://en.wikipedia.org/wiki/Kolmogorov_complexity
The Kolmogorov Complexity of a video (or any other data) is the size of the shortest program which outputs that video then halts. This 4k executable is similar in spirit, but also follows strict rules about efficiency: Kolmogorov complexity places no time limits on that shortest program, whereas this program must output pixels fast enough to make the video realtime.
Why would you think that? https://en.wikipedia.org/wiki/Procedural_generation https://en.wikipedia.org/wiki/Demoscene https://en.wikipedia.org/wiki/Real-time_computer_graphics
Sorry, I thought it was obvious, but the question is: Could procedural generation be used to achieve amazing compression rates given a currently impossible to code algorithm?
10 replies →
Well, it's not compressed, it's generated. You could generate an endless video with less code, but it would most likely be uninteresting. Scene demos are interesting because it's art and direction and music generated from algorithms rather than creating those things and compressing them efficiently.
But, yes, at some level there is an idea of a dna seed and a process to create something much more profound, we as humanity haven't come close to cracking that, though.
I suspect that if at all possible to have an algorithm that can generate the seeds plus the process to expand them, then that algorithm would take orders of magnitude longer to run then there would be practical in any meaningful time scale.
Not visuals, but along a similar vein, random number generators with high dimensionality and equidistribution can be coerced into generating very specific output, given enough exploration of the output space.
For example, and output of all zeros, or the source for a a random number generator itself, or a zipped archive of a work of Shakespeare.
It's fun to think about anyway.
http://www.pcg-random.org/party-tricks.html
But how many colours do we speak of now?
classic 4k, beautiful stuff. here's another one of my favourites from 5 years ago:
zeo-x-s/cdak
https://vimeo.com/25284552
http://prostheticknowledge.tumblr.com/post/73734927255/cdak-...
Beautiful soundtrack from Lassi Nikko (tpolm/orange) and crazy cool raymarching shader work from unc (quite)
shadertoy version with VR support:
https://www.shadertoy.com/view/MdyGzD
I know a RasPi doesn't have near the specs needed to run this but I'd love to gut an old flat screen monitor and put it in a frame with a RasPi running something like this, generating random "art" and hang it on a wall somewhere...
you mean like this? https://www.electricobjects.com/
That's awesome! Thank you for that link, a little pricy but the screen would be nicer than what I have laying around and it's more plug-and-play, bookmarked!
also impressive: https://en.wikipedia.org/wiki/.kkrieger
Conspiracies 64kb demo from earlier this year is fantastic too.
http://www.pouet.net/prod.php?which=67106 / https://www.youtube.com/watch?v=qRVTI2y0BBg
Awesome stuff, yet it was only voted 2nd in the competition (Revision 2016 64kb compo). Here's the winner (fermi paradox by mercury):
https://www.youtube.com/watch?v=JZ6ZzJeWgpY
The scenes with the moon being peppered with impacts (3:31) and what appears to be storms on Jupiter are beautiful.
1 reply →
How in the heck did they do that in 64kb
That is mind blowing.
Leaves quivering on a branch, realistic flames, realistic humans!
The water wasn't quite right but still.
1 reply →
That was amazing! Many focus on a narrow presentation or style. That had all kinds of visual and mechanical elements to it that were done smoothly.
First one I saw and part of how I mentally judge the rest. Amazing work.
here's a great read about how it was made by one of the members: https://news.ycombinator.com/item?id=7739599
If people wonder why I'm so obsessed with computers, this is the reason.
This was one of my favorite demos back in the day. I still have a copy of it (along with 10~ other favorites) sitting in a "Demoscene" folder somewhere. Many of them don't work on today's hardware/software, sadly (including this one).
It's great that it's open source now! That means if someone's really motivated, they can update it to run on modern environments (by no longer keeping it 4 KB), even OS X, etc.
And I had been planning to reverse engineer it for some time, but never got around to doing it.
Hats off to Inigo and others.
Am I the only one who's reminded of the Excel 97 easter egg?
Here's their release of how they made it as well. Super interesting read. It's shaders all the way down.
http://www.iquilezles.org/www/material/function2009/function...
Ha, that's my exact comment from the related reddit thread. Kinda cool, I wonder how often that happens?
https://www.reddit.com/r/programming/comments/4msf41/elevate...
Spammers, presumably trying to build up karma on HN. Needless to say we've banned them and will continue to.
Watchful users can help out with this a lot. If you or anyone notice comments being cross-posted in the future, please alert us at hn@ycombinator.com.
And that indeed is what we have to look forward to in online forums over the next 5+ years, as machine learning cuts its teeth on places like reddit, HN & disqus, and spammers try to find new irritating ways of getting eyeballs.
4 replies →
You do realize he copied your comment, don't you?
6 replies →
Yep, happened to me in the Inferno thread yesterday.
Do you know if there is a video for these slides?
Complete with creative tensions:
> With cinematic look, like taken with a real camera. We had some dissagreements here
> • I absolutelly wanted to avoid the CG look - the danish part of the team wanted something sharp and shinny
> • I wanted a hand-held TV camera - they wanted a sts04 like smooth lovely cameras
> • I wanted a realistic scenery - they wanted more action in the scene..
And exploiting impressionism as a compression technique:
> The idea is NOT to render perfect snow, but to draw something that evoques snow, and let the viewer’s brain to trick the viewer.
His website is full of great resources: http://iquilezles.org/www/
ok - this is pretty amazing too! https://www.youtube.com/watch?v=vga8FZzv5GE&list=PL0EpikNmjs...
Can someone TLDR how this stuff works?! I NEED TO DO IT!
That's unreal. On what kind of graphics hardware, though? Seems like it probably offloads most of the work on GPU whereas we'd have had to do most of it in software on HW weak enough that 4KB size actually mattered. And probably not achieve this demo.
>Seems like it probably offloads most of the work on GPU
It does just about everything on the GPU. All the CPU does is repeatedly render two triangles and play music: https://www.shadertoy.com/view/MdX3Rr
Edit: I'm wrong about the two triangles. From the .nfo-file:
Thanks for detailed response. I figured it mostly did GPU stuff. So, real computing necessary here is a massively-parallel chip with generic and custom hardware with a bunch of memory plus a regular core using 4KB on other end. I think a more interesting challenge would be to force use of a subset of GPU functions or memory plus tiny memory on CPU side. I don't follow demoscene close enough to know if they subset GPU's like that. Idea being making them run closer to the old Voodoo or pre-GeForce GPU's to see just how much 2D or 3D performance once could squeeze out of it.
Tricks could have long-term benefit given any emerging FOSS GPU is more likely to be like one of the older ones given complexity of new ones. I'd clone one like SGI's Octane ones they used to do movies on with mere 200MHz processors. Meanwhile, similar tricks might let one squeeze more out of the existing, embedded GPU's in use. Maybe subset a PC GPU in demoscenes like one of the smartphone GPU's. Yeah, that's got some interesting potential.
14 replies →
The 4kb restriction isn't there to make it run on weak hw, it's there to push people.
However it does irritate me that it's pregenerating the entire scene in memory. Being allowed to use 300MB of RAM doesn't strike me as very limiting.
2 replies →
I figured that. It's just that almost everythings done on the GPU for a rendering demo. That's really pushing people. ;)
9 replies →
The C64 subset of the demoscene is still going if you want it.
That's a real challenge. :) My comment to Kristine has some other details on how we might do something between that and a full GPU.
4 replies →
This is actually 4KB (kilobytes). The title lead me to believe it was 4Kb (kilobits). Still impressive though.
Back in the day demos were more impressive imo. A lot of them now use direct x or opengl. For the most part the stuff you see isn't written by hand anymore AFAIK, they just have programs to generate the actual demo. Basically they use modeling programs.
There are all kinds of demos: some of them use models, some of them do not, some are technical feats, and some are artforms. Many are both.
Models are just serialized polygon meshes. We've been using models for demos for way longer than DX/OGL have existed. They're just another tool in the box which you can use (if you want to).
Using DirectX or OpenGL nowadays is like using the CPU: it's just part of the stack. They are probably lower level than you think: using DX/OGL isn't just doing something like drawModel(model, x, y, z), it's way lower level than that.
This demo in particular is not very different from old school demos. Back in the day we had interruptions, now we have API calls, but in the end shaders are just code. Elevated uses D3D to execute them in the GPU but that's all. And the synthesizer is apparently coded in ASM.
Even if you really miss the old school platforms there are still demos produced for them, often pushing the limits of what can be done.
Check http://www.pouet.net/ for lots of impressive demos.
Ah, so yes I appreciate demos which pushes hardware to the limit or create some new effect which hasn't been seen before. I feel most demos on older hardware falls within this category then newer demos.
Additionally, please correct me if I'm wrong, older demos didn't have nice graphics apis to call. They had to create and store what's given for free by Apis these days into their binary. I think the Amiga did have some 3d stuff?
With opengl with a handful of lines you could get a spinning cube with lighting. So much more work had to be put into older demos to get to the same point. On top of that, the demo writers really had to know the hardware well, and diving into undocumented behavior. A lot more was being calculated on the cpu back then too.
I haven't been on pouet.net in a while but I will look at newer demos.
While true for large demos, the 4kb — perhaps even 64kb? — are still written very much by hand.
I hope so. I'm not involved with the scene at all, but I did read a bunch of history about the scene and watched tons of demos.
I think farbrausch was one of the major groups who did this sort of thing https://github.com/farbrausch/fr_public
Lots of groups use demotools, including conspiracy for their 64k prods. Check out the moleman documentaries for lots of interesting scene info and lore! http://vakondok.com/index_en.html
Cool idea, too bad it's not free software. It's less free than any software I've seen (explicitly saying that you can't use it for "settings where security is critical" -- something that doesn't even make sense from a software license perspective). It's like the "Good not Evil" line in the JSON license.
Check out how many of your commercial software licenses forbid running them in nuclear power plants.
I use free software, so none. :D
1 reply →