Comment by jwells89
3 years ago
The way these ML models and humans operate are indeed quite different.
Humans work by abstracting concepts in what they see, even when looking at the work of others. Even individuals with photographic memories mentally abstract things like lighting, body kinetics, musculature, color theory, etc and produce new work based on those abstractions rather than directly copying original work (unless the artist is intentionally plagiarizing). As a result, all new works produced by humans will have a certain degree of originality to them, regardless of influences due to differences in perception, mental abstraction processes, and life experiences among other factors. Furthermore, humans can produce art without any external instruction or input… give a 5 year old that’s never been exposed to art and hasn’t been shown how to make art a box of crayons and it’s a matter of time before they start drawing.
ML models are closer to highly advanced collage makers that take known images and blend them together in a way that’s convincing at first glance, which is why it’s not uncommon to see elements lifted directly from training data in the images they produce. They do not abstract the same way and by definition cannot produce anything that’s not a blend of training data. Give them no data and they cannot produce anything.
It’s absolutely erroneous to compare them to humans, and I believe it will continue to be so until ML models evolve into something closer to AGI which can e.g. produce stylized work with nothing but photographic input that it’s gathered in a robot body and artistic experimentation.
You're wrong in your concept of how AI/ML works. Even trivial 1980's neural networks generalize, it's the whole point of AI/ML or you'd just have a lookup-table (or, as you put it, something that copies and pastes images together).
I've seen "infographics" spread by anti-AI people (or just attention-seekers) on Twitter that tries to "explain" that AI image generators blend together existing images, which is simply not true..
It is however the case that different AI models (and the brain) generalize a bit differently. That is probably the case between different humans too. Not the least with for example like you say those with photographic memory, autists etc.
What you call creativity in humans is just noise in combination with a boatload of exposure to multi-modal training data. Both aspects are already in the modern diffusion models. I would however ascribe a big edge in humans to what you normally call "the creative process" which can be much richer, like a process where you figure out what you lack to produce a work, go out and learn something new and specific, talk with your peers, listen to more noise.. stuff like that seems (currently) more difficult for AIs, though I guess plugins that do more iterative stuff like chatgpt's new plugins will appear in media generators as well eventually..
ML generalization and human abstraction are very different beasts.
For example, a human artist would have an understanding of how line weight factors into stylization and why it looks the way it does and be able to accurately apply these concepts to drawings of things they’ve never seen in that style (or even seen at all, if it’s of something imaginary).
The best an ML model can do is mimic examples of line art in the given style within its training data, the product of which will contain errors due to not understanding the underlying principles, especially if you ask it to draw something it hasn’t seen in the style you’re asking for. This is why generative AI needs such vast volumes of data to work well; it’s going to falter in cases not well covered by the data. It’s not learning concepts, only statistical probabilities.
I know what you're saying, and for sure existing models can be difficult to force into the really weird corners of the distributions (or go outside the distributions). The text interfaces are partially to blame for this though, you can take the images into Gimp and do some crude naive modifications and bring them back and the model will usually happily complete the "out-of-distribution" ideas. The Stable Diffusion toolboxes have evolved far away from the original simple text2image interfaces that midjourney and dalle use.
The models will generalize (because that's the most efficient way of storing concepts) and you can make an argument that that means they understand a concept. Claiming "it's not learning concepts, only statistical probabilities" trivialises what a modern neural network with billions of parameters and dozens of layers is capable of doing. If a model learns how to put a concept like line width 5, 10 and 15 pixels into a continuous internal latent property, you can probably go outside this at inference at least partially.
I would argue that improving this is at this point more about engineering and less about some underlying unreconcilable differences. At the very least we learn a lot about what exactly generalization and learning means.
> The way these ML models and humans operate are indeed quite different.
Given that there is no comprehensive understanding of how human learning works, let alone how humans operate on and integrate on what they learned in a wider context...how do you know?
> Humans work by abstracting concepts in what they see
Newsflash: AI models do the same thing. That's the basis of generalization.
> ML models are closer to highly advanced collage makers that take known images and blend them together
Wrong. That's not even remotely how U-Net based diffusion models work. If you disagree, then please do show me where exactly the source images from where the "collage maker" takes the parts to "blend" together are stored. I think you'll find that image datasets on the scale of LAION will not quite fit into checkpoint files of about 2GB in size (pruned SD1.5 checkpoint in safetensors format).
Beautifully put. I wish this nuance was more widely understood in the current AI debate.