Comment by mikewarot

Right now I'm using Visual Studio code with ChatCPT5 (preview) to write a program that is training a deep neural network to generate a single photo, to see just how small a network can do it.

The network (at present) has 2 inputs(x,y), and 3 outputs (r,g,b) and is training to minimize loss. With 10 hidden layers of 10 elements per layer, it's oddly impressionistic so far.

I've tweaked things so that the batch size is the whole image, and I'm using FP64 weights. I have a clue about what the python code is doing, but not a huge clue. It reminds be of the things that I did back when I took the free AI classes from Stanford about a decade ago.

The inspiration that kicked this off was this video by Welch labs[1] "Why Deep Learning Works Unreasonably Well". Normally I'd just ponder such things, but thanks to LLMs, I can actually see just what works and doesn't.

I wonder just how detailed an image I can get from weights that are on the order of 50k or less. I'm willing to throw hundreds of CPU hours at it. Seeing the images as the training progresses is fascinating.

[1] https://www.youtube.com/watch?v=qx7hirqgfuU