{"id":32644443,"user":"aaaaaaaaaaab","time":1661812591,"time_ago":"4 years ago","type":"comment","content":"

Love this. OpenAI are livid. :^)","comments":[{"id":32644558,"user":"enchiridion","time":1661813416,"time_ago":"4 years ago","type":"comment","content":"

Why?","comments":[{"id":32644632,"user":"aaaaaaaaaaab","time":1661814001,"time_ago":"4 years ago","type":"comment","content":"

Because they no longer control the narrative.","comments":[{"id":32644701,"user":"Gigachad","time":1661814556,"time_ago":"4 years ago","type":"comment","content":"

It does ruin the illusion that you need crazy million dollar servers to run the model and the world would fall in to chaos if the public got their hands on these models.","comments":[{"id":32645131,"user":"optimalsolver","time":1661817993,"time_ago":"4 years ago","type":"comment","content":"

To be fair, the world only just got their hands on this (and by world I mean people with decent hardware), so too soon to say what the ramifications will be.","comments":[{"id":32645576,"user":"bogwog","time":1661822008,"time_ago":"4 years ago","type":"comment","content":"

Also to be fair, the job "AI ethicist" probably didn't exist as a real thing until a few years ago. So the people in those roles over at OpenAI likely have no idea what they're doing.","comments":[],"comments_count":0,"level":4,"url":"item?id=32645576"}],"comments_count":1,"level":3,"url":"item?id=32645131"},{"id":32645685,"user":"adultSwim","time":1661822958,"time_ago":"4 years ago","type":"comment","content":"

We wouldn't be able to run it ourselves if they hadn't trained it on 4000 GPUs for a month.","comments":[{"id":32645807,"user":"xtreme","time":1661824469,"time_ago":"4 years ago","type":"comment","content":"

The cost of training is actually quite a bit less. \nEmad, the creator of SD stated this on Twitter:

"We actually used 256 A100s for this per the model card, 150k hours in total so at market price $600k"","comments":[{"id":32646656,"user":"astrange","time":1661833998,"time_ago":"4 years ago","type":"comment","content":"

Even if it was hard to train, you could make your own by fine-tuning a larger model for much cheaper.

That's called "base models". (or "foundation models" if you're Stanford trying to co-opt it)","comments":[{"id":32648855,"user":"DoctorOetker","time":1661857284,"time_ago":"4 years ago","type":"comment","content":"

suppose one has an idea for a different architecture / functional form etc, assuming the receiving model is substantially smaller so that the dominant computational cost is in the SD model, how long would effective knowledge distillation take on say a CPU?","comments":[{"id":32658801,"user":"astrange","time":1661924667,"time_ago":"4 years ago","type":"comment","content":"

That’s called teacher-student learning. It could still take weeks on a single machine easily, but renting more GPU time or getting free credits from somewhere is perfectly plausible.","comments":[],"comments_count":0,"level":7,"url":"item?id=32658801"}],"comments_count":1,"level":6,"url":"item?id=32648855"}],"comments_count":2,"level":5,"url":"item?id=32646656"},{"id":32646158,"user":"boppo1","time":1661828271,"time_ago":"4 years ago","type":"comment","content":"

Christ, so what happens when google throws a cheeky 10 million at a model?","comments":[],"comments_count":0,"level":5,"url":"item?id=32646158"}],"comments_count":4,"level":4,"url":"item?id=32645807"}],"comments_count":5,"level":3,"url":"item?id=32645685"}],"comments_count":8,"level":2,"url":"item?id=32644701"}],"comments_count":9,"level":1,"url":"item?id=32644632"}],"comments_count":10,"level":0,"url":"item?id=32644558"}],"comments_count":11,"url":"item?id=32644443"}

Comment by aaaaaaaaaaab