{"id":46821062,"user":"in-silico","time":1769753110,"time_ago":"10 days ago","type":"comment","content":"

How would it do what it does without those things?","comments":[{"id":46821188,"user":"slashdave","time":1769754751,"time_ago":"10 days ago","type":"comment","content":"

Like all these models work, by simple interpolation.","comments":[{"id":46822288,"user":"in-silico","time":1769765173,"time_ago":"10 days ago","type":"comment","content":"

But how does it interpolate?","comments":[{"id":46828971,"user":"slashdave","time":1769802473,"time_ago":"10 days ago","type":"comment","content":"

Pixel by pixel, time-slice by time-slice, in a 2D+T convolution. You provide enough examples of videos of changing point-of-view, and the model reproduces what it is given.","comments":[{"id":46831775,"user":"in-silico","time":1769818281,"time_ago":"9 days ago","type":"comment","content":"

Yes, it reproduces what it is given by modelling the rules of physics, geometry, etc.

For example, image generators like stable diffusion carry strong representations of depth and geometry, such that performant depth estimation models can be built out of them with minimal retraining. This continues to be true for video generation models.

Early work on the subject: https://arxiv.org/pdf/2409.09144","comments":[{"id":46882416,"user":"slashdave","time":1770188197,"time_ago":"5 days ago","type":"comment","content":"

What? No, it does no such thing. Study the architecture. Pixels in. Pixels out.","comments":[],"comments_count":0,"level":4,"url":"item?id=46882416"}],"comments_count":1,"level":3,"url":"item?id=46831775"}],"comments_count":2,"level":2,"url":"item?id=46828971"}],"comments_count":3,"level":1,"url":"item?id=46822288"}],"comments_count":4,"level":0,"url":"item?id=46821188"}],"comments_count":5,"url":"item?id=46821062"}

Comment by in-silico