Comment by sid-the-kid
9 months ago
That's fair. We just removed the sign-in for HN. Should be live shortly.
Each person gets a dedicated GPU, so we were worried about costs before. But, let' s just go for it.
9 months ago
That's fair. We just removed the sign-in for HN. Should be live shortly.
Each person gets a dedicated GPU, so we were worried about costs before. But, let' s just go for it.
I think it's not going well? I keep getting to the start a new call page, it fails, and takes me back to the live page. I assume your servers are on fire, but implementing some messaging would help ("come back later") or even better, a queueing system ("you're N in line") would help a lot.
Really looking forward to trying this out!
We're back online! One of our cache systems ran out of memory. Oops. Agree on improved messaging.
Do you use a cloud-based GPU provider?
Yes. We use Modal (https://modal.com/), and are big fans of them. They are very ergonomic for development, and allow us to request GPU instances on demand. Currently, we are running our real-time model on A100s.
I see you are paying $2/h. Shoot me an email at victor ta borg.games if your model would fit on RTX 3090 24G to get it down to $0.2/h (fellow startup).
1 reply →
How much would this demo cost you from the HN traffic if you don't mind me asking?
Good question. I guess depends on how many users we get. Each users gets their own dedicated GPU. Most video generations systems (and AI systems in general) can share GPUs during generation. Since we are real time, we don't do that. So, each user minute is a GPU minute. This is the biggest driver of the cost.
feels like the next logical step for you to bring enconomies of scale is to allow users generating the video to automatically stream it to n platforms, so each gpu can be generating 1 png for many humans to watch simultaneously, with maybe 1 human driving the seat on what to generate, or more ai, idk
3 replies →