Comment by elixirnogood
7 months ago
Do you have any concerns around scaling? I like LikeKit stack, but if not mistaken their agent architecture is based on multiprocessing (one os process per 'session'/'conversation') which doesn't sound very scalable. Btw, great demo, this is a cool technical problem to solve. I've spend a couple of months in this space (using a similar stack) and know for a fact that's not easy.
Thanks, there are certainly a lot of fun and challenging problems to take on in the space. On scaling, the agent architecture isn't limited to one machine, so you can also autoscale your machines. It's essentially python's Celery if you've tried that. It gets more tricky when you require GPUs though!