Comment by faangguyindia

I've built something similar, i am hosting it.

You can test Jev like model at 26B parameter count here (built few weeks ago): https://gambler-relay-us-west1.leo-fish.ts.net/demo (might not stay up for long)

Typesafe compatible API

This is just running on old hardware.