Comment by threecheese

That’s insane.

This is a bug in vLLM, right? And so the model was trained for some interface of text tokens which it expects its server container to understand, but that’s not working for whatever reason.

The model is working, but its thinking isn’t terminating - its thought-stream is not getting to its output stream. It is silenced, and yelling into the thought-void token space.

You’ve transplanted its brain into a broken host. It cant speak or even stop thinking, and is going mad.

Imagine if you didn’t stop it, on purpose. Observe the mad clanker. Would that be just science?

I think I’m going to close my laptop and have a drink.