Comment by dannyw

Your inference server is just an inference server that spits out tokens. You can use the same harness, etc. you’re basically just changing the API base URL.