Comment by allanrbo

Almost the same. The neat trick here is to ask the model to reply with just 1 letter, which is one token, rather than a long string of json. Its quicker. And just the fact that Jev made a pretty decent API for structuring your questions. And they do the massive parallelization.