Comment by jnnnthnn
2 years ago
Thanks! I tried a few other approaches and found the LLM results were overall better (latency and cost aside). Maybe that should be an option made available to users though...
2 years ago
Thanks! I tried a few other approaches and found the LLM results were overall better (latency and cost aside). Maybe that should be an option made available to users though...
Cohere has a very cheap, fast and effective reranking API!
https://cohere.com/rerank
i think not, better results >>> better latency + cost
Maybe a combined approach beats either? Let some non-LLM reranker quickly spit out two results, and fill in the rest with the LLM.