Comment by chandureddyvari
1 year ago
Neat idea! It would be helpful to have LLMs ranked from best to worst for a given GPU. Few other improvements I can think of:
- Use natural language for telling offloading requirements.
- Just year of the LLM launch of HF url can help if it’s an outdated LLM or a cutting edge LLM.
- VLMs/Embedding models are missing?
Hey - thanks for the reply.
Do you mean remove the JSON thing and just summarise the offloading requirements?
Great Idea - I will try add this tonight.
Yeah I just have text generation models ATM as that is by far where the most interest is. I will look at adding other model types in another type, but wouldn't be until the weekend that I do that.