Comment by exe34
3 days ago
have you considered cosmopolitan? e.g. like llamafile that works on everything up to and including toasters.
3 days ago
have you considered cosmopolitan? e.g. like llamafile that works on everything up to and including toasters.
Oh llamafile is very cool! I might add it as an option actually :) For generic exports (ie to vLLM, llamafile etc), normally finetunes end with model.save_pretrained_merged and that auto merges to 16bit safetensors which allows for further processing downstream - but I'll investigate llamafile more! (good timing since llamafile is cross platform!)