← Back to context

Comment by danielhanchen

2 days ago

Oh yes I was working on providing binaries together with pip - currently we're relying on pyproject.toml, but once we utilize setup.py (I think), using binaries gets much simpler

I'm still working on it, but sadly I'm not a packaging person so progress has been nearly zero :(

8 comments

danielhanchen

Reply
ffsm8  2 days ago

I think you misunderstood rfoos suggestion slightly.

From how I interpreted it, he meant you could create a new python package, this would effectively be the binary you need.

In your current package, you could depend on the new one, and through that - pull in the binary.

This would let you easily decouple your package from the binary,too - so it'd be easy to update the binary to latest even without pushing a new version of your original package

I've maintained release pipelines before and handled packaging in a previous job, but I'm not particularly into the python ecosystem, so take this with a grain of salt: an approach would be

Pip Packages :

    * Unsloth: current package, prefers using unsloth-llama, and uses path llama-cpp as fallback (with error msg as final fallback if neither exist, promoting install for unsloth-llama)
    * Unsloth-llama: new package which only bundles the llama cpp binary

rat9988  2 days ago

Don't worry. Don't let the rednecks screaming here affect you. As for one, I'm happy that you have automated this part and sad to see it is going away. People will always complain. It might be reasonable feedback worth acting upon. Don't let their tone distract you though. Some of them are just angry all day.

  • danielhanchen  2 days ago

    Thanks - hopefully the compromise solution ie python input asking for user permissions works ok?

    • rpdillon  2 days ago

      As a guy that would naturally be in the camp of "installing packages is never okay", I also live in the more practical world where people want things to work. I think the compromise you're suggesting is a pretty good one. I think the highest quality implementation here would be.

      Try to find prebuilt and download.

      See if you can compile from source if a compiler is installed.

      If no compiler: prompt to install via sudo apt and explaining why, also give option to abort and have the user install a compiler themselves.

      This isn't perfect, but limits the cases where prompting is necessary.

      1 reply →