Comment by wrs

21 hours ago

The entire purpose of semver is to give you a way to resolve that conundrum. New major version = assume it's incompatible.

I mean, it may not actually work, but that's what it's for.

14 comments

wrs

Reply
kibwen  20 hours ago

The use or adherence to semver isn't the problem here. As you say, if a package follows semver, it's easy enough for the package managers to automatically update to newer compatible versions. The problem is when you want to have two different incompatible versions of the same package `foo` in the same program, because then you have to figure out what `import foo` means. You might say "just don't do that", but that package could be an indirect dependency of several of your direct dependencies. Some languages handle this natively, e.g. in Rust it just works if you have multiple versions of the same library in different parts of your dependency tree (and you'll get a compilation error if you try to pass a type from one version into a function of an incompatible version). But Python does not handle this use case very well.

  • skeledrew  19 hours ago

    > Python does not handle this use case very well

    I solved this issue a few months ago. Created a tool that essentially allows the use of multiple envs at once, with their own versions of packages at any level.

    • mroche  15 hours ago

      This sounds... not possible for the core problem of how Python handles dependency resolution during the life of an application? How are you setting things up so that the following scenario is valid?

          program
    ├── dependency_a
    │   └── dependency_c (1.0.0)
    └── dependency_b
        └── dependency_c (2.0.0)

      Otherwise, you've created a magic layer hack to enable multi-version dependency chains in a mono-version dependency chain language.

      1 reply →

    • arcatek  14 hours ago

      Curious how you did this; I looked into that couple of months ago but even with custom hooks the Python injection points seemed to limited due to the internal resolution cache.

  • __mharrison__  18 hours ago

    I'm curious. Do you have real world examples of when you want to do this?

    • gbear605  17 hours ago

      This is Java, but recently I had a case where one library depended on a version of an Apache Commons library, and another library depended on a different version of the same Apache Commons library, and neither version worked with both libraries. In my case, I was able to upgrade one of them to a newer version so that I could use just one Apache Commons version, but I got lucky there.

    • deathanatos  16 hours ago

      The monolith I work on has this dependency chain:

        monolith -> openai
  monolith -> langchain-openai -> openai

      openai, thus, is both a direct and indirect dependency. langchain-openai recently had a vulnerability, and the patch fix is only after a major upgrade to openai. Thus, to upgrade langchain-openai here, I also need to upgrade monolith's use of openai. (From v1 to v2.)

saghm  20 hours ago

> The entire purpose of semver is to give you a way to resolve that conundrum. New major version = assume it's incompatible.

I'm not sure I'd agree with that characterization. The point of semver is that you can assume that certain types of bumps won't include certain types of changes, not that you assume that the types of changes that can happen in a type of bump will happen. A major version bump not breaking anything is completely valid semver, and breaking one function (which plenty of users might not use, or might use in a way that doesn't get broken) in an API with thousands is still technically only valid in a major version bump (outside of specific exceptions like the major version being 0).

It's a subtle difference, and I'm optimistic that it's something you understand, but misunderstandings of semver seem so common that I can't help but feel like precision when discussing it is important. I've encountered so many smart people who misunderstand aspects of semver (and not just minutia like "what constraints are there on tags after the version numbers"), and almost all of them seemed to have stemmed from people learning a few of basic tenets of it and inferring how to fill in the large gaps in a way that isn't at all how its specified. The semver specification is pretty clear in my opinion even about where some of the edge cases someone less informed might assume, and if we don't agree on that as the definition, I don't know how we avoid the (completely realistic) scenario where everyone in the room has an idea of what "semver" means that's maybe 80% compatible with the spec, but the 80% is different for each of them, and trying to resolve disagreements when people don't agree about what words mean is really hard.

  • deathanatos  16 hours ago

    > not that you assume that the types of changes that can happen in a type of bump will happen

    … an assumption that something happened is not a definitive statement that it did happen, only that we're assuming it did, because it could happen, or perhaps here, that because the major was bumped, that it is legal, according to the contract given, for it to have possibly happened in a way that we depended on. They're not saying that it will/must; "assume a major version is incompatible" is not at odds with what you've written.

    • saghm  15 hours ago

      You and I have a very different version of what the equals sign means. "This is a reasonable action to take in this scenario" is not what I understand as equivalence.

      1 reply →

chippiewill  20 hours ago

There isn't a good way to know if a given package is using semver though.

There's a lot of packages in the Python ecosystem that use time based versioning rather than semver (literally `year.minor`) and closed ranges cause untold problems.

KlayLay  21 hours ago

Semantic versioning is about versioning individual dependencies, no? The issue here seems to be about transitive dependencies, where different versions of the same package is used by multiple packages which depend on it.

uv's default being to always select the latest version seems to be what Clojure's tools.deps does.