Comment by Zak

14 hours ago

I find that decision a bit odd given that accumulating a string with a loop is also quadratic in Python if you use = instead of +=, or even if you use += when the left operand isn't provably unshared. I don't believe removing loops was seriously considered.

The footgun isn't `reduce` in particular, but failing to use `join`.

10 comments

Zak

Reply
OJFord  5 hours ago

I suppose `reduce` as built-in is the footgun because it's too easy to reach for. Now if someone doesn't know about `join` perhaps they look up how to do it because they think 'surely there's a better way than a loop without an import'.

edflsafoiewq  13 hours ago

Doesn't reduce force the accumulator to be shared though? Both the reduce and the lambda are holding onto references to acc, which defeats any "single reference" optimizations.

  • vhcr  9 hours ago

    The problem with:

        ret = ""
    for s in strings:
        ret += s

    is that it re-allocates O(n) times, even if ret is referenced only once.

    • edflsafoiewq  9 hours ago

      If the s are small the usual geometric buffer growth mitigates that. Of course you can compute the final buffer size in this case, but often you have a bunch of dynamically-generated strings of different sizes.

  • ndriscoll  12 hours ago 

      def reduce(acc, f): 
    for v in self:
      acc = f(acc, v)
    return acc

    The current acc goes out of scope each time you call f. There's no shared reference (assuming f doesn't sneak store it elsewhere, which for string combining, f should just be `return a+b`?).

  • Zak  12 hours ago

    It might - let's assume it does. My point is that it's better to use the explicit optimized method for joining strings in a performance-sensitive context than to try to meet the conditions for an implicit optimization.