Comment by nextaccountic

2 days ago

> since getting data then doing something with it is a hidden branch if you already have the data

You mean that the naive, simpler code, despite not having an if, has a "branch" on the microarchitectural state? (which is like.. if we have this already in cache, do something. if not, do something else)

    // Find the optimal encoding for each symbol.
    // Chunk boundaries are located where encodings change.
    uint8_t encoding[n_symbols];
    uint8_t j = 0; // always start with encoding 0 for simplicity
    for (int i = 0; i < n_symbols; i++) {
        j = next_j[i][j];
        encoding[i] = j;
    }

It's not so much a branch, but a dependency between subsequent iterations of the loop, which means execution of each iteration has to wait for the result from the previous iteration, slowing things down. But if most of the time j == next_j[i][j], that dependency is often unnecessary. So the branch allows the system to predict that it can be parallelized. But that's only really worth it if the branch condition is false most of the time. Whenever it turns out to be true, the CPU will have to backtrack and redo that work.