Comment by ljosifov

I get what you are talking about. My gripe with that is - yeah, would be indeed great if we could at some point get the structure to such deep level, as to write down pen on paper on one page, a string of maths symbols that is a good enough description. However - it's possible that for many things, that's not possible. Suspect maybe not possible in e.g. biology. Possible that the great success of physics of the 20-th century over-indulged us. So our expectations are out of kilter with realities of our world.

Fwiw I personally describe than as white, not black boxes. For we know, and can trace back every single bit of the output, back to the input. That does not help us as much as we'd like though. When drilling down into "why did the model answer wrongly 1, and not rightly 2", it comes down to "well, it added one trillion small numbers, and the sum came close to 1, but didn't reach 2". Which is unsatisfactory, and your "understanding" v.s. "comprehension" delineates that nicely.

Maybe more productive to think of them more "artefacts", less "mechanical contraptions". We shape them in many ways, but we are not in complete control of their making. We don't make them explicitly with out hands: we make a maker algorithm, and that algorithm then makes them. Or even "biological", grown artefacts. Given we don't control the end result fully. Yes we know and apply the algorithm that builds them, but we don't know the end result before hand, the final set of weights. Unlike say when we are making a coffee machine - we know all the parts to a millimetre in advance, have it all worked out pre-planned, before embarking on the making of the machine.