Comment by colah3
9 years ago
Hi! This article is about momentum in the mathematical field of optimization. Acceleration also refers to a phenomenon in optimization. While there are deep connections to their physical analogues, they aren't aren't quite the same thing.
If you want to make your analogy work, the momentum algorithm adds mass to an object. In terms of literal acceleration at any point in time it is neutral, but the added mass causes you to get through difficult areas much faster.
That said, much of this article is moving away from that physical analogy. :)
The use of the term "accelerator" is misleading as it's introduced during the physical metaphor. You should consider editing that sentence to clarify your meaning:
The added inertia smooths out variation in velocity, dampening oscillations and causing us to barrel through narrow valleys, small humps and local minima. Keeping our speed steadier, we arrive at the global optimum faster.
Note the alliteration :-)
> momentum accelerates convergence
Here it's more clear that momentum is accelerating convergence, not the "heavy ball" itself.
> inertia acts both as a smoother and an accelerator
On the second read, it's more clearly a contradiction. Speed can't be both held steadier and accelerated simultaneously. If you meant that momentum alternately smooths and accelerates, then it's even more strange. For that behavior, some sort of motor would be a more appropriate metaphor.
To the author: I found the sentences quoted above quite clear. Please do not change them. They helped me rapidly comprehend what the article was going to be about.
You don't find the idea that increased momentum causes increased speed a bit strange?
1 reply →