Comment by amelius
9 years ago
Curious, has this method been used for solving linear systems? How would it perform e.g. against conjugate gradient?
And how would it perform for non-positive-definite systems?
9 years ago
Curious, has this method been used for solving linear systems? How would it perform e.g. against conjugate gradient?
And how would it perform for non-positive-definite systems?
Author here - yes! It can be used to solve symmetric PSD Systems, as the solution of Ax = b is also the minimizer of 0.5*x'Ax - b'x. Conjugate gradient can be seen as a special form of momentum with adaptive alphas and betas
> Conjugate gradient can be seen as a special form of momentum
Just to be clear, though, CG doesn't use the negative gradients as search directions, as steepest descent would.