Comment by themafia

> A simple linear combination of every weight did not degrade the performance of the model, but enhanced it.

Which could be a signal that your "performance" was so abysmal in the first place that even randomly applied training methods can't make it _worse_.