Comment by The_rationalist

4 years ago

Why woud someone use pearson or smearson coefficients vs this new one?

If the relationship is linear, I'm guessing a test based on Pearson has greater power.

  • Seems likely. The presented coefficient only looks at the ordering of the X-values, and how it relates to the ordering of the Y-values. All other information is thrown away. That's how it can be so general, but it should come at the expense of power.

    • Ah that makes sense intuitively. I was confused how non linear correlation could be detected without knowing anything about the function itself.

  • power and interpretability.

    Assuming a linear relationship, if you know the correlation coefficient, you can predict unobserved values of y based on a known x with good accuracy.

    y = ax + b + error

    where strong correlation means error is small.

Pearson is for affine/linear+intercept relationships. Spearman is for monotonic relationships.