Experiments allow you to talk about cause and effect. Without them, all you have is correlation. What is correlation?

IT’S NOT CAUSATION. (!!!)

Sure, you’ve probably already heard us statisticians yelling that at you. But what *is* correlation? It’s when the variables in a dataset look like they’re moving together in some way.

Two variables X and Y are correlated if they seem to be moving together in some way.

For example, *“when X is higher, Y tends to be higher”* (this is called

Thanks, Wikipedia.

If you’re looking for the formula for (population) correlation, your friend Wikipedia has everything you need. But if you wanted that, why didn’t you go there straight away? Why are you here? Ah, you want the intuitive explanation? Cool. Here’s a hill:

On the left, height and (left-to-right) distance are positively correlated. When one goes up, so does the other. On the right, height and distance are negatively correlated.

When most people hear the word correlation, they tend to think of _perfect _** linear correlation**: taking a horizontal step (X) to the right on the hill above gets you the same change in altitude (Y) everywhere on the same slope. As long as you’re going up from left to right (positive correlation), there are no surprise jagged/curved bits.

Bear in mind that **going up is positive** only if you’re hiking left-to-right, same way as you read English. If you approach hills from the right, statisticians won’t know what to do with you. I suppose what statisticians are trying to tell you is never to approach a hike from the right. That will only confuse us.

But if you hike properly, then “up” is “positive.”

In reality, this hill is not perfect, so the correlation magnitude between height and distance will be less than 100%. (You’ll pop a +/- sign in front depending on whether we’re going up or down, so correlation lives between -1 and 1. That’s because its formula (pasted from Wikipedia above) ** divides by standard deviation**, thereby

#technology #data-science #statistics #data analysis

1.10 GEEK