 # Statistics

### Mean

One of the most basic properties of a data set is it’s mean. Mean is a measure of the Central Tendency of data. In layman terms, if you were to pick one point in a dataset that is representative of the entire dataset, it would be the mean. Also called “average” in common vocabulary, calculating the mean is really simple. Just add up all the values in a particular variable of a dataset and divide it by the total number of values in it.

For example, if there are 5 Uber drivers with 5 different ratings, what is their mean ?

```from statistics import mean

ratings = [4.5, 3.9, 4.6, 4.8, 3.9]

mean(ratings)
```
```4.34
```

### Median

Mean does not always represent the average of the data. For normal distributions it typically does. However, for many other data distributions, mean does not represent the average. For example, take the US income distribution data. Since the number of housholds is huge, the data has been bucketed into income brackets for easy analysis.

```income_bracket    households
10-15k            5700000
15-20k            5620000
20-25k            5930000
25-30k            5500000
30-35k            5780000
35-40k            5340000
40-45k            5380000
45-50k            4730000
50-60k            9210000
60-75k           11900000
75-100k          14700000
100-125k         10300000
125-150k          6360000
150-200k          6920000
200k-plus         7600000

```
```from statistics import mean, median

income = [5700000,5620000, 5930000, 5500000, 5780000, 5340000, 5380000,
4730000, 9210000, 11900000, 14700000, 10300000, 6360000, 6920000,7600000 ]

print ( "mean =", mean(income) )
print ( "median =", median(income) )
```
```mean = 7398000
median = 5930000

```

As you can see, depending on the data distribution, mean and median could be totally different from each other.

### Mode

Mode is just the highest value in the dataset. If the dataset follows a gaussian distribution, it is the peak of the histogram. Think of mode of a dataset as the most commonly occuring number.

```from statistics import mode

ages = [12,13,14,11,12,13,15,10,13 ]

mode(ages)

```
```13
```

### Mean vs Median vs Mode

The reason why these terms exist is that, there is no one way to define the Central Tendency of a dataset. It depends on the nature of the distribution that the dataset conforms to. For example, if the dataset is a normal distribution, then most of the time the mean, median and mode are pretty close together. The relationship is beautifully visualized in this wikipedia diagram.

For a normal distribution, the 3 parameters ( mean, median, mode ) are pretty close together. For a skewed distribution, they are pretty staggered.

### Standard Deviation

While mean represents the “Central” or “Average” value of a dataset, variance represents how spread out the data is. For example, look at the 3 histograms below. They represent histograms of GRE scores among 3 different groups.

All these 3 graphs represent 10K rows of GRE scores from 3 different groups. All 3 of them have the same mean – 300. However, they are different, right ? What I want you to focus on is the shape of the distribution, not the height. Specifically, look at the x-axis. In the first plot, the data is focussed, pretty much around the average(300) mark. In the second plot, it is a bit spread out ( hence the reduction in size ) and the third plot is pretty spread out.

The green line represents the mean and the red line represents the Standard Deviation or σ ( represented by the Greek symbol sigma ). It is a measure of how spread out the distribution is. The more spread out the distribution is, the more flatter the bell curve is.

You can also think of Standard Deviation as a measure of uncertianity in Data Science. Imagine a drug trail working on reducing blood pressure. Say, on an average 3 different trails produce an average reduction of 30 points. However, trail 1 has very low standard deviation. Obviously, you would want to go with the first drug – because the third drug is more uncertain about the result.

```from statistics import stdev

income = [5700000,5620000, 5930000, 5500000, 5780000, 5340000, 5380000,
4730000, 9210000, 11900000, 14700000, 10300000, 6360000, 6920000,7600000 ]

stdev(income)

```
```2884498.470890811
```

### Variance

Standard Deviation is actually a derivative of variance. So, variance is calculated first. However, to illustrate that variance or standard deviation represents spread on the plot, we have learnt standard deviation first. However, we have to understand how variance is calculated, because this is the basis for calculating standard deviation .

Let’s calculate variance of a small dataset in excel to understand the process better.

variance_calculation

### What does Standard Deviation represent

Now that we know the basic definitions of Variance and Standard Deviation, let’s look at what it really represents.

### How far from the mean

Wikipedia has a nice picture to show this in the context of Gaussian distribution.