Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Unit 6: Descriptive Statistics

Descriptive Statistics summarizes and organizes data so it can be easily understood. Unlike inferential statistics, which uses samples to make predictions about populations, descriptive statistics merely describes the data at hand.

1. Measure of Dispersion

Dispersion measures the variation or spread of data in a given dataset. Common measures include:

Range

The difference between the highest and lowest values in a dataset. $$ \text{Range} = \text{Maximum Value} - \text{Minimum Value} $$

Quartile Deviation

Describes the spread of the middle 50% of data. $$ \text{Quartile Deviation (QD)} = \frac{Q_3 - Q_1}{2} $$ Where $Q_1$ is the first quartile (25th percentile) and $Q_3$ is the third quartile (75th percentile).

Mean Deviation

The average of the absolute differences between each data point and the mean (or median). $$ \text{Mean Deviation about Mean (MD)} = \frac{\sum |x_i - \bar{x}|}{N} $$

Standard Deviation and Variance

Standard deviation ($\sigma$) is the most widely used measure of dispersion. It squares the deviations to avoid negative signs, averages them (variance), and takes the square root. $$ \text{Variance } (\sigma^2) = \frac{\sum (x_i - \bar{x})^2}{N} $$ $$ \text{Standard Deviation } (\sigma) = \sqrt{\frac{\sum (x_i - \bar{x})^2}{N}} $$

2. Percentile Rank

Percentile Rank describes the percentage of values mathematically lower than a given score. $$ \text{Percentile Rank} = \left( \frac{\text{Number of values below } x}{N} \right) \times 100 $$

Example: If a student scores 85 on a test and 80 out of 100 students scored lower than 85, the student is in the 80th percentile.

3. Correlation

Correlation measures the strength and direction of a linear relationship between two variables.

Spearman’s Rank Correlation

When data ranks (instead of raw values) are evaluated, or data is non-parametric, Spearman’s rank correlation ($\rho$ or $r_s$) is used. $$ r_s = 1 - \frac{6 \sum d_i^2}{n(n^2 - 1)} $$ Where:

  • $d_i$ = difference between the ranks of corresponding variables.
  • $n$ = number of observations.

Interpretation:

  • $+1$: Perfect positive correlation.
  • $-1$: Perfect negative correlation.
  • $0$: No correlation.
Variable X Variable Y

Fig 6.1: Scatter plot demonstrating Strong Positive Correlation

Competency-Based Questions

  1. (Evaluating Dispersion): Two algorithmic models for predicting stock prices are tested over 10 days. Model A has predictions with a standard deviation of $$1.50$, while Model B has a standard deviation of $$0.20$. Both have the same mean error. Based on the measure of dispersion, which model is more reliable for trading and why?

  2. (Standard Deviation Calculation): A server farm records the following ping times (in ms) for 5 requests: $45, 50, 48, 52, 55$. Calculate the variance and standard deviation of these ping times to determine internal network consistency.

  3. (Percentile Rank Interpretation): In a data structure consisting of 5,000 algorithmic runtimes, a specific quick-sort operation took 12 seconds. If exactly 4,200 operations took shorter than 12 seconds, what is the percentile rank of this sorting operation? Interpret this result.

  4. (Spearman’s Rank Correlation): A psychologist evaluates 6 patients on two different psychological scales (A and B). Their ranks are as follows:

    • Patient 1: Rank A=1, Rank B=2
    • Patient 2: Rank A=2, Rank B=1
    • Patient 3: Rank A=3, Rank B=4
    • Patient 4: Rank A=4, Rank B=3
    • Patient 5: Rank A=5, Rank B=5
    • Patient 6: Rank A=6, Rank B=6 Calculate Spearman’s rank correlation coefficient and state if the relationship is positive or negative.

  5. (Comparing Measure of Dispersion): Differentiate between Quartile Deviation and Standard Deviation in terms of how they handle extreme outliers in a large economic dataset. Which one would you prefer if a dataset had heavy outliers?


Answers to Competency-Based Questions

1. Evaluating Dispersion: Model B is more reliable. A lower standard deviation ($ $0.20 $ vs $ $1.50 $) means Model B’s predictions are consistently closer to its mean error, indicating far less volatility and tighter precision on average.

2. Standard Deviation Calculation: Times: $45, 50, 48, 52, 55$. Mean $\bar{x} = \frac{45+50+48+52+55}{5} = \frac{250}{5} = 50$. Deviations ($x_i - \bar{x}$): $-5, 0, -2, 2, 5$. Squared deviations: $25, 0, 4, 4, 25$. Sum of squares $= 58$.

  • Variance ($\sigma^2$) $= \frac{58}{5} = 11.6$
  • Standard Deviation ($\sigma$) $= \sqrt{11.6} \approx 3.41$ ms.

3. Percentile Rank Interpretation: $$ \text{Percentile Rank} = \left( \frac{4200}{5000} \right) \times 100 = 84% $$ Interpretation: The 12-second sorting time sits at the 84th percentile. This means it took longer to run than $84%$ of all recorded operations (it is in the slower subset of occurrences).

4. Spearman’s Rank Correlation: Differences ($d_i$):

  • $d_1 = 1-2 = -1 \implies d_1^2 = 1$
  • $d_2 = 2-1 = 1 \implies d_2^2 = 1$
  • $d_3 = 3-4 = -1 \implies d_3^2 = 1$
  • $d_4 = 4-3 = 1 \implies d_4^2 = 1$
  • $d_5 = 5-5 = 0 \implies d_5^2 = 0$
  • $d_6 = 6-6 = 0 \implies d_6^2 = 0$ Sum $\sum d_i^2 = 4$. $n = 6$. $$ r_s = 1 - \frac{6 \times 4}{6(6^2 - 1)} = 1 - \frac{24}{6 \times 35} = 1 - \frac{24}{210} \approx 1 - 0.114 = 0.886 $$ Since $r_s > 0$, the scales show a Strong Positive Correlation.

5. Comparing Measure of Dispersion: Standard Deviation calculates squared distances from the mean for all points, meaning extreme outliers disproportionately inflate the result. Quartile Deviation solely measures the spread of the middle 50% of the data ($Q_3$ to $Q_1$), effectively ignoring extreme high or low tails. If a dataset has heavy outliers, Quartile Deviation is generally preferred as it is a more robust indicator of centralized spread.