Statistics collects and analyzes data to draw numerical conclusions, distinguishes population from sample, and uses central tendency and dispersion measures to describe distributions.
Statistics is the science of collecting, organizing, presenting, and then analyzing data to draw conclusions based on numbers rather than impressions. The lesson first distinguishes between population, which consists of all elements under study, and sample, which is a part selected to represent it. The reason for this distinction is that the symbols differ: population variance \(\sigma^2\) and sample variance \(S^2\).
To describe any data set, we need two types of measures together: a measure that determines central tendency, and a measure that determines dispersion; the average alone is not sufficient, because two sets may have the same average yet differ enormously in the spread of their values.
First: Measures of Central Tendency
Arithmetic mean: \(\bar{x} = \frac{\sum x_i}{n}\), that is, the sum of the values divided by their number. Its fame stems from using all values in the calculation, hence its greatest merit and greatest drawback: it is affected by outliers, so its value may rise or fall greatly due to a single extreme value. For example, if we add the value 100 to the set {2, 4, 6}, the mean rises from 4 to 28, illustrating its extreme sensitivity to outliers.
Median: the value that lies in the middle of the data after arranging them in ascending order; if the number of values is even, it is the average of the two middle values. It is not affected by outliers, therefore it is preferred for use with asymmetric data or data containing extreme values. For example, in the set {1, 2, 3, 100}, the median is \(\frac{2+3}{2}=2.5\), while the mean is 26.5, demonstrating the median's resistance to outliers.
Mode: The value that occurs most frequently. A data set may have more than one mode, or no mode at all if all values occur with equal frequency. For example, in the set {5, 5, 7, 7, 9}, the modes are 5 and 7, while in the set {1, 2, 3, 4}, there is no mode because each value occurs only once.
Measure
Idea
Is it affected by outliers?
Arithmetic mean
Sum of values divided by their count
Yes
Median
The middle value after ordering
No
Mode
Most frequent value
No
Second: Measures of dispersion
These measures reveal how far the values deviate from their center; the larger they are, the more scattered the data:
Range: \(R = \max - \min\); that is, the difference between the maximum and minimum values. It is easy to compute but depends only on two values, so it is affected by any outlier. For example, in the set {3, 5, 7, 9}, the range is 6, but if we add 100, the range becomes 97, illustrating its excessive sensitivity to extreme values.
Variance: The average of the squared deviations from the mean. For a population, it is written as \(\sigma^2 = \frac{\sum (x_i - \mu)^2}{N}\); for a sample, \(S^2 = \frac{\sum (x_i - \bar{x})^2}{n-1}\). The reason for dividing by \(n-1\) in the sample case is that it makes \(S^2\) an unbiased estimator of the population variance; dividing by \(n\) produces a downward-biased estimate, especially in small samples, whereas dividing by \(n-1\) corrects this bias using degrees of freedom.
Standard deviation: The square root of the variance \(\sigma = \sqrt{\sigma^2}\), and it is preferred over variance because it returns to the original unit of measurement; for example, if the data are measured in years, the variance is in square years, while the standard deviation is in years, which facilitates interpretation and comparison.
Interquartile range: \(IQR = Q_3 - Q_1\), which is the difference between the third and first quartiles, i.e., the spread of the middle half of the data; it is unaffected by outliers, and extreme outliers are defined by the bounds \(Q_1 - 1.5\,IQR\) and \(Q_3 + 1.5\,IQR\). For example, if \(Q_1=10\) and \(Q_3=20\), then \(IQR=10\), and the extreme outliers are values less than \(10-1.5\times10=-5\) or greater than \(20+1.5\times10=35\).
Worked example (1)
Data: \(2, 4, 4, 6, 9\).
Number of values \(n = 5\) and their sum \(25\), so \(\bar{x} = \frac{25}{5} = 5\).
Median: the values are already ordered, so the third value is \(4\).
Standard deviation: \(S = \sqrt{7} \approx 2.65\).
Worked Example (2): Effect of an outlier
Data \(10, 12, 14, 16, 18\): sum \(70\) so \(\bar{x} = 14\), median \(14\), range \(18 - 10 = 8\). Then we add the outlier \(90\).
Mean: \(\bar{x} = \frac{160}{6} \approx 26.7\), a very large change.
Median: the two middle values are \(14\) and \(16\), so median \(\frac{14 + 16}{2} = 15\), a slight change.
Range: \(90 - 10 = 80\), a tenfold increase.
Result: The outlier strongly affects the mean and range but leaves the median nearly unchanged, which is the practical criterion for choosing the appropriate measure.
Common Errors
Calculating the median before sorting the data in ascending order, and with percentages the answer differs completely; the correct approach is to sort the data first, then find the middle value or the average of the two middle values.
When calculating sample variance, division by \(n\) instead of \(n-1\) is incorrect; the correct approach is to use \(n-1\) to obtain an unbiased estimator of the population variance.
Confusing \(\sigma\) for the population and \(S\) for the sample in symbols and calculations; the correct usage is \(\sigma\) for the population and \(S\) for the sample, with attention to the corresponding formulas.
Judging dispersion solely by the range, despite it depending on only two values rather than all data; the correct approach is to use more comprehensive measures such as variance or interquartile range when a full representation of dispersion is needed.
Writing symbols as words or using an asterisk for multiplication; the correct notation is \(\sigma\) and \(\times\), not sigma and The point; for example, one writes \(\sigma^2\) and not sigma squared, and \(a \times b\) and not a*b.
Tips for the CSCA Exam
Read the question carefully: 'mean' and 'median' differ in most outlier questions, and choosing the wrong measure loses the full mark; therefore, determine whether the question asks for a measure sensitive to outliers or resistant to them before calculating.
If the question asks for 'a measure not affected by extreme values', the answer is the median or interquartile range, not the mean; for example, in questions comparing groups that contain extreme values, choose the median or IQR.
Remember the relationship \(\sigma = \sqrt{\sigma^2}\) well, as many questions give the variance and ask for the standard deviation or vice versa; for example, if \(\sigma^2=16\), then \(\sigma=4\).
Check the units: variance is in squared units, and standard deviation is in the original units; for example, if the data are measured in meters, then the variance is in square meters and the standard deviation is in meters.
When time is short, first verify the correctness of \(\bar{x}\), because any error propagates to all subsequent steps in variance calculation; an error in summing the values or dividing by n produces an incorrect variance.
Formulas
In this section, we learn about the basic statistical measures used to describe data distribution, whether they represent an entire statistical population or a sample from it. These measures include the arithmetic mean, variance, standard deviation, and interquartile range (IQR). Each measure has a specific formula used depending on the nature of the data (population or sample), and understanding the differences between them is essential to avoid common errors in statistical analysis.
Definitions and Formulas
Below are the definitions and mathematical formulas for each measure, with clarification of the symbols used:
Arithmetic Mean (for samples and populations): It is denoted by \( \bar{x} \) and calculated by summing all values and dividing by their count: \( \bar{x} = \frac{\sum x_i}{n} \), where \( x_i \) represents each value in the set and \( n \) is the number of observations.
Population Variance: It is denoted by \( \sigma^2 \) and calculated as the average of squared deviations from the population mean \( \mu \): \( \sigma^2 = \frac{\sum (x_i - \mu)^2}{N} \), where \( N \) is the total size of the population.
Sample Variance: It is denoted by \( S^2 \) and calculated as the average of squared deviations from the sample mean \( \bar{x} \), with adjustment using \( n-1 \) instead of \( n \) to correct bias: \( S^2 = \frac{\sum (x_i - \bar{x})^2}{n-1} \).
Population Standard Deviation:It is the square root of the population variance: \( \sigma = \sqrt{\sigma^2} \).
Interquartile range (IQR):It is calculated as the difference between the third quartile and the first quartile: \( \text{IQR} = Q_3 - Q_1 \), where \( Q_1 \) is the first quartile (25th percentile) and \( Q_3 \) is the third quartile (75th percentile).
Reason for using \( n-1 \) in sample variance
When calculating sample variance, we use \( n-1 \) in the denominator instead of \( n \) to obtainan unbiased estimatorof the population variance \( \sigma^2 \). If we used \( n \), the estimate would tend to be lower than the true population variance, especially when the sample size is small. This adjustment is calledBessel's correctionand ensures that the expected value of the sample variance equals the actual population variance.
Step-by-step solved examples
Example 1: Calculating the mean, variance, and standard deviation of a population
Given: A population consisting of the following values: \( 2, 4, 6, 8 \).
Deviations from the mean: \( (3-5)^2 = 4 \), \( (5-5)^2 = 0 \), \( (7-5)^2 = 4 \)
Sum of squares: \( 4 + 0 + 4 = 8 \)
\( S^2 = \frac{8}{3-1} = \frac{8}{2} = 4 \)
Step 3: Calculate what the variance would be if we used \( n \) instead of \( n-1 \) (for comparison):
\( \frac{8}{3} \approx 2.67 \), which is less than 4, so using n underestimates the true variance if the sample represents the population.
Result: \( \bar{x} = 5 \), \( S^2 = 4 \), and we note that using \( n-1 \) gives a larger, more unbiased estimate of the population variance.
Common errors and how to correct them
Common mistake
Correction
Using \( n \) in the denominator of sample variance instead of \( n-1 \)
Always use \( n-1 \) when calculating sample variance to obtain an unbiased estimator of population variance.
Confusing the symbols \( \mu \) and \( \bar{x} \)
The symbol \( \mu \) represents the population mean, while \( \bar{x} \) represents the sample mean. They cannot be substituted for each other in formulas.
Calculating standard deviation as the mean of absolute values instead of the square root of variance
Standard deviation is defined as \( \sqrt{\text{variance}} \), not the mean of absolute values. The definition must be followed precisely.
Considering IQR equal to \( \frac{Q_3 + Q_1}{2} \) or any other operation besides subtraction
IQR is explicitly defined as the difference: \( Q_3 - Q_1 \). Any other operation (such as addition or averaging) is incorrect.
Test tips
Make sure to read the question carefully to determine whether the data represent a full population or a sample, as this determines which formula to use: \( N \) or \( n \) in the denominator, and \( \mu \) or \( \bar{x} \) in the deviations.
When calculating variance or standard deviation, always compute the mean first, then the deviations, then their squares, then the sum, and finally divide by the appropriate denominator.
Remember that standard deviation cannot be negative, because it is the square root of a non-negative number (the sum of squares).
In IQR questions, you may be asked to find \( Q_1 \) and \( Q_3 \) first using the median-of-halves method or the method specified in the test, then subtract them.
Check the units: if the data are in certain units (such as cm or kg), then the variance is in squared units (cm², kg²), and the standard deviation is in the same unit as the original data.
Median: average of the fifth and sixth elements after sorting: \(10\) and \(11\) → \(\frac{10+11}{2}=10.5\).
First quartile \(Q_1\): median of the lower half (\(5,7,8,9,10\)) → \(8\).
Third quartile \(Q_3\): median of the upper half (\(11,12,13,15,42\)) → \(13\).
Interquartile range: \(IQR=Q_3-Q_1=13-8=5\).
Lower bound for non-outliers: \(Q_1-1.5\timesIQR=8-1.5\times5=8-7.5=0.5\).
Upper bound for non-outliers: \(Q_3+1.5\timesIQR=13+1.5\times5=13+7.5=20.5\).
The value \(42\) is greater than \(20.5\), so it is an outlier.
Upper bound: \(20.5\).
Definitions
Arithmetic mean: sum of values divided by number of values, denoted by \(\bar{x}=\frac{\sum x_i}{n}\).
Median: the value that divides the ordered data set into two equal halves; if \(n\) is odd, it is the \(\frac{n+1}{2}\)th value; if even, it is the average of the \(\frac{n}{2}\)th and \(\frac{n}{2}+1\)th values.
Mode: the value that appears most frequently in the data set; there may be no mode or more than one mode.
Variance: the average of the squared deviations from the mean; for a sample, it is computed as \(S^2=\frac{\sum (x_i-\bar{x})^2}{n-1}\).
Standard deviation: the square root of the variance, \(S=\sqrt{S^2}\).
Quartiles: divide the ordered data set into four equal parts; \(Q_1\) is the median of the lower half, \(Q_2\) is the median, \(Q_3\) is the median of the upper half.
Interquartile range: \(IQR=Q_3-Q_1\), measures the spread of the middle half of the data.
Outliers: values that lie outside the interval \([Q_1-1.5\times IQR,\; Q_3+1.5\times IQR]\).
Weighted mean: \(\sum (x_i\times w_i)\) where \(w_i\) are the weights summing to \(1\) (or \(100\%\) if percentages are used).
Common Mistakes
Error: calculating variance by dividing the sum of squared deviations by \(n\) instead of \(n-1\) for a sample.
Correction: For the sample, use \(n-1\) to obtain an unbiased estimate; for the population, use \(N\).
Error: Considering the mode as the mean when no value is repeated.
Correction: If no value is repeated, there is no mode; do not calculate the mean as the mode.
Error: Forgetting to order the data before calculating quartiles or the median.
Correction: First arrange the data in ascending order, then compute the positions.
Error: Using the threshold \(2\times IQR\) to detect outliers instead of \(1.5\times IQR\).
Correction: The standard threshold for detecting outliers in a box-and-whisker plot is \(1.5\times IQR\).
Error: Adding percentages directly without converting them to decimals in the weighted mean.
Correction: Convert percentages to decimals (e.g., \(40\%=0.4\)) before multiplication.
Test tips
Always check whether the data are ordered before calculating quartiles or the median.
Use the calculator carefully and verify the input of numbers and signs.
Remember that \(S^2\) for a sample is divided by \(n-1\); for a population, it is divided by \(N\).
When detecting outliers, first compute \(Q_1\), \(Q_3\), \(IQR\), then determine the boundaries.
In a weighted mean, ensure that the sum of the weights equals \(1\) (or \(100\%\) if using percentages).
Write each step clearly; even if the calculation is simple, showing work helps avoid errors and earns partial credit.
Review the units and context; if the data represent percentages, do not treat them as absolute numbers without conversion.
SVG
Box Plot
The box plot (Box Plot) is a graphical representation used to summarize the distribution of a numerical dataset by displaying key statistical values: the minimum (min), the first quartile (Q₁), the second quartile or median (Q₂), the third quartile (Q₃), and the maximum (max). This plot also shows the spread of the data and the presence of symmetry or skewness, and helps visually identify outliers.
In the box plot:
The long horizontal line represents the range of the data from the minimum to the maximum.
The box represents the interquartile range (IQR) from Q₁ to Q₃ and contains 50% of the data.
The line inside the box represents the second quartile Q₂ (the median).
The vertical lines at the ends of the box (from Q₁ to the minimum and from Q₃ to the maximum) are called 'whiskers'.
From the accompanying figure:
The minimum (min) = the value at x = 80.
The first quartile (Q₁) = the value at x = 140.
The median (Q₂) is the value at x = 190.
The third quartile (Q₃) is the value at x = 240.
The maximum (max) is the value at x = 300.
Thus, we can calculate:
Interquartile range: \( \text{IQR} = Q₃ - Q₁ \)
Total range: \( \text{Range} = \text{max} - \text{min} \)
Example 1: Reading values from a box plot
Solution: Step by step
Locate the minimum on the horizontal axis: it is the leftmost line, at x = 80 → \( \text{min} = 80 \).
Locate the first quartile Q₁: it is the left edge of the box, at x = 140 → \( Q₁ = 140 \).
Locate the median Q₂: it is the line inside the box, at x = 190 → \( Q₂ = 190 \).
Locate the third quartile Q₃: it is the right edge of the box, at x = 240 → \( Q₃ = 240 \).
Locate the maximum: it is the farthest right line, at x = 300 → \( \text{max} = 300 \).
Notice that the line inside the box (Q₂) is not exactly centered: it is closer to the left edge of the box (between Q₁ and Q₃).
This indicates that the data are right-skewed (positively skewed), because the median is closer to the first quartile.
Also note that the maximum (300) is farther from Q₃ (240) than the minimum (80) is from Q₁ (140):
Distance from Q₃ to max: \( 300 - 240 = 60 \)
Distance from min to Q₁: \( 140 - 80 = 60 \)
In this case, the distances are equal, but the position of Q₂ near Q₁ indicates a greater concentration of values in the lower quartile.
Therefore, the distribution is slightly skewed to the right (positively skewed).
Answer: The distribution is skewed to the right (positively skewed) because the median is closer to the first quartile.
Common Errors
Error: Assuming the box length equals the total range.
Correction: The box length equals IQR = Q₃ - Q₁, not the total range. Total range = max - min.
Error: Believing the median is always at the center of the box.
Correction: The median (Q₂) may be located anywhere within the box depending on the data skew; it is not required to be at the center.
Error: Confusing Q₁ and Q₃ due to similar color or shape.
Correction: Q₁ is the left edge of the box, Q₃ is the right edge; rely on position along the axis, not color alone.
Test Tips
Always begin by identifying the five key values: min, Q₁, Q₂, Q₃, max from the positions of lines and shapes on the horizontal axis.
Use the distances between these values to calculate IQR and range, and remember that IQR measures the spread of the middle half of the data.
To determine distribution skew, compare the position of Q₂ within the box: if closer to Q₁ → right skew; if closer to Q₃ → left skew; if centered → approximate symmetry.
Do not rely solely on color to distinguish the quartiles; colors may be similar or unclear in print, so position is the fundamental criterion.
In CSCA questions, you may be asked to interpret the diagram, calculate IQR, or determine whether potential outliers exist (although outliers are not shown in this simple figure).
Comparison
In this section, we study measures of central tendency (mean, median, mode) and measures of dispersion (population variance and sample variance). We explain the definition of each measure, its properties, and how to calculate it, with detailed step-by-step examples, warnings about common mistakes, and test-taking tips.
Mean vs Median vs Mode
Property
Mean
Median
Mode
Definition
\(\frac{\sum_{i=1}^{n} x_i}{n}\)
The value that divides the ordered data into two equal halves
The value that occurs most frequently in the data set
Affected by outliers
It is heavily affected by outliers because every value contributes to the sum.
It is not affected by outliers because it depends only on the position of the value in the ordered list.
It is not affected by outliers unless the outlier occurs more frequently than the original value.
A calculation for individual data.
Sum all values and divide by the number of values.
Arrange the data in ascending order; if \(n\) is odd, the median is the value at position \(\frac{n+1}{2}\); if \(n\) is even, the median is the average of the two values at positions \(\frac{n}{2}\) and \(\frac{n}{2}+1\).
Count the frequency of each value; the value with the highest frequency is the mode; there may be no mode or there may be more than one mode.
Example 1: Calculating the mean, median, and mode for an individual data set.
Solution: Step by step.
Given: The data set \( \{3, 7, 7, 2, 9\} \).
Calculating the mean: we sum the values \(3+7+7+2+9 = 28\). The number of values \(n=5\). Therefore, the mean \(= \frac{28}{5} = 5.6\).
Calculating the median: we arrange the data in ascending order: \( \{2, 3, 7, 7, 9\} \). Since \(n=5\) is odd, the middle position is \(\frac{5+1}{2}=3\). The value at the third position is \(7\). Therefore, the median \(=7\).
Calculating the mode: we count the frequencies: \(2\) appears once, \(3\) appears once, \(7\) appears twice, \(9\) appears once. The highest frequency is \(2\) for the value \(7\). Therefore, the mode \(=7\).
Results: the mean \(=5.6\), the median \(=7\), the mode \(=7\).
Example 2: Calculating the mean, median, and mode for an even data set with an outlier
Solution: step by step
Given: the data set \( \{10, 12, 14, 15, 100\} \).
Calculating the mean: we sum the values \(10+12+14+15+100 = 151\). The number of values \(n=5\). Therefore, the mean \(= \frac{151}{5} = 30.2\).
Calculating the median: we arrange the data in ascending order: \( \{10, 12, 14, 15, 100\} \). The middle position is \(\frac{5+1}{2}=3\). The value at the third position is \(14\). Therefore, the median \(=14\).
Calculating the mode: each value appears only once, so there is no mode (or we say the data set has no mode).
Results: mean \(=30.2\) (affected by outlier \(100\)), median \(=14\) (unaffected), no mode.
Common Errors
Error: Assuming the median is always the middle value in an unsorted list.
Correction: Data must be sorted in ascending or descending order before determining the middle position.
Error: Calculating the mean by dividing the sum by \(n-1\) instead of \(n\).
Correction: The mean is defined as \(\frac{\sum x_i}{n}\); using \(n-1\) is specific to calculating unbiased sample variance.
Error: Believing the mode must always be a single value.
Correction: There may be more than one mode (bimodal or multimodal set) or no mode if all values occur with equal frequency.
Test Tips
When large outliers are present, prefer using the median as a measure of central tendency because it is more resistant.
Always check the order of the data before calculating the median; if you forget to sort, you will get an incorrect value.
In multiple-choice questions, if asked for a measure not affected by outliers, choose the median or mode (depending on the context).
Remember the formula \(\frac{\sum_{i=1}^{n} x_i}{n}\) for the mean and do not confuse it with the formula for variance.
Population variance vs sample variance
M
Population σ²
Sample S²
Definition
\(\sigma^2 = \frac{\sum_{i=1}^{N} (x_i - \mu)^2}{N}\) where \(\mu\) is the population mean
\(S^2 = \frac{\sum_{i=1}^{n} (x_i - \bar{x})^2}{n-1}\) where \(\bar{x}\) is the sample mean
Denominator
\(N\) (population size)
\(n-1\) (sample size minus one)
Purpose
Measures the true variance of the entire population
Provides an unbiased estimate of the population variance based on a sample
Example 3: Calculating population variance and sample variance for a complete dataset (population)
Solution: Step by step
Given: A population consisting of the values \({4, 6, 8, 10}\). Population size \(N=4\).
Calculate the population mean \(\mu\): \(\mu = \frac{4+6+8+10}{4} = \frac{28}{4} = 7\).
Calculating deviations from the mean: \(4-7=-3\), \(6-7=-1\), \(8-7=1\), \(10-7=3\).
Error: Using \(n\) in the denominator when calculating sample variance instead of \(n-1\).
Correction: To obtain an unbiased estimate of the population variance, divide the sum of squared deviations by \(n-1\); using \(n\) yields a downward-biased estimate.
Error: Confusing the population mean \(\mu\) with the sample mean \(\bar{x}\) in variance formulas.
Correction: For population variance, we use \(\mu\) (the known or computed mean of the entire population); for sample variance, we use \(\bar{x}\) (computed from the sample only).
Error: Assuming that the sample variance always equals the population variance.
Correction: The sample variance is a random estimate of the population variance; it may be larger or smaller depending on the selected sample.
Test Tips
If the problem states that the data represent the entire population, use formula \(\sigma^2 = \frac{\sum (x_i - \mu)^2}{N}\).
If the problem states that the data represent a sample drawn from a larger population, use formula \(S^2 = \frac{\sum (x_i - \bar{x})^2}{n-1}\).
Remember the difference in the denominator: \(N\) for the population and \(n-1\) for the sample; this distinction is the most common source of errors.
When calculating variance, compute the mean first, then the deviations, then their squares, then sum them, and finally divide by the appropriate denominator; avoid fragmented steps to reduce computational errors.
Tips
CSCA Tips and Key Ideas
Sample variance: We divide by \(n-1\), not by \(n\), because \(n-1\) is the number of degrees of freedom after estimating \(\bar{x}\) from the sample itself, and dividing by it gives an unbiased estimate of the population variance, so do not confuse this with the population variance \(\sigma^2\).
Standard deviation: \(S=\sqrt{S^2}\); compute the variance first, then take its square root, and remember that the unit of standard deviation is the same as the unit of the original data, so it is the measure used for comparison.
Median: First arrange the data in ascending order; if \(n\) is odd, the median is the value at rank \(\frac{n+1}{2}\); if even, it is the average of the two values at ranks \(\frac{n}{2}\) and \(\frac{n}{2}+1\).
Outliers: A value is an outlier if it satisfies \(x < Q_1-1.5\,\text{IQR}\) or \(x > Q_3+1.5\,\text{IQR}\), where \(\text{IQR}=Q_3-Q_1\).
Practical shortcuts
Compute \(\text{IQR}\) first; from it, the outlier bounds are obtained directly by multiplying by \(1.5\), and it is not affected by extreme values, unlike the range.
Do not compare the dispersion of two datasets with different units using standard deviation alone, but use the coefficient of variation \(\frac{S}{\bar{x}}\).
Common Errors
Error
Correction
Dividing by \(n\) in sample variance
\(S^2=\frac{\sum (x_i-\bar{x})^2}{n-1}\)
Confusing variance with standard deviation
\(S=\sqrt{S^2}\)
Neglecting to order the data before computing the median
Arrange the data in ascending order, then select the rank
Confusing the range with \(\text{IQR}\)
\(\text{IQR}=Q_3-Q_1\)
Explanation and reason for each rule
Reason for using \(n-1\) in sample variance: When calculating the sample mean \(\bar{x}\), one degree of freedom is lost because \(\sum (x_i-\bar{x})=0\) always holds, so the number of independent variables remaining is \(n-1\) only, and dividing by this number gives an unbiased estimate of the population variance \(\sigma^2\).
Reason for defining standard deviation as the square root of variance: Variance is measured in squared units of the original data, so to restore the original unit of measurement, we take the square root. For example, if the data are in centimeters, the variance is in square centimeters and the standard deviation is in centimeters.
Reason for calculating the median via ordering: The median divides the data into two equal halves after ordering, so it cannot be determined without first sorting; when the number of observations is odd, there is a single middle value, and when even, there is no unique middle value but rather the average of the two middle values.
Reason for using \(1.5\,\text{IQR}\) in outlier detection: This criterion is based on the approximate normal distribution of data, where about \(99.3\%\) of the data lie within \([Q_1-1.5\,\text{IQR}, Q_3+1.5\,\text{IQR}]\), so values outside this range are considered very rare and likely due to errors or exceptional phenomena.
Step-by-step solved examples
Example 1: Calculating sample variance and standard deviation for the data \(\{2,4,6,8\}\) Step 1: Calculate the mean \(\bar{x}=\frac{2+4+6+8}{4}=5\) Step 2: Calculate squared deviations from the mean \((2-5)^2=9\), \((4-5)^2=1\), \((6-5)^2=1\), \((8-5)^2=9\) Step 3: Sum the squares \(\sum (x_i-\bar{x})^2=9+1+1+9=20\) Step 4: Calculate the sample variance \(S^2=\frac{20}{4-1}=\frac{20}{3}\approx6.6667\) Step 5: Calculate the standard deviation \(S=\sqrt{\frac{20}{3}}=\frac{2\sqrt{15}}{3}\approx2.58199\) Result: The variance \(\frac{20}{3}\) and the standard deviation \(\frac{2\sqrt{15}}{3}\).
Example 2: Determine the median and outliers of the data \(\{1,3,3,6,7,8,9,10,12\}\) Step 1: Ensure the data is sorted in ascending order (it is). Step 2: Calculate the number of elements \(n=9\) (odd) Step 3: Calculate the median rank \(\frac{n+1}{2}=\frac{9+1}{2}=5\) Step 4: The value at the fifth rank is \(7\), so the median \(=7\) Step 5: Calculate the first quartile \(Q_1\): median of lower half \(\{1,3,3,6\}\) is \(\frac{3+3}{2}=3\) Step 6: Calculate the third quartile \(Q_3\): median of upper half \(\{8,9,10,12\}\) is \(\frac{9+10}{2}=9.5\) Step 7: Calculate \(\text{IQR}=Q_3-Q_1=9.5-3=6.5\) Step 8: Calculate the lower outlier bound \(Q_1-1.5\,\text{IQR}=3-1.5\times6.5=3-9.75=-6.75\) Step 9: Calculate the upper outlier bound \(Q_3+1.5\,\text{IQR}=9.5+1.5\times6.5=9.5+9.75=19.25\) Step 10: Check each value: all values lie between \(-6.75\) and \(19.25\), so there are no outliers. Result: The median \(=7\), \(Q_1=3\), \(Q_3=9.5\), \(\text{IQR}=6.5\), and there are no outliers.
List of "Common Mistakes" with correction for each error
Error: Dividing by \(n\) in sample variance Correction: Must divide by \(n-1\) to obtain an unbiased estimate, i.e., \(S^2=\frac{\sum (x_i-\bar{x})^2}{n-1}\)
Error: Confusing variance with standard deviation Correction: Variance is \(S^2\) and standard deviation is the square root of variance \(S=\sqrt{S^2}\), so do not substitute one for the other without taking the square root or squaring.
Error: Forgetting to sort before calculating the median Correction: Always sort the data in ascending order first, then apply the rank formula depending on whether the number of data points is odd or even \(n\).
Error: Confusing range with \(\text{IQR}\) Correction: The range is \(\max-\min\) and is affected by outliers, while \(\text{IQR}=Q_3-Q_1\) measures the spread of the middle half and is not affected by outliers.
Test Tips
Always check whether the required variance is population variance or sample variance; if the data are a sample from a larger population, use \(n-1\) in the denominator.
After computing any variance, ask yourself: Are the units squared? If so, take the square root to obtain the standard deviation in the original data units.
Before calculating the median or quartiles, sort the data in ascending order without exception; this prevents common sorting errors.
When searching for outliers, first compute \(\text{IQR}\), then multiply by \(1.5\), and do not use the range for this purpose.
When comparing the spread of two data sets with different units (e.g., centimeters and kilograms), compute the coefficient of variation \(\frac{S}{\bar{x}}\) for each set, then compare the dimensionless values.
In questions containing frequency tables, remember that the median may lie within a class; use the formula \(L+\left(\frac{\frac{n}{2}-F}{f}\right)\times w\) where \(L\) is the lower limit of the median class, \(F\) is the cumulative frequency before the median class, \(f\) is the frequency of the median class, and \(w\) is the class width.
Be sure to write symbols only inside \( ... \), and do not write any LaTeX commands outside the brackets; do not use * for multiplication, but use \(\times\) if necessary.
Exercises
Exercise 1
Given: \(12,15,18,20,22,25,30\). Mean ≈ \(20.29\). Median = \(20\).
Exercise 2
Given: \(2,4,6,8,10\). Variance \(S^2=10\). Standard deviation \(S≈3.16\).
Exercise 3
Given: \(2,4,4,5,6,7,8,9,10,12\). First quartile \(Q_1=4\). Second quartile (median) \(Q_2=6.5\). Third quartile \(Q_3=9\).
Exercise 4
Frequency table: \(5(2),6(4),7(6),8(5),9(3)\). Mean = \(\frac{143}{20}=7.15\).
Exercise 5
Given: \(5,7,8,9,10,11,12,13,15,42\). The value \(42\) is an outlier because the upper bound for non-outliers is \(20.5\).