Chapter 13 · Statistics and data displays
Outliers and Mistakes in Measures of Variability
Spot a value far from the rest of a data set and see how it changes the range, IQR and MAD. Then learn to catch common mistakes in finding these measures.
Sometimes one value in a data set sits far from all the others. Suppose your trip to school takes between 18 and 22 minutes on most days. One day you get a ride, and the trip takes 5 minutes. That 5 does not fit with the rest.
A value like this is called an outlier: a data value that is far from the rest of the data, either much greater or much less than the other values.
An outlier matters because it can change a measure of variability, which is a single number that describes how spread out the data values are. One measure you know is the range: the greatest value minus the least value. In this lesson you will see which measures an outlier changes a lot and why. Then you will use those ideas to catch mistakes in someone else's work.
Worked example
Spot the outlier and check the range
Eight students recorded how many pages they read last night: . Is there an outlier? If there is, how does it affect the range?
Put the page counts in order from least to greatest. In order, neighbors sit side by side, so you can see the gap between each pair.
Look at the gaps between neighbors. From 11 up to 19, every gap is only 1 or 2 pages. The gap from 19 to 47 is far bigger than any other gap. So 47 is far from the rest of the data, and that makes it an outlier.
Find the range with 47 in the data. The range is the greatest value minus the least value, so it measures the distance between the two ends of the data.
Find the range without 47. With the outlier taken out, the greatest value is 19 and the least value is still 11. Notice that 19 is not an outlier. It is only 2 pages above 17, so it is close to the rest.
Compare the two ranges by subtracting. Both ranges subtract the same least value, 11, so they differ by exactly the gap between 19 and 47. The outlier adds that whole gap to the range.
Answer
Yes. The value is an outlier because it is pages above the next greatest value, while the other seven values are within pages of each other. The range is pages with and pages without it, so the outlier makes the range pages larger.
The range changed so much because it uses only two values, the greatest and the least, and the outlier is one of them. Other measures of variability use the data in different ways, so an outlier can change some of them much more than others.
Two ideas you already know will help you see why. The mean is the sum of the values divided by how many values there are. The median is the middle value of an ordered list, or the mean of the two middle values when the count is even.
Here is how two more measures use the data.
- The interquartile range (IQR) comes from two quartiles. Split the ordered data into a lower half and an upper half of the same size. When the number of data values is odd, this course always leaves the overall median out of both halves. , the first quartile, is the median of the lower half. , the third quartile, is the median of the upper half. The IQR is . It is a difference between two numbers, not a count of values.
- The mean absolute deviation (MAD) is the mean of the distances between each data value and the mean of the data. It uses every value, including an outlier.
Before you calculate, make a prediction. Which will 47 change more, the IQR or the MAD?
Worked example
Find the outlier's effect on the IQR and the MAD
Use the ordered page counts . Find the IQR and the MAD with in the data, then without it. Which measure does the outlier change more?
Find the IQR with 47. There are 8 values, an even count, so the lower half is 11, 13, 14, 15 and the upper half is 16, 17, 19, 47. Each half has four values, so its median is the mean of its two middle values. The first quartile is (13 + 14) ÷ 2 = 13.5. The third quartile is (17 + 19) ÷ 2 = 18. The IQR is the third quartile minus the first quartile.
Find the IQR without 47. Now there are 7 values, an odd count. The median is the 4th value, 15. This course's rule for an odd count is to leave the median out of both halves. The lower half is 11, 13, 14, so the first quartile is its middle value, 13. The upper half is 16, 17, 19, so the third quartile is 17.
Find the mean of each list, because the MAD measures distances from the mean. Without 47, the sum is 11 + 13 + 14 + 15 + 16 + 17 + 19 = 105. With 47, the sum is 105 + 47 = 152. Divide each sum by its number of values. The outlier pulls the mean up from 15 pages to 19 pages.
Find the MAD with 47. A distance tells how far a value is from the mean, so it is never negative. From the mean 19, the distances for 11, 13, 14, 15, 16, 17, 19 and 47 are 8, 6, 5, 4, 3, 2, 0 and 28. The MAD is the mean of these distances, so add them and divide by 8.
Find the MAD without 47. Measure from this list's mean, 15. The distances for 11, 13, 14, 15, 16, 17 and 19 are 4, 2, 1, 0, 1, 2 and 4. Add them and divide by 7, the number of values.
Compare. Without the outlier, the IQR drops by 0.5 pages and the MAD drops by 5 pages. The IQR barely moves because both quartiles come from values inside the ordered list, not from 47 at the end. The MAD moves a lot because it uses every distance, and 47 adds a distance of 28.
Answer
With , the IQR is pages and the MAD is pages. Without , the IQR is pages and the MAD is pages. The outlier changes the MAD far more than the IQR.
| Measure of variability | With 47 (8 values) | Without 47 (7 values) | Drop |
|---|---|---|---|
Range | |||
IQR | |||
MAD |
Now put the effect into words. Seven of the eight students read within 8 pages of each other. With 47 in the data, the range is 36 pages and the MAD is 7 pages. Those numbers make the group look much more spread out than seven of the eight students really are. The IQR stays near 4 pages either way, so the outlier hardly affects it.
Taking 47 out is only a way to measure its effect. It does not mean the value is wrong. One student may really have read 47 pages.
Key concept
What an outlier does to variability
An outlier is a data value that is far from the rest of the data. How much it changes a measure of variability depends on how that measure uses the data.
- Range: a value far above or far below all the others is the greatest or least value, so the range is larger with it than without it.
- MAD: the outlier adds one large distance and shifts the mean, so the MAD can change a lot.
- IQR: often changes only a little, as long as the outlier is not a value used to find or .
To describe the effect, find the measure with the outlier and without it, then compare. Taking out one value switches the count between even and odd. For an odd count, follow this course's rule: leave the overall median out of both halves.
You can use the same rules to check someone else's work. Finding a mistake is not the same as starting over. Read the work one line at a time and test each line against the rule for that measure. When a line breaks a rule, you have found a mistake. The lines after it may be worked correctly and still give wrong numbers, because they depend on that line.
The table lists mistakes that come up often. Each row names one mistake and the check that catches it.
| Measure | Mistake to look for | Check that catches it |
|---|---|---|
Range | Subtracting the first and last values of a list that is not in order | Find the greatest value and the least value, then subtract the least from the greatest. |
Quartiles | Finding the median and the halves before putting the values in order | Order the values from least to greatest first. |
Quartiles, odd count | Keeping the overall median in one half or in both halves | This course's rule: leave the overall median out of both halves, so the halves are the same size. |
IQR | Counting the values between and , or subtracting two other numbers | Subtract: . It is a difference, not a count of values. |
MAD | Adding differences from the mean that include negative numbers | Use distances from the mean. Every distance is or greater. |
MAD | Dividing the sum of the distances by the wrong number | Divide by the number of data values, even when a distance is . |
Worked example
Find the mistake in a student's IQR work
Jordan finds the IQR of nine quiz scores, in points: . Jordan writes four lines. Line 1: the median is . Line 2: the lower half is , so . Line 3: the upper half is , so . Line 4: five scores lie between and , so the IQR is . Which line has a mistake? Correct it.
Check line 1. The scores are in order from least to greatest, as they must be before you find a median. There are 9 scores, an odd count, so the median is the 5th score, which is 9. Line 1 is correct.
Check the halves in lines 2 and 3. This course's rule for an odd count is to leave the median out of both halves. Jordan did that. The 9 is in neither half, and each half has four scores. The halves are correct.
Check the quartiles in lines 2 and 3. A half with four scores has two middle scores, so its median is their mean. Both of Jordan's quartiles are correct.
Check line 4. Jordan counted the scores between the quartiles: 6, 8, 9, 12 and 13. This is the mistake. The IQR is the third quartile minus the first quartile. It is a distance in points, not a count of scores. A count tells how many scores there are. It does not tell how far apart they are.
Correct line 4. Subtract the first quartile from the third quartile to find how far apart they are.
Answer
Line 4 has the mistake. Jordan counted the scores between the quartiles instead of subtracting. The IQR is points, not .
An outlier is a data value far from the rest of the data. To describe its effect, find a measure of variability with the outlier and without it, then compare. In the page data, taking out 47 cut the range from 36 pages to 8 and the MAD from 7 pages to 2, but the IQR only moved from 4.5 pages to 4.
To find a mistake in someone's work, test each line against its rule. Order the values first. For an odd count, leave the overall median out of both halves. Find the IQR by subtracting , not by counting. Find the MAD from distances, which are never negative.