News agencies today report that "RAF strikes since 2014 have killed around 330 IS fighters". The UK defence secretary said the figure was "highly approximate" since there were no UK ground troops there to confirm casualties.
He also said ministers did not believe the action had caused civilian casualties.
It looks like both these claims are highly speculative and based on very little information. No source is given for the claim about civilian casualties. To be fair to the minister, he was answering a Parliamentary question so had to say something, but what he did say should probably be heavily discounted.
Stories such as this (not just about warfare) appear all the time in the press. It is always useful to ask yourself 'what is the evidence for this?'. If little or none is given then one should not take the claims too seriously (even if the issue itself is serious).
Thursday, September 17, 2015
Wednesday, September 2, 2015
Material for the new edition - how to improve a graph
Here is something that I am intending to put in the new edition, showing how one can improve a graph to make it more readable and impart a message more clearly. This will be in the form of a 'boxout', separated off from the main explanatory text. When learning (about it any subject) it is useful to see examples of bad presentation as well as of good. Do you think this is a useful example from which you learn something? Add your comment below.
Today we are assailed with information presented in the form of graphs, sometimes done well but often badly. We give an example below of how presentation might be improved for one particular graph, showing employers’ perceptions of Economics graduates’ skills. One can learn a lot from looking at examples of graphs in reports and academic papers and thinking how they might be improved. The original graph is not actually a bad one but it could be better.
Improving the presentation of graphs - an example
Problems
with this picture include:
1.
The category labels are
difficult to read, being small and wrap-around text
2.
The vertical axis title is
sideways, so difficult to read
3.
It is difficult to compare
across categories. For example, which
skill has the most ‘very high’ or ‘fairly high’ responses?
4.
A subjective judgement, but
the colours are not particularly harmonious.
The
version below takes the same data but presents it slightly differently:
Turning
the graph on its side means that the labels are much easier to read, as is the
horizontal axis label. Making it a
stacked bar chart saves space and makes it look less cluttered. It is fairly easy to see that ‘interpreting
quantitative data’ scores the most ‘very high’ or ‘fairly high’ responses –
hopefully this book makes some contribution towards that! Using different shades of the same colour
makes for a better appearance (and probably works better if printed in
grayscale too).
You might
have noticed that the categories are now in a different order. This is a quirk of Excel, the same data table
was used for both charts. Fortunately
the ordering does not matter. We shall
give similar examples at other places in this book.
New edition!
I am currently working on a new edition of my textbook, which will be the seventh. It has been pretty successful - it is the market leader in the UK - and by now its structure and content are reasonably settled. However, I do want to make some changes to the new edition based on the opinions and suggestions of reviewers and on my own experience of teaching and working with statistics.
It would be great if I could hear from you also about both the existing text and about my ideas for changes in the next edition. What do you like about the book and what do you not like? Are there parts where you find the explanations difficult to follow? Is there anything you think is missing? Do you use the associated web site with quizzes etc? Please contribute by leaving comments on this post.
Now here's what I plan to change in the next edition:
It would be great if I could hear from you also about both the existing text and about my ideas for changes in the next edition. What do you like about the book and what do you not like? Are there parts where you find the explanations difficult to follow? Is there anything you think is missing? Do you use the associated web site with quizzes etc? Please contribute by leaving comments on this post.
Now here's what I plan to change in the next edition:
- In chapter 1 on descriptive statistics - have some examples of good and bad graphs, and showing how to improve a bad graph into a better one.
- Chapter 2 on probability - a more detailed explanation of the principles of probability including the use of Venn diagrams to illustrate those principles, making them more intuitively clear.
- Chapter 8 on multiple regression - have examples showing how one can graph the regression coefficients, rather than have them listed in a table. The graphical presentation is easier to comprehend and interpret.
- Chapter 9 on data collection - update the sources of data to reflect the huge increase in online sources, including issues around so-called 'big data'
- Generally - add a few more examples related to business and management; currently the text focuses a bit more on Economics.
I will be following up with posts covering the new material. You can also comment on these as they go online.
Sunday, March 24, 2013
Teachers get concerned about body image
The BBC reports this story about adolescents' body image. The main point made in the article is:
"The Association of Teachers and Lecturers claims the promotion of ideal body images is reducing both boys' and girls' confidence in their own bodies."
However, reading the story uncovers little actual evidence to support the argument. It reports a survey of 693 members of the teachers union ATL and makes statements such as "78% [of teachers] thought girls suffered low self-esteem and 51% thought boys had low confidence in their body image".
It is easy to misread this as 78% of girls and 51% of boys had low confidence, rather than this being the proportion of teachers who thought that children has such issues. Hence the evidence from the sample could be quite consistent with a much smaller proportion of children with body image issues. It is just that a lot of teachers know about the small proportion of such children.
There are also other questions we should ask about the survey. Although the sample size is a decent 693, how was the sample selected? If it is from among union members, who tend to be more left wing and sensitive to social issues, does this constitute a representative sample? How did the teachers conclude that it was the promotion of an ideal body image that causes the lack of confidence?
All this is not to say that there is not such a problem amongst children. It's just that this survey gives us very little new information about it.
"The Association of Teachers and Lecturers claims the promotion of ideal body images is reducing both boys' and girls' confidence in their own bodies."
However, reading the story uncovers little actual evidence to support the argument. It reports a survey of 693 members of the teachers union ATL and makes statements such as "78% [of teachers] thought girls suffered low self-esteem and 51% thought boys had low confidence in their body image".
It is easy to misread this as 78% of girls and 51% of boys had low confidence, rather than this being the proportion of teachers who thought that children has such issues. Hence the evidence from the sample could be quite consistent with a much smaller proportion of children with body image issues. It is just that a lot of teachers know about the small proportion of such children.
There are also other questions we should ask about the survey. Although the sample size is a decent 693, how was the sample selected? If it is from among union members, who tend to be more left wing and sensitive to social issues, does this constitute a representative sample? How did the teachers conclude that it was the promotion of an ideal body image that causes the lack of confidence?
All this is not to say that there is not such a problem amongst children. It's just that this survey gives us very little new information about it.
Thursday, November 8, 2012
Type I and Type II errors
A recent news item (see here for the BBC report) covered research into the UK breast cancer screening. The problem with the screening is that it is not perfect: it can miss some genuine tumours but it can also 'over-diagnose', i.e. signal a tumour which is actually harmless.
These should be familiar as Type I and II errors. The null hypothesis is the absence of a tumour, so a Type I error is that of incorrectly diagnosing a tumour, when there isn't one. A Type II error is missing a genuine tumour. (One could look at this the other way round, with the null being the presence of a tumour, etc.)
According to the report, for every life saved, three women had unnecessary treatment for cancer. This seems quite a high ratio but partly reflects the fact that the incidence of cancer is actually quite low. The probability of a Type I error is given as 1% in the article. This would be consistent with something like the following: for every thousand women tested, 10 are incorrectly diagnosed and treated, while three are correctly diagnosed and treated (hence approximately three times as many false positives as genuine ones.)
As well as the probabilities, the costs of the errors should also be taken into account. The cost of missing a diagnosis is apparent to us, which is why there is a national system of screening. The costs of over-diagnosis are less obvious but can be substantial. The treatment is unpleasant, to say the least. The costs of over-diagnosis might also be masked because it is concluded that the treatment has worked, rather than that there never was a cancer.
These should be familiar as Type I and II errors. The null hypothesis is the absence of a tumour, so a Type I error is that of incorrectly diagnosing a tumour, when there isn't one. A Type II error is missing a genuine tumour. (One could look at this the other way round, with the null being the presence of a tumour, etc.)
According to the report, for every life saved, three women had unnecessary treatment for cancer. This seems quite a high ratio but partly reflects the fact that the incidence of cancer is actually quite low. The probability of a Type I error is given as 1% in the article. This would be consistent with something like the following: for every thousand women tested, 10 are incorrectly diagnosed and treated, while three are correctly diagnosed and treated (hence approximately three times as many false positives as genuine ones.)
As well as the probabilities, the costs of the errors should also be taken into account. The cost of missing a diagnosis is apparent to us, which is why there is a national system of screening. The costs of over-diagnosis are less obvious but can be substantial. The treatment is unpleasant, to say the least. The costs of over-diagnosis might also be masked because it is concluded that the treatment has worked, rather than that there never was a cancer.
Election polls and odds
The recent US election provides some interesting opportunities to look at the opinion polls. One of the most accurate turned out to be that of Nate Silver of the New York Times who gathered up all the opinion poll data and turned it into a prediction of victory.
Many journalists and opinion-formers in the US were saying the election was 'too close to call' even on the eve of the election itself. But this seems to confuse two quite different possibilities:
1. Strong evidence of a narrow win for Obama
2. No evidence of a strong win for either side
Many commentators went with 2 above, but 1 is the correct interpretation. Let's see how this works.
Silver gives evidence for Colorado, one of the 'tipping point states' that could be decisive in the election. Based on the various polls, Silver projected vote shares of 50.8% for Obama, 48.3% for Romney. On this basis it looks fairly close, and this is probably how the commentators viewed it (especially as there is a +/-3% points margin of error on the polls). However, Silver also gives the projected probabilities of winning, which are 80% Obama, 20% Romney. This looks much more decisive. How do we get from the poll figures to the odds of 80:20?
If we take the margin of error as representing two standard errors, as is usual, then the standard error is 1.5%. Disregarding the 0.9% of voters not supporting either candidate, we have p = 50.8/99.1 = 51.26% for Obama and hence 48.7% for Romney.
We then ask, is it likely that Obama's true share of the vote is less than 50%? This is a question using a sample proportion so we calculate the z-score as:
z = (0.5126 - 0.5)/1.5 = 0.84
This cuts off 20.05% in the tail of the distribution. This tells us that there is a 20% chance of getting such evidence (sample proportion of 51.26% or more) if Obama's true vote share is 50% or less. Hence there is a 20% chance of a Romney victory, 80% for Obama.
This is my own take on the evidence, Silver's procedure is probably more sophisticated, but our approximation seems to work. (You could try it out on other states to see if you too can replicate it. Here's Virginia, another tipping point state: Obama 50.7, Romney 48.7, margin of error +/-2.5. Silver's odds for this are 79:21 for Obama.)
Note also that +/-3% (points) is a typical margin of error for polls. Recall that the standard error of a proportion is the square root of p(1-p)/n. If p is approximately 50% and n is about 1000 (a typical poll) then the formula gives a standard error of 1.58. Doubling this gives our margin of error.
Many journalists and opinion-formers in the US were saying the election was 'too close to call' even on the eve of the election itself. But this seems to confuse two quite different possibilities:
1. Strong evidence of a narrow win for Obama
2. No evidence of a strong win for either side
Many commentators went with 2 above, but 1 is the correct interpretation. Let's see how this works.
Silver gives evidence for Colorado, one of the 'tipping point states' that could be decisive in the election. Based on the various polls, Silver projected vote shares of 50.8% for Obama, 48.3% for Romney. On this basis it looks fairly close, and this is probably how the commentators viewed it (especially as there is a +/-3% points margin of error on the polls). However, Silver also gives the projected probabilities of winning, which are 80% Obama, 20% Romney. This looks much more decisive. How do we get from the poll figures to the odds of 80:20?
If we take the margin of error as representing two standard errors, as is usual, then the standard error is 1.5%. Disregarding the 0.9% of voters not supporting either candidate, we have p = 50.8/99.1 = 51.26% for Obama and hence 48.7% for Romney.
We then ask, is it likely that Obama's true share of the vote is less than 50%? This is a question using a sample proportion so we calculate the z-score as:
z = (0.5126 - 0.5)/1.5 = 0.84
This cuts off 20.05% in the tail of the distribution. This tells us that there is a 20% chance of getting such evidence (sample proportion of 51.26% or more) if Obama's true vote share is 50% or less. Hence there is a 20% chance of a Romney victory, 80% for Obama.
This is my own take on the evidence, Silver's procedure is probably more sophisticated, but our approximation seems to work. (You could try it out on other states to see if you too can replicate it. Here's Virginia, another tipping point state: Obama 50.7, Romney 48.7, margin of error +/-2.5. Silver's odds for this are 79:21 for Obama.)
Note also that +/-3% (points) is a typical margin of error for polls. Recall that the standard error of a proportion is the square root of p(1-p)/n. If p is approximately 50% and n is about 1000 (a typical poll) then the formula gives a standard error of 1.58. Doubling this gives our margin of error.
Thursday, July 12, 2012
Seasonal adjustment can be dangerous
Here's an interesting post from Paul Krugman talking about the implications of smoothing a data series, in this case GDP. Although it's about a technique called the Hodrick-Prescott (H-P) filter, this is just a fancy means of smoothing, as covered in chapter 11 of the book, on seasonal adjustment.
Essentially, Krugman's argument is that economists have smoothed the GDP data and then called this 'potential output'. Since actual GDP is not far off the smoothed value (in 2012, as I write), some interpret this to mean there is not much of a recession. Hence little need for active fiscal or monetary policy to address the problem.
However, the smoothed series inevitably follows the actual series (even though it changes more slowly) and hence is bound not to be too far away. The implicit assumption is that when actual output falls, so does potential output, which is generally not warranted. The H-P filter only filters out short-term fluctuations and does not deal well with large changes, such as a recession.
Who would have thought that such a technical issue could have such a powerful effect upon the debate around economic policy?
Essentially, Krugman's argument is that economists have smoothed the GDP data and then called this 'potential output'. Since actual GDP is not far off the smoothed value (in 2012, as I write), some interpret this to mean there is not much of a recession. Hence little need for active fiscal or monetary policy to address the problem.
However, the smoothed series inevitably follows the actual series (even though it changes more slowly) and hence is bound not to be too far away. The implicit assumption is that when actual output falls, so does potential output, which is generally not warranted. The H-P filter only filters out short-term fluctuations and does not deal well with large changes, such as a recession.
Who would have thought that such a technical issue could have such a powerful effect upon the debate around economic policy?
Subscribe to:
Posts (Atom)