23
Aug

## Stratifying PISA scores by poverty rates suggests imitating Finland is not necessarily the way to go for US schools

248142

For the past several years a steady stream of articles and opinion pieces have been praising the virtues of Finish schools and exalting the US to imitate this system. One data point supporting this view comes from the most recent PISA scores (2009) in which Finland outscored the US 536 to 5o0. Several people have pointed out that this is an apples (huge and diverse) to oranges (small and homogeneous) comparison. One of the many differences that makes the comparison complicated is that Finland has less students living in poverty ( 3%) than the US (20%). This post defending US public school teachers makes this point with data. Here I show these data in graphical form. The plot on the left shows PISA scores versus the percent of students living in poverty for several countries. There is a pattern suggesting that higher poverty rates are associated with lower PISA scores. In the plot on the right, US schools are stratified by % poverty (orange points). The regression line is the same. Some countries are added (purple) for comparative purposes (the post does not provide their poverty rates).   Note that US school with poverty rates comparable to Finland's (below 10%) outperform Finland and schools in the 10-24% range aren't far behind. So why should these schools change what they are doing? Schools with poverty rates above 25% are another story. Clearly the US has lots of work to do in trying to improve performance in these schools,  but is it safe to assume that Finland's system would work for these student populations?

Note that I scraped data from this post and not the original source.

• Fran

The plot on the left shows PISA scores
versus the percent of students living in poverty for several countries.
There is a pattern suggesting that higher poverty rates are associated
with lower PISA scores

Do you truly see a (linear) pattern there??

doesn't appear to be completely linear-I'd be interested to know the R^2 value...though he still seems to do a fair job of demonstrating his point on the right graph

• Rafael

No, i don't see nor claim a linear pattern. The line is added as summary statistic for illustrative purposes. Sorry I wasn't clear on this.

• Ken Butler

I'd say the (weak) negative correlation is driven mainly by Turkey, Chile, and Mexico at the bottom right, and that otherwise there isn't much of a correlation at all.

• rafael

those were not included in the regression. the regression line was calculated using only the points from the plot on the left

• Ken Butler

ok, but the line on the left graph only shows a drop of about 10 points on the PISA scale as poverty goes from 0% to 20%. Given the variability in PISA scores overall (even the interquartile range would be at least 20 points), I don't see this trend as anything to get excited about.

• Peter Chapman

Before you start fitting statistical models you need to apply some judgement about whether the model is relevant or realistic. In the left hand graph there is clearly no relationship so a straight line regression should not have been fitted. Even if the spread of points around the line did look as though they could have come from such a model (they don't) it would be useless for prediction because of the huge variation around the line.

• Giusi

Are there any schools at all in Finland with poverty rates above 10%? If so where would the Finland point fall when removing them?

• rafa

I have the same question, but these data were not available in the post from which I scraped data. I doubt Finland has school with poverty rates above 50% though.

• David Hood

I have a problem with the entire stratification process. It is using the free and discount meals rate as the analogue to the OECD figures based on household income within the country. While it may be kinda sorta true for a measure within the US, I don't see how it can be used as a direct percentile comparison with the international figures without many caveats. Basically, I question any resulting trend line.

• Rafa

Good point, but keep in mind the regression line was computed using only the green points.

• RonnieF

This recalls an analysis Steve Sailer did 3 years ago http://isteve.blogspot.com/2010/12/pisa-school-test-scores-by-ethnicity.html. Of course he did not infer that the cause was "poverty."

• JayLivingston

The data in the graph (Finland 536, US 500) seem to be for reading scores only. In math and science, the differences are larger -- 541 - 487 and 554-502, respectively.

• Markku Hannula

It's not that Finland is much more wealthy than USA. More importantly, our society does not stratify poor and rich apart from each other as strongly as the US. In addition, in Finland the relationship between student achievement and their socio-economic status is much weaker than the same relationship in the USA.

However, the graph does reveal the truth that in USA your success in school (and also in life) is based on the wealth of your family. I wonder what happened to the American Dream. http://en.wikipedia.org/wiki/American_Dream

• Cassanders

But here are other and quite interesting differences in the PISA scorees. One can for instance compare the egalitarian and all relatively afluent nordic/Scandinavian countries. For some reason Sweden has been omitted from this graph, but the results have caused quite some hoopla there, I can assure you. From an above-average OECD score, they have had a very clear fall to well below OECD average during the last decade..."Stinking rich" Norway ( who likely spend the most rescoures per student) does not perform very well, etc.

So, at least for the (socioeconomically comparable) Nordic countries, an affluence/PISA score relation does not seems to be a promising hypothesis.
Cassanders
In Cod we trust

• Elina Huu

This is a silly way to compare the countries! If you want to compare the pupils from wealthy families say even the wealthiest 50 % of families in US to some other country, you need to compare them to the wealthiest 50% in that other country!

• http://politivajustice.com/ Politiva_com

poverty levels are silly?

• 1625toots

Finland has only a few million people, and their scores fall according to childhood poverty too. The scores show us the correlation between the two.