Tuesday, 4 November 2014

Defence Against the Dark Arts: Statistics

I want to carry on the discussion of two different ways of wizarding begun in last week's post by looking at an example of what I'd call dark magic. Just as Saruman and Gandalf took different approaches to power in The Lord of the Rings, there are different ways of using magic as I've described it here- simply put, the ability to perceive and manipulate meaning.

By that definition, of course, everyone's a little bit magical. The fact that you can turn a series of black markings on a screen into living ideas that grow and move around inside of you is an indication of that trait, and a marvelous thing to consider. The fact that certain objects in your home are arranged so as to bring to life certain memories or feelings is an indication that you already have a sense that meaning is a dimension of the world as much as mass, temperature, and colour are. The fact that there have been systems of knowledge and practice developed to tap into this dimension, in order to, for example, train the will toward greater clarity of thought and action, or to activate the natural healing processes of human bodies, indicates that other peoples in other times and places have been more attuned to this dimension than we are here and now.

I am not learned in such arts. But in my own small way I am trying to understand the meanings of things, and to get behind the fog that surrounds certain aspects of our lives. The example I want to show you today involves the youtube, so find a pair of headphones or a quiet nook and check out this five minute video.

Did you watch it? How about that? Pretty neat, huh? Now let's talk about what just happened.

To begin with, we are invited to follow an elderly but spry wizard into his workshop as he tells us about his work as a teacher and his familiarity with data. He is genial but definitely learned beyond the average mortal. Next he shows us his team, all skilled in the technical arts, and invites us to join him on a venture which even he has never tried before: 'animating the data in real space'. Having uttered these words, whose meanings are hidden from the untutored, he proceeds to dazzle us with his art.

Data is one thing, presentation is another. You'd think that 120 000 data points would speak for themselves, but that's not so. As I've said in previous writings here, how one chooses to narrate world history says a lot more about the teller than about how we've got to where we are. Hans Rosling's perspective is clear: we're all becoming healthier and richer as time progresses, and we live in "a converging world" in which "the historical gap between the West and the rest is now closing".

But take another look at that graph- pause the video if you wish, and get out a tape measure or ruler. I'd highly recommend actually trying this, because it illustrates what I'm talking about far better than my description can. When I open the video in full-screen on my computer, my tape measure indicates that the left half of the graph, up until the $4000 mark, measures 3.4 inches. 3.4 inches to the right of this mark should land us at $8000 on the x-axis, but instead I find myself at well over $40 000. It's clear that the right half of the graph is working on a different scale than the left half; indeed, all three portions of the graph are operating at different inches-to-income ratios, or else the scale accelerates continuously at an unspecified rate. This would be fine if it were acknowledged and explained, but it isn't.

Now pause the video around 4:15 and pick a dot in the upper right corner of the graph, I don't know, say, the USA- it's the big yellow dot just past the $40 000 mark. If I use my tape measure to extend the x-axis along a uniform scale of 3.4 inches for every $4000, the $40 000 mark falls at 34 inches (2' 10") from the y-axis, putting the US nearly three screens away from Congo, the poorest nation on the graph. (It's not clear whether the 'Congo' dot represents the Republic of Congo or the Democratic Republic of Congo, which are separate but neighbouring nations, but from the size of the dot I'd guess it's meant to indicate the DRC, which is about 17 times more populous).

The tape measure exercise is a way of more accurately visualizing the data suggested by this graph. You can corroborate by comparing the 1810 stats (pause video at 1:15) and the 2010 stats (pause at 4:15). It's hard to tell at the outset which blue dot represents 'Congo', but the poorest nation in 1810 seems to have an average annual income of about $150. Assuming a uniform scale between the $400 and $4000 mark, the US falls at around $2000 a year. The income ratio between the poorest nation and the US in 1810 is thus 3:40. By 2010, the graph indicates that the poorest nation, Congo, earns double what the poorest nation in 1810 did, around $300 a year. Meanwhile the average US citizen has seen their annual income rise to twenty times what it was in 1810, at over $40 000 a year. The income ratio between the poorest nation and the US is now 3:400.

What this data demonstrate is not that we live in a converging world, but that over the past two hundred years the West and the rest have vastly outstripped the poorest nations, in terms of both real and relative wealth. In 1810 the highest-earning nation in the world, the Netherlands, earned 20 times what the lowest-earning nation did. In 2010 the highest-earning nation (I'm assuming that green dot is the United Arab Emirates) is off the chart, well past the US, which itself is well past earning 100 times what the lowest-earning nation makes.

These examples are from the extreme ends of the data spread, but you can easily pick a segment of the 2010 graph and notice nations whose citizens earn 10 times what people in other nations earn. In 1810, by contrast, only the very richest (the Netherlands, the UK, and those just behind them) and the very poorest (the African nations clustered around $300 a year and below) were separated by an order of magnitude.

You'll notice that I've been rather thorough so far about not doing any research outside of Hans Rosling's video, aside from dipping into my gazetteer to ascertain the relative populations of the two Congos. I'm taking it for granted that his team's numbers are true and accurate. The visual element, the graph, is somewhere between a truth and a lie, since it contains no falsehood but suggests something other than the truth. What sways me toward naming this little show as dark magic is that it supports an impressive-sounding expert like Hans Rosling in drawing conclusions that are just lies.

What's curious to consider is why he would do such a thing. I've no doubt that Rosling is a brilliant statistician, fully capable of creating a visual representation of world health and income statistics that doesn't distort the data. I've also no doubt that he knows what kind of picture a uniformly-scaled graph would paint, and that he made a deliberate decision to hide that picture from view. Perhaps he wanted to put the full weight of his scientific and technical authority behind a message of hope in "aid, trade, green technology and peace". The person who introduced me to this video certainly seemed encouraged by it. On the other hand, perhaps Rosling wanted to use his arts to summon such emotions in his viewers for other ends. You just can't tell by watching.

What you can tell is that the US, along with its cronies 'the West and the rest', have become fabulously wealthy in the past two centuries. This was mostly accomplished by monopolizing and burning through millions of years' worth of fossil fuel energy. It's a funny coincidence, to say the least, that as US power and wealth have grown, the number of nations hosting US military personnel has increased from 3 in the 1920's, to 39 during the Second World War, to 64 in 1968, to 148 in 2011. The world just seems to love American soldiers. It also seems to love American aid, trade, and peace, especially when those things come in the form of IMF debt, sweatshops, and regime changes. Too bad they aren't making a lot of green technology in the US these days, because the world really doesn't have enough yet of what America has to offer.

Of course, what we can actually expect from the future is a convergence of several factors not listed by Professor Rosling: disease, economic crisis, faltering technological innovation, and war, all hinging on the underlying factors of energy scarcity and an overburdened global empire. Each of those factors will feed into the others to create conditions such as those you see depicted at 1:55 in the video. These are not pleasant facts and numbers, but if we want to grasp the kind of real hope that sustains and empowers, we will have to first move through an honest assessment of what there is to hope for. It's not the end of the world, but it is the end of our world. There is no hope in the continuation of progress.

Now is the time for wizards in the tradition of Gandalf the Grey to arise and speak truth to power, because there is much that is good and that can be saved from the wreck of the empire of progress. The role of these wizards is to shake us free of the lies that tell us either to despair in the face of overwhelming power, on the one hand, or to remain naively hopeful in the security offered by power, depending on whether we see that power as benevolent or destructive.

The lies of Saruman the White and his followers are seductive, but as I've said before, sometimes a quick history lesson is enough to clear the air. And let's not forget that there is a third wizard in The Lord of the Rings, one whom Saruman laughs to scorn but whose wisdom is hidden from the minds of the mighty: Radagast the Brown, friend of all birds and beasts and growing things. Perhaps we'll have occasion to speak more about him in coming weeks.


  1. :-)

    Here's his website with many different datas you can play around with.

    While statistics as a field of inquiry are dangerously prone to dark magic, I've found one of the best ways of limiting their potential to do bad (and even facilitating their use for good!) is by understanding them. So in this context I would like to introduce you to two concepts in statistics that you may not have heard of, or at least didn't let slip that you know of in this post.

    The first is scale. The strangeness you noticed on the economic axis is because it is a "logarithmic" scale, rather than a linear scale. This just means that each unit along the axis represents multiplication instead of addition; the values of the axis increase exponentially. While not necessarily intuitive, it can be useful (when explained/understood and used openly). It can stretch graphs and make them easier to read, and can be useful in cases where smaller differences matter more at the low end and bigger differences matter more at the higher end. This is particularly true, it seems, with money. Your first 100 dollars are far more important to your health than your tenth 100 dollars, for example. Though I do agree, this video hides this idea (along with the many others you illuminated).

    The second thought I had about statistics while watching this video is one of 'outliers'. The general trend at any time seems to be that being richer makes you healthier. But this obviously simplifies the issue, for what of the Botswana, who die far younger than they 'should' given their income? And what of the Vietnamese, who seem to grow older than their money should allow? To me, what is most interesting when looking at a graph are those people and places who seem to inexplicably do better than our deductions based on statistical trends should allow. What is it about the Vietnamese ways of thinking and being that allow them to live so long despite their economic situation? Surely we should be more interested in these cases than in the general inflationary effect on longevity of the american empire and its economic system (given ITS seeming lack of potential longevity). But these stories are exactly the ones this kind of statistical perspective on the world cannot consider.

    It seems that the most interesting observations that arise from looking at these data in this way is in seeing what they are missing. Like the fact that people in the richest nations 100 years ago lived as long as the people in the poorest nations do today. What this exploration shows me is that we're living on the top edge of a very large soap bubble, only the soap isn't rendered from the fat of goat's milk, it's a petroleum byproduct that my my grandkids likely won't have. And so we shouldn't look at 2009 US and ask how we can be more like them. We should look at Iceland in 1800 and ask how we can be more like them. And we should try, desperately, to hold on to that knowledge which we've created in the last couple hundred years that could still be helpful in a post-oil context. It may have taken oil to invent the germ theory of disease, but I hope it doesn't take oil to remember it.

    Thank you, as always, for this interesting thought fodder.

  2. Dave, well put as regards the petroleum soap bubble.

    I did pick up on the logarithmic scale idea from a few of the youtube comments posted below the video (no, I'm not submitting these citations to an academic forum any time soon). But I'm not satisfied that Rosling was justified in using this kind of visual to suggest that "most of us live in the middle" of the data spread. When you expand the logarithmic scale into a linear scale, it's easier to see that that statement is not really false, but that it's also about as helpful (and as complimentary) as saying that half of us are above average intelligence.

    Regarding outliers, we used to have a rule at our dinner table that correlation always implied causation, but that rule was not allowed to stand in other times and places. Following up on those who seem to buck the health/wealth correlation does sound like a way to use this data for the pursuit of earthly good.

    I think we've brought up three different ways of looking at this data: one is the "progress seems poised to continue" lens, which seems to be Hans Rosling's view. The second is the deep green perspective, which rightly points out that all these ballooning numbers are horribly unsustainable. The third is one I'd like to start to exploring more in these writings, which is just the observation that these numbers are horribly unfair.