Friday, December 21, 2012

Newtown-inspired data analysis

Let me share with you a truly horrible bit of data analysis.

Inspired by the analysis of 9/11 in Nate Silver's book, I plotted the mean number of years y between school massacres in which x people died, on a log-log scale.  I used the last 50 years of data from a table in Wikipedia (of unknown completeness) and put everything on a log-log plot.  The largest few counts were singletons, so the mean time is shown as 50 on the plot.  Surprisingly, this all fits a line pretty well (r^2 is about 0.57), discounting the singletons.  

Both Newtown and VA Tech fall quite close to the line I drew below (which was done by eye, but looking at the regression coefficients).  For example:  Columbine is a unique case - there is only one event with 13 deaths in the dataset - but there are several cases with 12 deaths; so extrapolating, it looks like we can "look forward" to a Columbine (or worse) every 15-20 years.  We will have a Newtown or VA Tech every 40-50 years.  (So we've had more of these than expected - but not by much).  Killings of 5-10 children can be expected to happen every 10 years or so, and probably sometime in your lifetime you will see one Breivik-scale massacre in a school - with 70-80 dead.

These events are worldwide, not US-only - but on the other hand these are only at schools, not all gun-related killings.  And US policy is very relevant since many of these events (3 of the worse 5, 4 of the worse eight) are here.  

If you've been, like me, feeling a little stunned and avoiding the news lately, be warned: Newtown is horrible, but it isn't unprecedented.  Occurrences like these are not common, but they should not be unexpected. This is just the way things will be - unless we change.

Tuesday, September 04, 2012

Are we better today than four years ago?

It's an interesting question - how would you answer it?  Having recently read Kahneman's Thinking Slow and Fast I suspect most people don't actually answer it very accurately without doing a little work.  For starters, what do you actually remember from 2008?

Here are some ways to tickle your memory:

 There are actually a bunch of news-headline archives if you drill down to a date:  Newseum is one, and there are also archives available from Reuters,  and for blogs, Memeorandum.

Sunday, September 04, 2011

Just give up and fight, already

Some sort of local tipping point as people read about the ozone decision - now even my friends are fed up with Obama.

The first year or so of his presidency wasn't surprising if you'd read his books. If you look at the policies and discount the pretty rhetoric, he's not especially liberal - about where the Clintons are. And sure, he's avoiding the angry black man thing. And, sure he's prioritizing in a bad economy, and so on. And most importantly, he really believes strongly in the importance of re-establishing a center in American politics. And I agree, that's a great goal.

But that ain't gonna happen, and he should have figured this out by now. Instead, he keeps moving rightward, leaving the left stretched thinner and thinner behind him ... and every time he does, instead of moving in to meet him part way, Team Red just scuttles further away to the right.

Friday, February 04, 2011

NELL gets political

Tom Mitchell's "Never Ending Learning System" is set up to Tweet a random selection of facts is extracts from the web. Recently it tweeted "I think "Barack H . Obama" is #American"... poor system doesn't know when it's getting into controversial issues.

(H/T Burr Settles)

Sunday, February 14, 2010

Rightwing bloggers admit: no, we really don't understand statistics?

I have to admit, I'm fascinated by the dynamics of how information is pushed around the Web. The academic work on this seems to mostly ignore that what people link to is dictated by what they want to believe.

Today's Memeorandum is dominated by one story, which started with a BBC interview where Phil Jones took a bunch of questions from climate skeptics. One was this one:

B - Do you agree that from 1995 to the present there has been no statistically-significant global warming?

Yes, but only just. I also calculated the trend for the period 1995 to 2009. This trend (0.12C per decade) is positive, but not significant at the 95% significance level. The positive trend is quite close to the significance level. Achieving statistical significance in scientific terms is much more likely for longer periods, and much less likely for shorter periods.

Rough translation: we're slightly less than 95% confident there is warming in that period. Alternatively, there's a little more than 1 chance in 20 that there is not warming 1995-now.

Then a UK paper, Mail Online, published a report on the interview, with the headline - apparently based solely on the paragraph above - Climategate U-turn as scientist at centre of row admits: There has been no global warming since 1995 - a remarkable interpretation. And this is the discussion on the blogs now, via Memeorandum Colors (click to expand):

One or two of these are pointing out that the Online Mail headline is misleading - but most seem to be just repeating it uncritically ....even though the BBC interview is online, just a google search away, and even though a quick read of even the text Online Mail story shows that the headline is misleading.

Monday, September 14, 2009

Free Markets and Healthcare

After a bunch of Facebook comments back and forth an old friend of mine pointed me to this article...which presents a more market-oriented discussion of possible reforms.

It's interesting, and raises a number of good points. Goldhill's comments on the inefficiency of insurance are, of course, also the selling point for single-payer:

for every two doctors in the U.S., there is now one health-insurance employee—more than 470,000 in total. In 2006, it cost almost $500 per person just to administer health insurance. Much of this enormous cost would simply disappear if we paid routine and predictable health-care expenditures the way we pay for everything else—by ourselves.
Since these costs also go away if the insurance system is simplified. And many of the issues he discusses - IT inefficiencies, study of treatment effectiveness, etc - are in some of all of the current healthcare proposals.

But there some places where Goldhill seems off base - viewing things through a very middle-class eye. The essence of his proposal is savings accounts for health expenses, plus catastrophic insurance with a high cutoff - the current typical $2000 cutoff moved up to $50k or so. But those numbers probably soundsquite different for you and I than for most people - in the US the median household income in the US is about $50k. In fact there have been pre-tax savings accounts for a long time and I don't see they've been much of a help in limited costs.

More fundamentally, I have two misgivings about the approach. Goldhill starts off with philosophy:

based on my own work experience, I also believe that unless we fix the problems at the foundation of our health system—largely problems of incentives—our reforms won’t do much good, and may do harm. To achieve maximum coverage at acceptable cost with acceptable quality, health care will need to become subject to the same forces that have boosted efficiency and value throughout the economy. We will need to reduce, rather than expand, the role of insurance; focus the government’s role exclusively on things that only government can do ... and rely more on ourselves, the consumers, as the ultimate guarantors of good service, reasonable prices, and sensible trade-offs between health-care spending and spending on all the other good things money can buy

Well and good, I suppose, but it's pretty dangerous to start with philosophy rather than actual data about what works and what doesn't. There are indeed some things governments do well, and some things they don't. But a hundred years ago, private fire companies were popular - but the US has long since gone to a government-funded model for that. If you look at the rest of the first world, every single other country has more government involvement than we do--and our current model is hardly free-market, being based heavily on tax incentives with corresponding regulations for employers. And the vast majority spend less and get results that are better than the US on most measurable dimensions.

I believe the flaw in Goldhill's free market argument is illustrated well be his examples. For Lasik surgery, an elective surgery that is seldom covered by insurance, prices have come down over the past few years, in response to market pressures. For his poor Dad's fatal infection, the final bill has an astronomical $600k+ (at least on paper). But people are just not that likely too shop around for the best price in that kind of a medical emergency - due to some combination of pain, panic or unconciousness - so here the hospital is more like a fire company than a Lasik-surgery beautification clinic.

My guess is that expenditures for these sorts of major-illness emergencies swamp the expenses that would be more sensibly allocated with better incentives. (Data, anyone?)

Goldhill admits "Some of the ideas now on the table may well be sensible in the context of our current system" and "These ideas [i.e. in Goldhill's essay] stand well outside the emerging political consensus about reform" other words, he knows he's proposing a fairly radical solution - one that hasn't been tried elsewhere. Honestly, unless you're a philosopher, who would want to take a chance like that? A more rational way of introducing more marketplace into the system would be to try some incremental ways to get more competition, say in a single US state...maybe Massachusetts?...and see how they work before trying them out at scale.

In fact I suspect that a workable government-funded portable the health care plan would be a huge boon to the free market. When a former startup I worked for went bust, I had the delightful experience of needing to find insurance for my family in a hurry (nine days, I still remember) - with no COBRA or any other backup plan to lean on. That experience was been a significant factor in subsequent job decisions I've made. It's a huge distortion of the job market when smaller companies have this sort of handicap to deal with in competing for hires. Pushing for a "free market" approach where it's not appropriate in health - at the expense of biasing the job market against startups and small companies - does not seem like a sensible tack to take.

Sunday, June 07, 2009

CMU ahead of the curve again

I just listened to Obama's speech in Cairo from last week: with all the noise about it, I haven't heard anything posted about what I thought was the most interesting bit:

And this is important because no development strategy can be based only upon what comes out of the ground, nor can it be sustained while young people are out of work. Many Gulf states have enjoyed great wealth as a consequence of oil, and some are beginning to focus it on broader development. But all of us must recognize that education and innovation will be the currency of the 21st century -- (applause) -- and in too many Muslim communities, there remains underinvestment in these areas. I'm emphasizing such investment within my own country. And while America in the past has focused on oil and gas when it comes to this part of the world, we now seek a broader engagement.

On education, we will expand exchange programs, and increase scholarships, like the one that brought my father to America. (Applause.) At the same time, we will encourage more Americans to study in Muslim communities. And we will match promising Muslim students with internships in America; invest in online learning for teachers and children around the world; and create a new online network, so a young person in Kansas can communicate instantly with a young person in Cairo.

On economic development, we will create a new corps of business volunteers to partner with counterparts in Muslim-majority countries. And I will host a Summit on Entrepreneurship this year to identify how we can deepen ties between business leaders, foundations and social entrepreneurs in the United States and Muslim communities around the world.

On science and technology, we will launch a new fund to support technological development in Muslim-majority countries, and to help transfer ideas to the marketplace so they can create more jobs. We'll open centers of scientific excellence in Africa, the Middle East and Southeast Asia, and appoint new science envoys to collaborate on programs that develop new sources of energy, create green jobs, digitize records, clean water, grow new crops. Today I'm announcing a new global effort with the Organization of the Islamic Conference to eradicate polio. And we will also expand partnerships with Muslim communities to promote child and maternal health.

This was the closing portion of the speech, so maybe everyone else flamed out with indignation at something or other before it...but it was also by far the most concrete proposal. Sounds like CMU in Qatar is way ahead of the curve here.....