The scientific evidence for an advantage for Oscar Pistorius

The second research study - what it really found about Oscar Pistorius

We may as well jump straight in with the continuation of the scientific summary of the research done on 400m sprinter Oscar Pistorius.  Yesterday, I described how the first round of testing, done at the request of the IAAF in Germany, found that Pistorius used 25% less oxygen during a simulated sprint, and that his running mechanics were vastly different to those seen in able-bodied runners.

That study led the IAAF to ban him from competing, and resulted in the appeal to the Court of Arbitration for Sport.  That brings us to the second research study, led by Hugh Herr, and, at the time, Peter Weyand.  Below follows an explanation of a key part of the paper they produced a few months AFTER the CAS hearing - the oxygen uptake question, which I introduced as being important in yesterday's post.  The timing is important because one of the first key points about the CAS-process is that the research was not peer-reviewed until after the decision was made, which contrasts with claims that the decision was based on peer-reviewed research.

With respect to the CAS hearing, there were without doubt procedural errors in the IAAF process of the scientific testing in Germany.  These are legal issues, pertaining to things like the timing of making documents available, inaccurate reporting of data in an IAAF summary, and dictating testing conditions.  This procedural dispute led the CAS to declare that "the manner in which the IAAF handled the situation of Mr Pistorius...fell short of the high standards that the international sporting community is entitled to expect from a federation such as the IAAF".  Unfortunately, it wouldn't be the first time that this has happened for a major sporting federation...

However, this is a look at the science, following on from yesterday, where I explained the specific scientific issues that were taken to the CAS panel, along with responses by Bruggemann and a few quotes from other biomechanists, as well as my views and interpretation of those points.

Today, I move on to the Herr research, which I believe is equally flawed.  But I'll go through it systematically, building up the main findings and figures, and then allow you to draw conclusions about what was done to come up with this overall finding: "We conclude that running on modern, lower-limb sprinting prostheses appears to be physiologically similar but mechanically different from running with intact limbs." (Weyand et al, 2008)

The metabolic cost of running - similar?

To begin with, one of the key points challenged at the CAS hearing was the measured reduction in oxygen use during a simulated sprint (figure shown to the right).  The problem is that during a sprint, metabolic energy is provided by a combination of oxygen dependent and independent sources, so Bruggemann provided only half the picture.

You would not be surprised to learn that in the CAS hearing, the IAAF focused on the biomechanical aspects of their research, and downplayed this particular finding, because their conclusion based on the measurement of oxygen use in a sprint was always going to be challenged very strongly.

And challenged it was - it would also not surprise you to learn that the response from Pistorius' research was to repeat the above test, but this time to measure oxygen use at sub-maximal speeds, because this provides a better indication of the metabolic cost of running.  And so this is what they did - Pistorius ran on a treadmill at a range of SLOWER running speeds.

The method used was to have Pistorius run for 5 to 7 minutes at a range of speeds, with 3- to 5-minute rest periods.  The starting speed was 9km/h (a very slow jog) and Pistorius reached exhaustion (unable to complete 5 minutes) at 15km/h (which is very slow, tellingly - Pistorius did this testing at a time when he publicly announced that he was untrained and unfit.  That this impacts on the results you'll see below is important).

Using measurements of oxygen use from Pistorius during these runs, they were able to calculate what is called running economy, or the volume of oxygen used per kilogram per kilometer. Think of this as fuel use in a car - the less oxygen you use per kilometer, the more economical you are.  Remember that the hypothesis based on one interpretation of the theoretical knowledge at the time would be that Pistorius would use less oxygen, and hence less energy, at a given speed because of the reduced mass and the increased energy return of the carbon fiber limb (see yesterday's post for the explanations).

(By the way, if you adopted the exactly opposite position based on the theory, then you might hypothesize that Pistorius would use MORE oxygen and more metabolic energy because of the demand for balance and the increased work required to control the limbs - this is what his coach, Ampie Louw, explained in a TV interview in 2007 for a UK news insert.  The point is that either theoretical claim can be tested by measuring oxygen use - it is a barometer for energy use, and will either be similar, in which case the theory is disproven, or will be different, in which case one of the theories will be proven, the other disproven.)

There are a few pertinent considerations here.  First, the Cheetah blades are built for speed (the manufacturer's own claim), not slower running, and so I would argue that testing them at these slow jogging speeds (9km/h is very slow) will provide a false picture, skewed in favour of finding a higher oxygen use.  However, even without that potential confounder, the data still provide some really interesting insight.

What is more crucial, for reasons that will emerge as we progress, is the vital question of who do you compare Pistorius' data to?  The answer must surely be able-bodied sprinters of similar performance levels.  That's because the literature shows that sprinters use more oxygen (or are less economical) than distance runners.  So the "control" group would have to be able-bodied SPRINTERS.  And this is where I begin, with four able-bodied sprinters compared to Pistorius (note - the development of the graphs below (which is Figure 2B in the research paper) does not necessarily follow the timeline of the testing, but it builds the argument and highlights the key issue)

The first graph, shown below, reveals the first finding - Pistorius compared to able-bodied sprinters at the same performance level:

So, Pistorius uses 17% less oxygen than the able-bodied controls, or 2.7 Standard Deviations less.  Without going into the stats, that's a big difference - generally, anything more than 2 SD is considered an "outlier", very different.  And so on the basis of this finding, Pistorius is very different to able-bodied sprinters.  This is not different to the 25% difference that Bruggemann found during sprinting, and it's not inconceivable that the 17% would be higher (approaching 25%, maybe?) at sprinting speeds, where the Cheetah blades are more effective, doing what they were built to do.  This first finding confirms the Bruggemann research.

Yet the paper concluded, you'll recall, that Pistorius is physiologically similar, and Hugh Herr was recently quoted as saying "It is not true that Oscar uses less oxygen than a person with two biological legs, although he is very economical".

The addition of the distance runners

So how do you get to that conclusion?  Answer - you add distance runners to the able-bodied population. Herr and co decided that in order to increase the size of the control group, the next group that should be compared to Pistorius was not more sprinters, but rather a group of sub-elite and elite DISTANCE runners, who had been measured in a study in 1995, thirteen years earlier in a completely different laboratory.

Once these distance runners are added, the following graph can be drawn:

So, with the addition of the distance runners to the control group, the gap between the sprinter Pistorius and the able-bodied runners is coming down - presumably this is a "good finding" if you're involved in this research and "want" to disprove the earlier IAAF study finding.

The difference in oxygen use between Pistorius and sub-elite distance runners was 6.7%, while Pistorius uses 3.8% LESS oxygen than elite distance runners.  However, statistically speaking, the differences are 0.8 SD and 1.3 SD to those groups, respectively, and so it can now be concluded that Pistorius is, in fact, not different to able-bodied runners.  Provided those able-bodied runners are elite distance runners, that is.

For good measure, they also add John Ngugi and Zersenay Tadese to the sample, and the graph is now complete, as found in the research paper (Figure 2B on Page 906):

Just a word on the Tadese value shown above: this is a study that I actually wrote quite a bit about back in 2007, and the main reason was because Tadese was the most economical runner in history, but there was something not quite right with that running economy value.  His economy, measured at 150 ml/kg/km has some bizarre implications - it means that running at world record pace for the marathon (2:56/km), Tadese would be using only 51 ml/kg/min of oxygen.  Take it from me, that's just not possible, especially when you consider that Tadese's VO2max was supposedly 83 ml/kg/min - his marathon world record pace would be an easy jog!  My point is - something didn't add up, and I'd be very cautious about accepting that Tadese value.  And this illustrates just why you should not take data from other labs and use it as part of your data set as was done - you simply cannot guarantee the validity of that data.

To do this, to borrow from other research, where methods are different, equipment is different, and athletes are different, is to violate a key concept in the control of science.  It's the same thing as if I want to test the effects of something like compression socks on muscle pain after running, and I make a few runners do a trial, and then compare them to a study done in a different lab with different methods.  Certainly, you will use other studies to explain your data and provide context in the discussion, but to borrow data like this is extra-ordinary.

Having said this, I can appreciate why it was done - the time was limited, and the research had to be done for the CAS panel.  The control group was small initially, and had to be increased in size.  So given the time constraint, I can appreciate that this would be an exceptional circumstance.  But the question is this:  Why compare a sprinter to a distance runner in one of the key variables (running economy) that is known to be very different between them, when data on sprinters exist for a more valid comparison?  At the very least, show BOTH distance runners and sprinters...

Why compare to distance runners, when data for sprinters is available?

The question is so important, I repeat it - why would Herr and co have chosen to compare Pistorius to elite and sub-elite DISTANCE runners, when the data exist for sprinters' running economy?  It's not as though we don't have the data for sprinters - it exists.  But the research chose to ignore these data, and instead focus on distance runners, both elite and sub-elite.  Why?  I hope it's becoming clear in your minds as you read this.

But before continuing, take a look at this sentence from the research article, in the discussion (pg 909):

"However, his values were 17% (2.7 SD) lower than those of the intact-limb 400-m specialists tested here and two or more SDs below the means reported for four other groups of subelite male sprinters"

In other words, the authors are aware of FOUR studies on sprinters that could have been used to expand on the subject numbers used in the very first graph I showed above.  Four studies that would have increased the size of the able-bodied control group and allowed a VALID comparison of a sprinter (Pistorius) to other sprinters, rather than elite distance runners.

One acknowledged consideration is that the sprinters in these studies are not elite, but then neither are most of those in the distance group used in their place.  The sprint data could, at worst, have been included in the analysis.  But instead, they chose to ignore the sprint data on athletes who were, at the time, at more or less the same performance level as Pistorius.  Rather, they presented the distance data, which conveniently supported the ultimate finding of physiological similarity.  Perhaps the comparison with other sprinters produces a finding that you'd rather not show...

The graph that compares sprinters to Pistorius - the real finding and comparison

So I went and found the four studies in question, and I've redrawn the graph, this time showing the oxygen use of Pistorius compared to other sprinters and middle-distance runners, rather than elite distance runners:

So, that graph looks a little different to the one shown above, the one that was eventually published in the research paper (Fig 2B), comparing the sprinter Pistorius to the elite and sub-elite distance runners.  Here, when you compare Pistorius to sub-elite sprinters, you get a very different picture.  I've shown in red below each bar the difference in percent between Pistorius and these sprinters, and in black, the difference in Standard Deviations.

So 17% or 2.7SD is the Weyand et al finding (comparing Pistorius to performance matched sprinters).  13% or 7.2SD is the difference to 400m sprinters (performance time 50s). It's 3.2 SD to 800m runners, 5.1 SD to 1500m runners, and overall, Pistorius uses 14% less oxygen than able-bodied sprinters and middle-distance runners.  That's 2.3 SD and a difference that, according to the definition set out by Herr, would have led to the conclusion that "Pistorius is physiologically different"  And importantly, it's a difference that strongly supports the hypothesis of advantage.

Again, these are sub-elite athletes (50sec 400m time, for example), but they do include middle distance runners who would be expected to have better economy than sprinters (remember, Pistorius is a 200m-400m sprinter), and they are close to the performance level achieved by Pistorius in the actual testing protocols (they actually exceed his level - remember, he hits a peak sustained speed of 15km/hour - some of the above runners were measured at 20 km/hour).

Conclusion - look at what is included as well as what is not

So the point in all this, I would hope, is obvious.  The selective addition of distance runners to the able-bodied group allowed the physiological differences to disappear (statistically speaking).  Meanwhile, selective omission was happening - there were at least four studies on sprinters that would have confirmed the Bruggemann finding, and even the Weyand 2008 finding that Pistorius has significantly different physiology and metabolic cost compared to other sprinters.  This would go a long way to confirming the initial hypothesis, and so it's not a trivial matter at all - it is in fact the crux of the issue.  The question is why make this selective addition and omission?

One answer may provided by the background to the research.  Pistorius has the Bruggemann research, he knows the methods and the results, and knows why he has been banned.  He now has to cast doubt on the Bruggemann research, showing the CAS that there is insufficient scientific evidence to ban him.

The first point to challenge for the CAS hearing, arguably the "weakest point" in the IAAF case, was the issue of energy use, or metabolic cost of running.  So the researchers Pistorius works with, one of whom is a big recipient of funding from Ossur, who make Pistorius' blades, are tasked with disproving the Bruggemann findings.  That means one thing - show that Pistorius uses similar volumes of oxygen compared to able-bodied runners.

Unfortunately, he doesn't.  At least not compared to sprinters, who train like he does, who share the genetic make-up of being a sprinter rather than a distance runner - he's 17% different despite everything being in his favour - he's untrained, he's running at slow speeds on equipment designed for higher speeds.  I dare say that three months of training would reduce Pistorius' values even further.

But when distance runners are added, the control group changes sufficiently so that even though Pistorius uses less oxygen and energy than them, he is statistically similar, and it is concluded that he is physiologically similar.

CAS accepted this (which defies belief, but anyway), and so the key barometer in the debate, the key variable that allows one to evaluate whether the theory is correct regarding the reduced energy cost of running on the carbon fiber blades, suddenly appears questionable.

Go right back to the hypothesis - either Pistorius has a metabolic advantage because of the reduced mass and the elastic energy return, or he doesn't.  Measure the oxygen used, and it will decide for you - the measurements of oxygen used confirm the metabolic advantage - across a range of speeds, from barely a jog to a 400m race, Pistorius uses significantly less oxygen than able-bodied sprinters.  Even dipping into previously published literature confirms this.  My take - hypothesis proven.

The mechanical debate and the "10 second advantage"

There is still the issue of mechanical advantage.  But it's late here in SA, and that is an issue for another day, when I'll look at the split that occurred between Pistorius' own scientific team, leading Weyand to publish a letter to JAP suggesting a 10-second advantage.  That is the final piece of the puzzle.

Thanks for the comments to yesterday's piece, it's good to inspire debate and as mentioned, in all this, there's allowance for disagreement.  I mean, we can't even agree on something as basic as whether dehydration causes heatstroke, I don't expect consensus on this.  But I've seen more ambiguous data, that's for sure - this is, I suggest, pretty clear cut.

Ross

Tweet

Jangan sampai ketinggalan postingan-postingan terbaik dari Sport Blog Mas. Berlangganan melalui email sekarang juga:

Atau sobat juga bisa follow Sport Blog Mas dengan mengklik tombol di bawah ini:

Artikel keren lainnya:

Ditulis oleh reyhan iki pada tanggal Selasa, 23 Agustus 2011