I have just come across a striking but slightly scary paper. The paper looked at errors in scientific papers reporting on the use of stem cells to treat heart problems. Our heart is a muscular pump, and its muscle tissue has very little regenerative capacity*, so a heart attack can damage it permanently. The idea here is that by transplanting stem cells into a damaged heart, the stem cells can partially regenerate damaged muscle tissue, and so reverse damage due to the heart attack. The repair can be measured by what is called the ejection fraction (EF), which measures basically the fraction of the blood inside a heart chamber that is pumped out in one heart beat.
Heart disease is a major cause of illness and death and attacking it with stem cells is a cute idea. So a number of studies have been done. Nowbar and coworkers identified 133 papers (on 49 trials). They then carefully looked at each paper, looking for inconsistencies. These included data problems, such as misreporting of patient details such as gender and treatment protocols, and stats problems such as reported standard deviations that are incompatible with the data. They found a lot, which is a bit worrying.
But the finding I find eye opening is the striking correlation, shown above**, between the size of the improvement in the ejection fraction (EF), and the number of discrepancies in the paper. In particular, the papers without discrepancies report that the stem cells essentially have no, or a very weak effect, while the largest effect is seen in those studies with the most errors.
Of course this is only correlation, but it does make you wonder if there is any real medical improvement by using the stem cells, when apparently the more careless are the medics and scientists, the bigger effect they see.
* Interestingly, fish and some other animals are different here, if your gold fish has and survives a heart attack, they may be better replaced to recover than you in similar circumstances.
** Above figure is Figure 3 of Nowbar et al‘s 2014 paper in the BMJ. The paper is open access.
Chance & Necessity 