error by design?
Old news, perhaps, that companies should seek to misrepresent data – sometimes it seems easier to accept the default position that all large companies are lying. But it’s important to try to remain objective, I guess.
Anyway, to get to the point of this, in 2007 and again in 2009, Boots got major media attention for one of its cosmetic products, No. 7 Protect Perfect Beauty Serum (Google has lots of links). Especially following a BBC Horizon programme extolling the scientific virtues of the product.
In a short and elegant article in the latest copy of Significance Martin Bland examines the most recent evidence published in support of this product trial. Oddly enough this the trial paper is the only one in that issue of the British Journal of Dermatology to be Open Access (ie available to everybody).
Bland reports basic methodological and statistical oddities and errors.
- Item 1: the control group study lasted only 6 months, and active treatment was substituted in this group for the next 6 months.
- Item 2: the non-significant differences between control and active groups at 6 months did not appear to include data from drop outs in the control goup, which might skew the results.
- Item 3: Nor were comparisons made directly between groups. Instead significance tests were done in each group and then the results compared. That’s a no-no, apparently.
- Item 4: no adjustments were made for multi-testing (of outcomes that were in any case not pre-defined) at 12 months.
Bland concludes that the data do not support the story. Also that science, used as what he calls a ‘brand’ or an ‘endorsement’ can have major influence and that journalists, even those in the public eye, do not understand statistics, especially the use of P values. Which of course Boots and other such companies know, very well.
It seems only fair to point out that since it’s first online publication Watson et al have corrected one of Bland’s gripes, that sources of funding for the study were not mentioned, with a statement that ‘This study was funded by Alliance Boots Ltd’.
As if we hadn’t guessed.
Reference
Bland M. Keep young and beautiful: evidence for an ‘anti-aging’ product? Significance 2009, 6, 4: 182-183.
Filed under: news, statistics | Leave a Comment
Tags: brand, cosmetics, data, Google, No. 7 Protect Perfect Beauty Serum, P values, pharma, publication, science, statistics
No Responses Yet to “error by design?”