Skip to main content
SpringerLink
Log in

Research Fundamentals

  • Chapter
  • First Online:
More Harm than Good?

Abstract

Research is the gathering of data, information and facts for the advancement or completion of knowledge. By adding to the store of human knowledge, scientific research has great intrinsic value. Research also has substantial practical value, in the guise of beneficial technologies flowing from such knowledge.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
eBook
USD 14.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 22.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Although frequently referred to in the singular, the ‘placebo effect’ in fact comprises several distinct components.

  2. 2.

    For example, suppose an ancestral human eats a toadstool and a few hours later is violently sick. If this experience makes her to believe that the toadstool caused the vomiting, it will aid her survival by helping her avoid a possible source of poisoning in the future. This pattern-recognition mechanism ought to be error-prone in the direction of making false assumptions about causality: if the toadstool is not actually toxic (and the sickness actually resulted from some other unseen cause), the resultant false knowledge carries some cost (in terms of the erroneous avoidance of a potential food source), but this is unlikely to greatly threaten survival. By contrast, if the cause-effect link fails to be made in the case of a genuinely toxic toadstool, the cost is likely be much higher: if more toadstools are eaten, the effects could be lethal. Thus, the genetic sequences underlying the error-prone cause-and-effect heuristic are transmitted to the following generations through natural selection. It is clear that such pressures have influenced the evolution of the human brain such that we have a strong inbuilt tendency to make error-prone assumptions about causality.

  3. 3.

    For example, see Nuzzo (2014), and Schwalbe (2016).

  4. 4.

    There is ongoing academic debate on the extent to which the medical literature is corrupted with false findings. For example, a recent survey of major (mainstream) medical journals claimed that the false positive rate is 14%—a high rate but one that is less than the claim of ‘most’ published findings being false (Jager and Leek 2014). However, Ioannidis and other academics have repudiated this claim of 14%, pointing to various flaws in the paper in terms of sampling, calculations, and conclusions, and pointing out that it uses only a very small portion of select papers in top journals (Ioannidis 2014; Benjamini and Hechtlinger 2014).

  5. 5.

    In the context of observational studies (as opposed to clinical trials), the equivalent term is false discovery rate, in the context of the problem of multiple comparisons.

  6. 6.

    There is debate amongst statisticians as to whether p < 0.05 or p = 0.05 is the better interpretation. The former is used by many papers on medical statistics, however it is arguably less realistic than the latter, which tends to generate higher false positive risk values. An in-depth exposition of this subtle but important distinction is beyond the scope of this book; see Colquhoun (2017) for more detailed discussion.

  7. 7.

    It is ethically questionable to conduct such small-scale RCT s, because they are inherently underpowered and thus prone to generating misleading results. Moreover, because effect size and sample size are interrelated, small samples can lead to overestimations of effect sizes. However, in some specific cases the determination of effect size can be aided by data from such trials.

  8. 8.

    This will only be true if the power calculation has been valid.

  9. 9.

    This is optimistic because, as discussed elsewhere in this book, the alleged specific effects of acupuncture are supposedly due to completely implausible physiological features and mechanisms, including Qi (‘vital energy’) flowing through meridians (body channels), none of which have been discovered by science and all of which are implausible.

  10. 10.

    Various ways exist to calculate false positive risk; the values that result vary according to methodology, but all valid approaches yield risks that are substantially greater than the 5% assumed by the common but disastrously wrong assumption that p = 0.05 equates with a 5% false positive risk. For example, in simplified terms, we can compute the expected false positive risk for p < 0.05 by: [a] multiplying the sample size by the prior probability, then multiplying the proportion of the sample with a real effect by the power value, to establish the number of true responders; then [b] multiplying the proportion of the sample who are expected to have showed no effect with the threshold p-value (0.05) to establish the expected number of false positives; and finally [c] expressing as a percentage the number of false positives from the overall number of positive results. For the example given above, this computes to 86%. Note that for p = 0.05, using the methodology used by Colquhoun (2017), the false positive risk computes to be even higher, at 97%.

  11. 11.

    An alternative way of expressing this is to say that if you observe p = 0.05 then, in order to achieve a false positive risk of 5%, you would need a prior probability of 87%—clearly preposterously high (Colquhoun 2017).

  12. 12.

    The numbering is ours, and the wording of #2 had been adapted slightly, to remove reference to ‘business or policy decisions’.

  13. 13.

    This example is chosen to illustrate an inherently absurd modality, recognisable as such to all reasonable people. Sadly however, proponents of such ‘intercessory therapeutic prayer’ exist; indeed, some of them have even conducted ‘clinical trials’ into this form of CAM (Roberts et al. 2009). We shall consider one such real-life case later in the next chapter.

  14. 14.

    An online statistics tool was used to calculate this sample size (ClinCalc LLC 2017).

  15. 15.

    This was calculated using a z-test; other suitable tests exist, but all of these will approximate to this value.

  16. 16.

    There is an interesting ethical side question here: should subjects who cannot feasibly be affected in any way whatsoever by a remote experiment (such as intercessory prayer) be required to give consent?

  17. 17.

    The statistical analysis of such combined data is referred to as ‘meta-analysis’, and some systematic review publications are titled as meta-analyses.

  18. 18.

    For example, a search of the academic publications database Web of Science (which includes all the major CAM journals) covering 2007–2017 using various relevant search terms ( p-value , reproducibility crisis, p-hacking) led to merely one CAM paper dealing with the issue of problematic statistical interpretation of clinical data (Benbassat 2016).

References

Download references

Author information

Authors and Affiliations

  1. University of Exeter, Exeter, UK

    Edzard Ernst

  2. School of Science, Engineering and Technology, Abertay University, Dundee, UK

    Kevin Smith

Authors
  1. Edzard Ernst
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kevin Smith
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Edzard Ernst or Kevin Smith .

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Ernst, E., Smith, K. (2018). Research Fundamentals. In: More Harm than Good?. Springer, Cham. https://doi.org/10.1007/978-3-319-69941-7_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-69941-7_2

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-69940-0

  • Online ISBN: 978-3-319-69941-7

  • eBook Packages: MedicineMedicine (R0)

Publish with us

Policies and ethics

Search

Navigation