National Center for Complementary and Integrative Health (NCCIH)
NIH…Turning Discovery Into Health

Información en Español

Health Topics A-Z

NCCIH Research Blog

On Scientific Plausibility

September 10, 2012
John Killen
John Killen, Jr., M.D.

Deputy Director
National Center for Complementary and Alternative Medicine

View Dr. Killen's biographical sketch

Scientific plausibility permeates discussions and debates about research on complementary, alternative, or integrative health approaches. This is no surprise; many interventions that fall under this rubric are ensconced in belief systems about illness and health—some ancient and some modern—that lack foundations in modern science. In addition, those who support research on these approaches often fail to articulate a scientifically grounded rationale or approach to research. Thus, it is common to see criticism based on scientific plausibility in the scientific literature, news stories, and blogs.

This criticism often suggests that the existence of research implies either belief in scientifically implausible explanations or ignorance of basic scientific principles and concepts. So how do we justify investment of public resources in research on complementary interventions that are associated with pre-scientific or unscientific explanations? Simply, it is both possible and necessary to disconnect scientific interest from unscientific “trappings.” For example, an objective look at the body of accumulated evidence (from patient reports, clinical observations by many good clinicians, and clinical studies) suggests that some people with chronic low-back pain are deriving meaningful benefit from acupuncture, yoga, or procedures involving spinal manipulation. It is entirely possible to be scientifically curious about that body of evidence and investigate it further, while not in any way embracing scientifically unfounded explanations for those practices. For instance, it is not necessary to believe in meridians or qi to study the effects of the procedure of acupuncture on pain, or to explore the hypothesis that acupuncture mediates pain by conditioning or expectancy effects produced by a convincing ritual combined with a counter-irritant.

NCCAM’s strategic plan now includes a framework of four factors we use to sharpen the focus of our research investments. Two of them address important aspects of scientific plausibility. One is “scientific promise”: how strong is the body of evidence supporting the concept? In the case of acupuncture, yoga, or spinal manipulation for chronic back pain, credible signals from a variety of clinical sources provide a sufficient basis for interest in further research. There is no need to bring associated non-scientific explanations into consideration of scientific promise. By contrast, unscientific notions should assume increasingly greater importance when clinical signals are weak, unconvincing, or non-existent.

In all cases the question of whether and how to invest valuable resources in research must also move to consideration of the second factor—“amenability to rigorous scientific inquiry.” Do we have reliable and reproducible tools, methods, diagnostics, outcome measures, quality control processes, etc., to allow us to mount a study that will elucidate a clear and unambiguous answer? In some cases we do. For example, it is very possible (although not necessarily easy) to design a study that employs the most rigorous mainstream clinical research methodology to investigate whether acupuncture, spinal manipulation, or yoga alters a patient’s low-back pain. On the other hand, it is not possible to design studies that will yield clear and unambiguous answers when objective, validated measurement tools, or processes and procedures to ensure and document quality control, are lacking.

Let me be clear that I do not mean to suggest that we can or should launch expensive clinical trials of anything and everything complementary or alternative or integrative just because we have some intriguing anecdotes. The strength, reliability, reproducibility, and other particulars of the signals from clinical observation and preliminary clinical investigations are critically important. Adequate methods and tools are equally important. I also do not mean to suggest in any way that mechanisms are irrelevant to questions of plausibility. One lesson NCCAM has learned from extensive experience is that mechanistic insight into biological effects creates sharper scientific hypotheses and allows one to design better clinical trials to investigate those hypotheses. I simply mean to suggest that it is a mistake to assume that scientific inquiry is equivalent to acceptance of unscientific mechanistic thinking.

NCCAM’s first decade entailed a relatively broad and investigator-initiated approach to funding. This was appropriate to the time and the state of the available scientific evidence. The four factors we now consider evolved out of lessons learned during those years. So with the benefit of hindsight, it is pertinent to note that a number of studies funded during that timeframe would probably not be funded today because they could not pass our current hurdles regarding plausibility. In fact, the portfolio of research NCCAM has actually funded over the past several years demonstrates clearly that both the peer review process and NCCAM are now using these factors to shape our investments in research.

So plausibility matters a great deal, but it is a mistake to equate interest in research with acceptance of unscientific trappings. By the same token, I urge those who support research on these interventions to carefully parse rationale from “trappings” and give due recognition to the validity of concerns about scientific plausibility.


Comments are now closed for this post.

Why not drop the nonsensical “trappings” altogether? For instance, why not study *counter-irritation* – e.g. a plausible hypothesis for acupuncture – and not mention acupuncture and its fanciful meridians for which there is no scientific plausibility? The funding of studies of *acupuncture* gives credence to the fanciful trappings. Labeling such a study as being about “acupuncture,” any discovered effect from counter-irritation would give undue credence to fanciful meridians and the entire practice of acupuncture. It should be the responsibility of researchers to “articulate a scientifically grounded rationale” and for NCCAM to demand it. / If NCCAM doesn’t go along with all the “trappings” of acupuncture, could not its projects just as well be studied in other NIH centers who are more familiar with pursuing scientific rationales? Isn’t it the very “trapping” that justify NCCAM’s existence, i.e. that some mysterious mechanism is being overlooked? If NCCAM is to be unique, it needs to get down to brass tacks of its mission, i.e. study the *uniqueness* of acupuncture – the meridians themselves. Do meridians exist or not? The “human energy field” – does that exist? “Homeopathic memory” – does that exist? But Dr. Killen appears to admit that these “trappings” do not pass the plausibility test. Therefore, would not medical ethics conclude that the existence of NCCAM itself does not pass the justifiability test?

It is encouraging to see NCCAM explicitly recognizing that concerns about scientific plausibility are legitimate and should figure into the allocation of scarce public research dollars. However, the current policy still seems to ignore a couple of problems.

For one thing, it is rare for even an intervention that has a plausible hypothetical mechanism and supportive pre-clinical research data to live up to its promise in clinical trials. That is why pharmaceutical research is so expensive and time-consuming. The chances of a therapy proving to have a significant impact of health and well-being when it has only unscientific “trappings” and anecdotal experience to support it can only be even less. NCCAM has an ethical duty to patients and to taxpayers to maximize the chances for a meaningful return on the intellectual and financial capital it spends on research. As Dr. Killen suggests, past policies have not fulfilled this duty, and such capital has been squandered on interventions extremely unlikely to bear fruit given their incompatibility with current scientific knowledge. A true change of direction, in which systematic and rigorous development of appropriate evidence at the level of basic science and preclinical research is a prerequisite for launching clinical trials is necessary to truly utilize our healthcare research resources efficiently and effectively.

Secondly, while investigating a therapy scientifically in the absence of an established, scientifically plausible mechanism may not be equivalent to accepting the unscientific theories or traditions that historically have been used to legitimize the therapy, such research unquestionably gives the impression of scientific legitimacy and the NIH imprimatur to these interventions. And there is preliminary evidence to suggest that, when such research is done, negative results do not diminish the use of CAM therapies whereas equivocal or weakly positive results convey an inaccurate impression of scientific validation (Offit PA. Studying complementary and alternative therapies. J Amer Med Assoc 2012;307(17):1803-1804.) It is crucial, then, if NCCAM wishes to conduct a truly legitimate scientific investigation of CAM therapies, that it emphasize the importance of coherence and consistency in the research evidence and the need to accept the failure to validate a practice after sufficient efforts to do so as reasonable grounds for abandoning the practice. While it may not be possible to entirely avoid conveying the impression that an intervention is legitimate because NIH has funded research on it, NCCAM can help to discourage this interpretation by actively endorsing a stricly science and evidence-based approach to clinical practice.

To add to Brennen McKenzie’s point and expound on the example offered by Dr. Killen:

It is indeed true that “the existence of research [does not necessarily imply] either belief in scientifically implausible explanations or ignorance of basic scientific principles and concepts.” And in the case of acupuncture “it is not necessary to believe in meridians or qi to study the effects of the procedure of acupuncture on pain, or to explore the hypothesis that acupuncture mediates pain by conditioning or expectancy effects produced by a convincing ritual combined with a counter-irritant.”

However, when one looks at the TCM definition of acupuncture, at how it is practiced, and what the curricula of acupuncture schools actually are, then one finds that “ritual” and “counter irritant” are not the basis of the actual practice or definition of acupuncture. In science it is vitally important to define variables and conditions precisely. If the argument, Dr. Killen, is that the NCCAM researchers do not believe in the magic of qi and meridiens, but are instead studying counter irritant effects and psychological responses to ritual then it seems quite obvious that they should be studying counter irritant effects and the psychological responses to ritual - NOT acupuncture.

The issue at heart of all this is that using the term “acupuncture” in the study in the manner described above sets the supposed definition of the term the researcher is using as different to the much more commonly used and practiced definition. If I were to grant you that study of it does not mean belief in scientifically implausible explanations, the only thing you have achieved is to demonstrate a support for sloppy scientific inquiry.

There is a well established body of evidence demonstrating placebo effects and responses to pain and how ritualistic medicine takes advantage of this higher order modulation of the pain response and interpretation pathways. You look for prior plausibility and explanations, and even admit that acupuncture is at best a placebo ritual that takes advantage of these well documented phenomena (with perhaps some counter irritant effects, which is another discussion entirely) and yet still advocate that the study of *acupuncture* rather than ritual and counter irritant effects is warranted and reasonable. I fail to see how this is so.

As I commented on Dr. Briggs’ post - what is it that separates the investigation of CAM from any other medical investigation? If we are to take into account prior plausibility and apply full scientific rigor then we would strip away the term “acupuncture” and study the parts already evinced as being in play and merely have plain old medical research. How is the NCCAM any better equipped to study CAM in this manner than the NIH? What specific utility does having this separate department bring to the progression of medical knowledge and understanding?

Once again these are genuine questions since I have explored this topic extensively and rigorously and can find no satisfying answer though I am always willing to be presented with rigorous evidence otherwise.

It seems that Linda Rosa beat me to the punch on this one, though her comment posted after mine despite being dated earlier.

Ignoring research studies that have documented low-resistance “electropermeable” points and pathways correlating with acupuncture points and meridians seems either misinformed or disingenuous. The presence of such low resistance electrical pathways and alterations in the flow of charge along these pathways does indeed offer a scientifically plausible theory for some of the effects of acupuncture beyond the usual endorphin, counter-irritant or psychologic explanations. I agree completely that good science should not be saddled with metaphysical baggage or other trappings rooted in non-verifiable belief systems. If there is a willingness to translate non-scientific terminology and thinking into something that can be measured and tested then we may actually be able to learn something useful about underlying mechanisms of action. Wrong or seemingly fanciful explanations for observable phenomena doesn’t mean it’s not worth studying those phenomena or make them any less real. The jaded, therapy-bashing, belief-bashing, NCCAM-bashing comments serve no purpose. I agree completely with Dr. Killen’s comment that “it is a mistake to equate interest in research with acceptance of unscientific trappings.” There needs to be enough intellectual curiosity to get past the unscientific trappings so we can explore what’s scientifically plausible.

Firstly, I would be interested in those studies you reference Mr. Hammerly.

Secondly, how does one reconcile those studies with the large corpus of studies demonstrating that the effect of “acupuncture” is the same regardless of whether the needle actually penetrates the skin, whether the needle is placed in a “correct” location or just any random location, or whether a needle is used at all? If your argument is that these “low electrical resistance pathways” are plausible explanations for an MOA specific to acupuncture, then one should find an INCREASED effect when needles are placed in the proper spots, a DECREASED (or zero) effect when a sham needle (which does not penetrate the skin) is placed in the proper spot, a DECREASED (or zero) effect when the needle is placed in a random spot, and ZERO effect when nothing more than a toothpick is twirled over the skin. Yet that is not what we see. We find that in each of those scenarios the effect is indistinguishable. So I am very highly skeptical that there is actually rigorous evidence demonstrating these “electropermeable” points corresponding to acupuncture points and meridians. Especially because meridians seem to correspond quite closely with the typical distribution of veins and the historical tools and use of acupuncture was nearly identical to the old European traditions of blood letting. It was not until well into the 20th century that some semblance of acupuncture as we think of it today came into existence.

Furthermore, reliance on basic science plausibility is more than just proposing some highly hypothetical putative pathway of action. If you argue that such “low resistance electrical pathways” (I’ll just call it LREP for short) exist - and for the sake of argument I will grant that they do - you must then follow that up with how that would be physiologically useful and how an acupuncture needle would affect that pathway AND how affecting that pathway would affect disease. Demonstrating that people have LREPs is not merely enough. One would reasonably expect that difference in tissue composition, electrolyte concentration, and surrounding tissues would invariably lead to *something* being an LREP. But are these LREP’s the same in everyone or are they randomly distributed? Do they correspond to useful physiological pathways or not? How can inserting a needle so small as to be imperceptible affect such an LREP? And most importantly is this effect actually clinically useful?

When taken as a whole, the corpus of data does clearly demonstrate that the prior plausibility of any specific effect of acupuncture is quite small. I’ll grant that 20 or 30 years ago, that may have been a different story (though it would still have been rather implausible even then since we now the magnitude of standard “needling” or counter irritant effects to be clinically significant but small and that involves using a vastly larger needle that is actually moved around). But now, after finding that sham acupuncture, toothpick twirling, and random points of insertion all yield the same results, coupled with the history of acupuncture as a novel construct from a blood letting past I can see no argument for prior plausibility of acupuncture as anything more than placebo effects and study and reporting bias.

Part of the advancement of scientific knowledge, whether it be medical or otherwise, is not just the acquisition of new knowledge and ideas, but the discarding of old ideas and knowledge that have proven to be of little or no use. Acupuncture fits that bill, as does the vast majority of what is so-called “CAM” (which once again cannot be adequately and objectively defined. Calling out “what arises outside of the mainstream” means essentially nothing since all novel medical knowledge must, by definition, not have been “mainstream” prior to incorporation as standard medical practice.)

While there is evidence in the treatment of pain, that sham acupuncture and real acupuncture have similar effectivness, I believe that more objective studies, based on fMRI have shown that actually different measurable effects exist between needling a real point and a sham point nearby. A few examples are PNAS (1998) vol. 95 no. 5 2670-2673;Neuroradiology, Volume 46, Number 5 (2004), 359-362;Neuroscience Letters, Volume 383, Issue 3, (2005) Pages 236–240.

RG: I have read through the articles you referenced.

First off, the largest trial is 37 people, the other two are 15 and 12. In any sort of trial, that is significantly underpowered to generate any sort of reliable results. In brain scan studies specifically this is even worse. John Ioannidis demonstrated that the corpus of brain scan studies are vastly under powered and - even when generous accessions were made - that positive results from brain scan studies were double what should be statistically possible (Arch Gen Psychiatry. 2011;68(8):773-780. doi:10.1001/archgenpsychiatry.2011.28). Granted this may make for a solid pilot study to then do more rigorous analysis, but that has been completely lacking. In fact, looking over a PubMed search on the topic of fMRI scans and acupoints, it is notable to see that there are only 50 studies, they are all small, and they all either come out of China, the authors are all affiliated with a Chinese institution, and/or are published in journals with a low impact factor and an obvious likelihood of bias such as “J Acupunct Meridian Stud” (including the ones that you referenced). It is also well known that studies coming out of China on TCM and acupuncture are extremely unreliable with a massive publication bias. It has been documented that 99% of acupuncture trials out of China are positive (Control Clin Trials. 1998 Apr;19(2):159-66.) which is simply an impossible statistic outside of obvious publication bias and/or poor study methodology. In fact at least two large review of reviews has demonstrated that while “Utilization of Chinese-language databases greatly increased the number of potentially relevant references for each search. Unfortunately, due to methodological flaws, this additional information did not generate any usable information.” (Journal of Evidence-Based Medicine
Volume 5, Issue 2, pages 89–97, May 2012) and “The quality of trials of traditional Chinese medicine must be improved urgently. Large and well designed randomised controlled trials on long term major outcomes should be funded” (Review of randomised controlled trials of traditional Chinese medicine
BMJ 1999; 319 doi: 10.1136/bmj.319.7203.160)

As for the studies you cite specifically: As with pretty much all other studies that come out of China there are significant methodological flaws. In all 3 proper blinding is not observed. One in particular (the PNAS paper) is also striking in that they found differences in the fMRI signal which they then attributed to the participant being either of “yin” or “yang” disposition. They try to claim that the confirmation of “yin” and “yang” characteristics was independently and blindly verified, but they do not at all elucidate what those characteristics are, how the are determined, and whether or not the designation was given before or after the fMRI results were obtained. Additionally, they admit that 1 blind determination was incorrect, which is rather significant given that it is essentially a 50/50 chance of getting it right by pure luck and the sample size was so small. Additionally they discuss the methodology used for selecting whether the pixel (which should be called a voxel in MRI studies) was active by comparing a correlation coefficient to a threshold value (TH) which they say was set at 0.4 for “most of the study.” There is no discussion as to why that particular value was chosen, nor a discussion as to why only “most of the study” used that value, nor which parts used that value, how it was determined that a different value would be used, nor what different value was actually used. The remainder of the studies follow this similarly poor methodology which, as I described above, is known to be par for the course in Chinese studies.

Additionally, large studies with adequate power have shown that in fact there is no difference between sham and verum acupuncture in outcomes. A study of 1162 patients with chronic low back pain (which we would a priori expect to have the largest placebo responses and thus the largest effect sizes with placebo intervention) demonstrated ” At 6 Months, Response Rate Was 47.6% In The Verum Acupuncture Group, 44.2% In The Sham Acupuncture Group…Verum Vs Sham, 3.4% (95% Confidence Interval, −3.7% To 10.3%; P = .39)” (Arch Intern Med. 2007;167(17):1892-1898) In other words, it really didn’t matter where you put the needles it had the same effect which clearly demonstrates that there is no intrinsic utility to acupuncture but a distinct placebo response amongst the cohort of subjects. A study in the journal “Pain” did a large review of 57 systematic reviews. Of note only 4 were considered “high quality.” The review of reviews concluded “numerous systematic reviews have generated little truly convincing evidence that acupuncture is effective in reducing pain. Serious adverse effects continue to be reported.” (PAIN Volume 152, Issue 4 , Pages 755-764, April 2011).

For all of these reasons I am quite unimpressed with the studies you have provided and would most certainly NOT consider them to be more objective. To stress the point, any brain scan data - and especially fMRI data - is subject to a large amount of observer bias in interpretation. A lack of blinding the researchers can very easily lead to biased results (whether intentional or not). As John Ioannidis has pointed out on this topic, the only way to prevent this would be to both blind the researchers AND require that the trial declare beforehand exactly what type of measurements will be used and what will be measured. The resolution of brain scans and our understanding of the complex interconnectedness of neurophysiology is simply too poor to consider them immune to bias and indeed can be very easily biased unintentionally.

So this is what I mean when I refer to prior plausibility of a so-called CAM modality. A rigorous look at the corpus of data surrounding the topic is required and must be implemented, which is much more than simply reading the abstract which will *always* paint the conclusions in the most favorable possible light. I still see no plausibility for a putative mechanism of acupuncture beyond placebo effect and large analysis demonstrate clearly that there is no putative effect. Even the most recent meta-analysis demonstrates a statistically significant but clinically insignificant effect on pain from acupuncture. An intellectually honest researcher would recognize that in subjective assessments there is invariably a large amount of noise in the signal and that the threshold for accepting something as significant must accordingly be higher. Even in that case though, clinical significance is also of importance and no study in acupuncture without serious methodological flaws has demonstrated clinical significance for any use of acupuncture.

We are pleased to have an engaged audience with varying perspectives contributing to the NCCAM Research Blog. When commenting, please be mindful of the NCCAM Blog Comment Policy, We want this blog to be a space for collegial, productive exchange. All contributors should be comfortable providing their opinions. Please stay on-topic and refrain from personal attacks and disparaging comments directed at individuals.

I fail to see a “personal attack” or “disparaging” comment directed at anyone. Could you explain what comment(s) skirted the line?

Such discussion and debate is very similar with in China. In China, there are many arguments whether TCM is scientific or not. Actually, TCM does work in clinical including acupuncture, herbs, although its theory is so difficult to understand and explain using scientific approaches, for example, what is the meridian? Qi could be flowing along the meridian after acupuncture, it is an exist phenomenon, however we could not know the meridian until now.

Sham acupuncture could not be manipulated (left and right, up and down) by doctor while real acupuncture could. No pain no gain. So I am afriad that sham acupuncture is same as real acupuncture. However, we should study that if there is any difference between needling a real point and a sham point nearby (meridian point and non-meridian point) even “Ashi” point.

Many years ago, I attended a lecture at the University of California, Berkeley, Department of Physiology, given by a historian of science named Merrily Borrell. The topic was this: The discovery of the existence of “hormones” involved what was at the time a revolutionary idea, that these chemicals (as opposed to the nervous system) organize physiology in the body. Borrell asked how a novel, revolutionary idea could come about. According to her lecture, during a period of time that nothing seemed to be happening in the laboratory research community, there was a great deal of other enthusiasm and exploration. She discussed, in particular, testicular extracts used as tonics to restore masculine vigor. Out of a stewing pot of apparent snake oil remedies, animistic-sounding explanations (if you ingest the organs of animals you take on their qualities), stimulation of enthusiasm and interest in other organ extracts among professionals (including the pancreas), and interest by people with laboratory research backgrounds, this work eventually found its way into scientific laboratories. A dog, for example, was hooked up to an apparatus that recorded heart rate, etc., then given testicular extracts, and, lo and behold, there were measurable physiological changes. However, the conclusion was not that there was a causal relationship between the extracts and dependent variables—that, for example, since heart rate changed that must have caused the effect. The effect on dependent variables indicated that some kind of phenomenon existed, but its cause turned out to be an unexpected, novel cause that had hitherto been outside the limited framework of the science of the day. While I wouldn’t argue that something similar is happening with CAM interventions, I also wouldn’t confidently assert that it is not. It is possible that practices that are difficult to explain on the basis of scientific plausibility, but that are shown to have clinical effects, might be drawing on principles that do not fall within current ideas but are nonetheless important. This is especially so given the limited concepts that traditional science has for understanding the psychological, interactional, the physical context, etc., and limited concepts bridging all of these and physiology.

Paula Derry:

You are indeed correct and I, nor any of my like minded peers and colleagues, are denouncing the notion of scientific exploration to find new and unique effects and models.

There are three main differences between your allegory and the reality of (the vast majority of) CAM research.

1) In the current age we have a much larger knowledge base with many robustly validated basic principles which we did not have in the past. At the time that Dr. Frederick Banting discovered insulin his research was extremely novel and cutting edge. Today, however, we have a vastly more robust understanding of the basic premises and have had enough time with enough rigor to also exclude certain modalities from the realm of possibility - energy and distance healing, for example. Furthermore, our current state of knowledge and the frontier of medical science is much more subtle and nuanced. The low hanging fruit has been picked, so to speak, and we are unlikely to discover some completely novel yet clinically profound mechanism heretofore undescribed.

2) Back then we were witnessing clear and evident effects that begged an explanation. This is not the case now. When doing rigorous studies we find very little, if any, clinical effects from CAM modalities. Part of being rigorous is also an understanding of confounding, bias (researcher, publication, and otherwise), Hawthorne effects, placebo effects and responses (which are different and many), and the many pitfalls of clinical and translational research. If a CAM modality demonstrates a clear and undeniable clinical effect, then I would be the first to advocate its further investigation. But there have been none to date that do so.

3) We are able to be more precise in our description and understanding of what it is we wish to study. In your example the ingestion of testicular extracts was attributed to animism. Back then we didn’t know any better. Now we do. So if one wishes to study the counter irritant effects of acupuncture, or the placebo response of pain, then that is what we should study. Acupuncture is too broad and vague a term to be the reasonable topic of serious research since all research to date demonstrates that the whole of it and the parts are simply not efficacious nor in line with scientific plausibility.

20 years ago, I agree it would have been reasonable to consider it. There was a claim for efficacy, with strong anecdotes and promising pilot studies. There was some plausibility as to a putative mechanism - needles penetrating the skin is not ludicrous to think of as having an effect. But now, after years of study finding mostly negative and the occasional equivocal study, further demonstrating that there is nothing intrinsic nor unique to acupuncture, why are we still studying it?

As for whether or not “something similar,” as you say, is happening in CAM interventions - what defines something as CAM? The only logically consistent definition that I have ever come across is “That which has yet to be proven or has been disproven.” And for the research of the former, there seems to be no need to isolate specifically “CAM” as a research target since it is quite in the “mainstream” to research unknown and not yet proven things. The NIH does this regularly.

So I ask again of Dr. Killen - what study would be justifiably done by the NCCAM but NOT by the NIH at large? Wherein lies the unique aspect of researching something SPECIFICALLY CAM that the NIH cannot reasonably handle under its purview? The crux of the matter is that the CAM designation is simply a false category - one that borrows from the science based fields of pharmacognosy, nutrition, and exercise to lend a false legitimacy to what is otherwise pre-scientific magical thinking. Applying the set of standards set forth by Dr. Briggs (and recognizing that herbal medicine is not CAM but is pharmacognosy) what remains to set apart the study of CAM specifically from any other proposed therapy? I think an entire blog post here would be warranted to explain as I am genuinely interested in the answer.

To nybgrus—With regard to your perspective that “our current state of knowledge and the frontier of medical science is much more subtle and nuanced. The low hanging fruit has been picked, so to speak, and we are unlikely to discover some completely novel yet clinically profound mechanism heretofore undescribed”: A classic response to this is that at one time physicists would say the same thing. Their science built on the shoulders of giants like Newton and basic principles were well in hand. Then along came Einstein. And modern physics.
I agree that conceptualizing the qualities of CAM would be a useful exercise. I actually have taught an introductory overview of CAM/holism as a professional CEU in the past that examines some of this. My own opinion is that this is a complex, multi-layered, dialectical set of issues.

@Paula Derry: I absolutely agree. But do note two very important things. Firstly I said it would be UNLIKELY not impossible. Secondly, quantum mechanics did not INVALIDATE or REVERSE Newtonian mechanics - it BUILT upon it and REFINED the understanding that Newtonian mechanics gave us. Things like energy, faith, and distance healing would invalidate large swaths of well established science. Homeopathy would have to invalidate the fields of chemistry, physics, and pharmacology to actually be correct and “work.” Additionally, you should further note my distinction - revolutionary and interesting ideas that can change the direction of medicine can and do arise - but there is nothing inherently unique to “CAM” that it can lend a leg up on finding them. My argument has never been that something described by some as “CAM” cannot possibly be a profound source of medical discovery. It is that there is no difference in the kind of research that would lead to such a discovery. The NIH is just as equipped to find paradigm shifting results as the NCCAM would be. The dichotomy between the “CAM” and “mainstream medicine” or “Eastern” and “Western” or “holistic” and “allopathic” is a false dichotomy. There is no real objective criteria to define the two nor any reason to separate them. And you may have taught an introductory overview but I also hold an entire undergraduate degree (with honors) in the field of CAM/holism and medical anthropology. I was excited to learn about these differences and new ways of approaching health care and medical problems until I realized that it was all imaginary. So my point still stands - what does the NCCAM uniquely offer to the research of medical science that the NIH is not equipped to do? What is actually unique and special about CAM outside of the magical, pseudo-, and pre-scientific thinking which is the genesis of most of it?

I applaud Dr. Killen’s refinement of the selection process. It may be worth remembering that science is a process - not a product (multiple sources). We always don’t know what we don’t yet know. The task here is not to satisfy self-described orthodox scientists, but to help understand healing processes in the context of real people who exist inside of multiple cultural and psychological frameworks. Yes, rigorous science is great, but please get over yourselves! We need to investigate CAM, where investigation is even possible, without marginalizing mind-body connections, because they give us important clues into human healing.

NCCAM is not here because mainstream scientists decided that CAM was worth investigating. NCCAM is here because scientists absolutely refused to even look at treatment modalities that were outside of mainstream medicine. Look around and you’ll see that mainstream clinical medicine is very different than it was at the time of NCCAM’s founding. Primary care doctors are increasingly concerned about the enormous gaps between mainstream scientific pharmaceutical science and theory and low-risk, effective treatments for many common conditions. Mainstream pharmaceutical-driven science has been woefully inadequate in understanding and treating pain. The costs of many mainstream treatments are soaring, and their invasiveness is part of what adds to the fiscal crises facing our nation. So, while the effectiveness of NCCAM will be judged in the context of rigorous science, we need to keep pushing the envelope for the sake of humanity, not scientists.

That puts NCCAM in an unenviable “hot seat.” I agree with the mission of NCCAM, so I do not agree with those who question the scientific legitimacy of NCCAM, and who have been attacking NCCAM and its decisions from day one. Some of the commenters don’t want to give any inferred endorsements to Acupuncture, yet they seem to easily accept the deaths, addictions and disabilities caused to hundreds of thousands of Americans from mainstream science applied as directed.

” I agree with the mission of NCCAM, so I do not agree with those who question the scientific legitimacy of NCCAM, and who have been attacking NCCAM and its decisions from day one. Some of the commenters don’t want to give any inferred endorsements to Acupuncture, yet they seem to easily accept the deaths, addictions and disabilities caused to hundreds of thousands of Americans from mainstream science applied as directed.”

Have you any links to any studies for that remark “….yet they seem to easily accept the deaths, addictions and disabilities caused to hundreds of thousands of Americans from mainstream science applied as directed.”?

I agree that it could be defensible to fund research to “explore the hypothesis that acupuncture mediates pain by conditioning or expectancy effects produced by a convincing ritual combined with a counter-irritant.” In order to have maximum possible analgesic effects, the practice of acupuncture supposedly requires technical knowledge and skill, and not just the theatrical ability to offer a convincing ritual. Thus, a serious exploration of this hypothesis should examine whether acupuncturists can offer clinically superior pain relief than can people who don’t have the technical expertise, but who can perform the theatrical component of convincing ritual. Studies comparing verum acupuncture to sham acupuncture address these issues to some extent, but such studies are not worth doing unless they are rigorously controlled. Double-blinding is essential; single-blinded acupuncture studies are problematic especially when subjective appraisal of pain is the primary outcome variable studied.

I have no objection to funding research to investigate the safety and efficacy of spinal manipulation or mobilization in the treatment of some types of musculo-skeletal conditions. Such treatments do not clearly flunk the plausibility test. NCCAM and Dr. Killen have referred to spinal manipulation as if it falls under the umbrella of “complementary and alternative medicine” or “CAM.” I don’t see what basis there is for characterizing spinal manipulation as a form of CAM since most physical therapy methods with plausible rationales for clinical use are not typically categorized as CAM. I suspect that spinal manipulation is referred to as CAM mainly because of the error of equating spinal manipulation with methods that flunk the plausibility test and therefore benefit by being euphemistically described as “complementary” or “alternative”: chiropractic adjustment methods to correct supposed chiropractic subluxations. Applying NCCAM’s standards, it is problematic to fund research to clinically test chiropractic adjustment methods for correcting subluxations.

lilady, I have no links to studies on whether it is more likely that those who oppose research on CAM modalities are more likely to be unconcerned about harm caused by mainstream medicine. That might be an interesting avenue for research.

William M. London, I’d ask whether there are studies that evaluate the placebo effect of the theatrical component of convincing ritual as it pertains to mainstream medical practitioners and pharmaceuticals. Regarding spinal manipulation, which dates back at least to Ancient Greece, and was common in the age of “bonesetters,” I suggest considering the history before dismissing these origins.

The fact that physical therapy is considered mainstream medicine, to me, is not probative regarding what other practices are considered CAM or mainstream. Physical Therapy, like mainstream medicine, probably includes many practices that were grandfathered into the mainstream without double-blind, placebo-controlled studies.

Paula Derry, regarding your comment, “So my point still stands - what does the NCCAM uniquely offer to the research of medical science that the NIH is not equipped to do? What is actually unique and special about CAM outside of the magical, pseudo-, and pre-scientific thinking which is the genesis of most of it?” Within the cultural context of Western medicine, what sets NCCAM aside as a potential research leader for CAM is the refusal of the other NIH institutes to conduct this research. I believe that the cultural biases against objectively studying and considering CAM modalities (including images of “primitive” practices and ritual imaginations are what prompted Congress to initiate OAM and NCCAM. Similar initiatives have led to demands to provide more research on the health and diseases of women, the costs of addiction to prescription drugs, increased research on HIV/AIDS, etc. Scientists are not the ultimate deciders of where medical research is needed, and what bodies are most qualified and trusted to objectively design and supervise that research.

Historical use is proven a poor indicator of utility, efficacy, and safety. Blood letting is the common and straighforward example that comes to mind. Objective research is what lets us say that most CAM is patently absurd to study. It is lacking objectivity and instead applying cultural SUBJECTIVITY that allows for one to give undeserved favor to CAM modalities and falsely elevate their prior probability of efficacy and consider lackluster evidence to be on par with actual scientific medicine. If the NIH refuses to study certain things, it is because of those reasons. As I said, WHY would the NIH refuse to study something that is reasonable to study and shows medical promise? If you need to create a new department to lower the standards of entry to study that demonstrates the poor quality of the studies to ensue. And the NCCAM was created specifically by two congressman with obvious political motivations. Lastly, while scientists are not the ULTIMATE decision makers as to what to study, they ARE the ones that should dictate what, out of the entire universe of things out there, is USEFUL to study. The people should choose what to focus limited resources on out of a pool of POTENTIALLY USEFUL research projects with high prior probability, not simply get to vote on spending money for studies with absolutely zero chance of success, such as intercessory prayer to lower HIV titers or homeopathy for asthma.

I agree that there is a strong bias against performing research on CAM. Pursuing research along non-traditional lines is not typically a way to win friends, influence people, get funding and advance your academic career. Creating the space, funding mechanism and a rigorous process for allowing credible scientific inquiry into CAM can allow us to discover new things that we may not learn through only researching along conventional lines of thinking.

Many people feel that studying “energy-based therapies” (such as acupuncture) is a waste of time and seem to take offense at any suggestion that “energy” plays a significant role in physiology. There are a number of Western “energy therapies” such as defibrillation, electroconvulsive therapy, TENS, PENS, PEMF, ultrasound, e-stim, and radiation therapy, to name a few. These all involve levels of energy that can cause cell membrane depolarization, or heating/destruction of tissue. Could it be plausible that lower levels of energy can alter physiology? There is in fact a wealth of research showing that pathology is associated with currents of injury and abnormal charge distributions. The rate at which white blood cells migrate to injury sites is more likely explained by charge differentials (negative WBC surface charge attracted to positive charge at injury site) than chemotaxis. A host of changes in physiology can be seen in response to minute changes in charge, current and electromagnetic fields. It seems presumptuous to think our ingenuity in discovering and manipulating these effects might not be overshadowed by innate physiologic regulatory mechanisms that harness these phenomena. The body is continually generating charges through biochemical, piezoelectric and other mechanims. The biophysical properties of tissues vary widely but predictably, resulting in anatomic patterns of conductance and impedance. Since charge tends to flow along low resistance pathways it is not unreasonable to suspect there are currents flowing in certain patterns with normal physiology and altered patterns when there is pathology. If this is the case, the flow of charge and currents can be used diagnostically (as in EKGs and EEGs). Conversely, since charge and current can affect physiology, the normal and altered flow of charge can actually regulate physiology and, when understood, be used therapeutically.

The purpose of my description of a possible energy-based model that complements the more traditionally accepted models of physiology is not to necessarily convince anyone of its veracity—that would take far more information, discussion and debate than is possible in this type of setting. My point is that scientifically plausible arguments can be made to explain effects seen with CAM. These types of arguments and lines of reasoning would not even be considered were it not for a willingness to study CAM with a sincere intellectual curiosity.

Creating a safe space for rigorous scientific inquiry along non-traditional lines of thinking can open up frontiers that might not be explored otherwise.

[commercial link removed, per policy]

In my last post I include links to some studies documenting the alterations of charge and current associated with injury/pathology and biologic effects of low levels of charge and current. The links were removed, per policy, as being commercial. There were no products or services being promoted so I’m not sure how the links can be construed as commercial. In any case, this type of information is easy to find online from credible sources.
Deciding what is useful to study depends on what criteria are used. Deciding what is academically, professionally and personally useful exclusively within the constraints of prevailing dogma is an exercise in confirmation bias and is intellectually incestuous. A willingness to look past the scientifically unappealing packaging and metaphysical baggage that CAM often comes wrapped in takes courage. Does NCCAM promote courageous curiosity from researchers who are willing to take a risk, more of the same from those who are complacent and risk averse, or perhaps wasting of resources on dead ends? As the saying goes, “What you see depends on where you stand.”

The links submitted by Milt Hammerly do not violate policy, and we have decided to publish them.

I would like NCCAM to provide us with the list of every study in two separate columns, that they have funded since the formation of the NCCAM twenty years ago. One column should show us the studies that have not yielded positive results and the other column should list those studies of CAM supplements and treatments that have yielded results.//I’d like to know why tax dollars (two billion dollars with the meter still running) have been used to fund NCCAM studies of CAM treatments and modalities.//I’d also like to see which of these CAM treatments and modalities have bridged the wide gap to be accepted as effective treatment alongside science-based and proven medical treatments.

If you can equate the mechanical induction of ionizing radiation and ultrasound and direct electrical current on the order of milliamps as something that could be fine by a human by thinking he can, and that this can cause any measurable physiological effect then you have a very different, and I would argue vastly and inappropriately lower, standard for scientific determination of prior probability. An open mind is mandatory for good science, just so long as one’s brains don’t fall out

That should be done by a human, not fine. apologies as I am posting from my mobile device

Expanding on lilady’s reasonable request: Please add another column indicating which, of all studies funded by NCCAM, were published and where.

nybgrus and Milt Hammerly: how might one get ahold of you offline to discuss your thoughts and position on other energy based modalities? Particularly curious to find out what you may think of PEMFT to enhance and complement traditional medicine and if a paradigm shift towards “preventative” care rather than “symptomatic” care is a topic that merits/warrants further investigation under that umbrella topic of “scientific promise” and “amenability to rigorous scientific studies” that Dr. Killen deftly suggested. Perhaps others in this forum would be equally interested in hearing your matter-of-fact styles that cuts to the core of CAM.


First off, thank you for what I believe is a compliment. However, getting a hold of me off line is not something I am particularly keen on. I don’t go excessively out of my way to maintain my anonymity, but I do have a rather busy life and like to constrain my activities to fora like this rather than taking the time for individual responses. Perhaps that will change at some point.// As for your specific question - “energy based modality” should be precisely defined. PEMFT and Reiki are not both reasonably called “energy based modalities” without distinction - the kind, amount, manner of employment, and putative effects of the energy in question are fundamentally and vastly different. As I said above, using measurable magnetic and electric fields generated by devices specifically designed to do so is fundamentally different than channeling the energy of the “one true source” through your hands at undetectable levels to exert wide and far reaching physiologic effects. I do not know enough about PEMFT specifically to comment adequately (and don’t have the time to research it at the moment), but in general magnetic therapies are mostly hype though cannot be dismissed since there does exist some prior probability (at least for some applications; blood effects are the most common claim and they cannot have a reasonable prior probability of efficacy because the iron in the heme moiety is diamagnetic and thus would not be affected by a magnetic field except in conditions not conducive to life). So in sum, if by “energy based modality” you mean things like proton beam therapy, transcranial magnetic stimulation, or ionizing radiation, yes that is a reasonable application. If you mean things like Reiki and Qi-Gong, then no, there is no reasonable prior probability there/// As for your question about “preventative” versus “symptomatic” treatment… this is a no-brainer. Of course preventative medicine is paramount and it something every medical student is taught in great detail and is knowledge of this is required for our medical board exams. I agree that the American system is less adequately equipped to apply what we know of preventative health, but that is a symptom of the system, not the medical practice guidelines. Preventative medicine is INHERENTLY scientific in basis and has nothing at all alternative about it. Screening guidelines, healthy eating and exercise habits, early intervention in disease processes, vaccines, etc, are all scientifically explored and validated and things I personally advocate on a daily basis to the patients I see. This “paradigm shift” has been ongoing and strong for decades now, long before and completely independent of anything labeled as CAM. I myself am personally interested in the methods to further expand and empower physicians and populations to apply what we have already learned in terms of preventative health. But I’ll stress one last time that this has absolutely nothing to do with CAM and is fundamentally a scientific question with a very long history of use in “mainstream” medicine.

To stress the point, any brain scan data - and especially fMRI data - is subject to a large amount of observer bias in interpretation. A lack of blinding the researchers can very easily lead to biased results (whether intentional or not). As John Ioannidis has pointed out on this topic, the only way to prevent this would be to both blind the researchers AND require that the trial declare beforehand exactly what type of measurements will be used and what will be measured.

[commercial link removed, per policy]

Exactly correct Marty. As I delved deeper into the exact workings of fMRI I became astounded at the amount of “researcher degrees of freedom” (a term coined by Dr. Steven Novella to my knowledge) that come into play.  Also, I can’t imagine that a comment like Marty’s actually has an inappropriate commercial link. I’d bet it was to Ionnidis’ paper or a discussion thereof.

The idea of conducting treatment of a disease on the basis of overall analysis of symptoms and signs, including the cause, nature and location of the illness and the patient’s physical condition, is the crystallization of thousands of years of medical practice in China and has served for centuries as the guiding theoretical principle of traditional Chinese medicine in both prophylaxis and disease treatment. For this reason many people seek for alternative therapy and Chinese medicines.

I do agree with the above Fumin Tang’s opinion. We should pay attention to the CAM theory when we study on its mechanism and clinical applications, otherwise, the negative results will be produced. 

If a modality fails to meet established scientific criteria for efficacy and thus demonstrates negative results, the rational option is not to change the criteria to suit the conclusion one wishes to demonstrate (that CAM does, in fact, “work” despite evidence to the contrary) but to do exactly the same thing we would do with any negative result - realize it doesn’t work and move on.

Both my wife and myself suffer with cervical spondylosis and we are actively trying to find an alternative therapy which will help with our day to day pain management.To date, we have tried acupuncture, electro acupuncture,  cupping and guasa therapies as well as taking natural herbs on a daily basis. We have also tried various combinations of these therapies. We have not yet tried magnetic therapy.Inspite of all the claims made by the therapists we have visited, neither my wife or myself have felt any benefit from any of these therapies and we are rapidly coming to the conclusion, that these are simply “placebo’s” and whilst some people report positive results, the vast majority feel no added beneifit at all.We do find this both surprising and dissapointing, given that some medical institutions do advocate the use of some of these therapies.Graham 

@Graham Maddison - We appreciate your writing about your experiences with treating pain with complementary health approaches. Our Center has placed a high priority on research for pain-related conditions and symptom management, as many people, such as you and your wife, often turn to these approaches for relief of chronic pain.  As is the case in all areas of research, not all complementary health approaches have demonstrated value. Such “negative” findings can be just as important as “positive” findings, because they ensure that the public and health care providers have information with which to make more informed decisions.  The ultimate goal of research on chronic pain is to build an evidence base that can guide pain management decisions.

You may be interested in viewing our fact sheet, “Chronic Pain and CAM: At a Glance “ (at, which reviews the strength of the evidence for various approaches, including acupuncture and herbal remedies.

This page last modified September 24, 2017