On July 24, in Helena, I attended a fun and fascinating meeting sponsored by the Carter Center. I spent the day with a group of incredibly smart people dedicated to improving mental health in Montana.
The focus was twofold. How do we promote and establish mental health parity in Montana and how do with improve behavioral health in schools? Two worthy causes. The discussions were enlightening.
We haven’t solved these problems (yet!). In the meantime, we’re cogitating on the issues we discussed, with plans to coalesce around practical strategies for making progress.
During our daylong discussions, the term evidence-based treatments bounced around. I shared with the group that as an academic psychologist/counselor, I could go deep into a rabbit-hole on terminology pertaining to treatment efficacy. Much to everyone’s relief, I exhibited a sort of superhuman inhibition and avoided taking the discussion down a hole lined with history and trivia. But now, much to everyone’s delight (I’m projecting here), I’m sharing part of my trip down that rabbit hole. If exploring the use of terms like, evidence-based, best practice, and empirically supported treatment is your jam, read on!
The following content is excerpted from our forthcoming text, Counseling and Psychotherapy Theories in Context and Practice (4th edition). Our new co-author is Bryan Cochran. I’m reading one of his chapters right now . . . which is so good that you all should read it . . . eventually. This text is most often used with first-year students in graduate programs in counseling, psychology, and social work. Consequently, this is only a modestly deep rabbit hole.
Enjoy the trip.
*************************************
What Constitutes Evidence? Efficacy, Effectiveness, and Other Research Models
We like to think that when clients or patients walk into a mental health clinic or private practice, they will be offered an intervention that has research support. This statement, as bland as it may seem, would generate substantial controversy among academics, scientists, and people on the street. One person’s evidence may or may not meet another person’s standards. For example, several popular contemporary therapy approaches have minimal research support (e.g., polyvagal theory and therapy, somatic experiencing therapy).
Subjectivity is a palpable problem in scientific research. Humans are inherently subjective; humans design the studies, construct and administer assessment instruments, and conduct the statistical analyses. Consequently, measuring treatment outcomes always includes error and subjectivity. Despite this, we support and respect the scientific method and appreciate efforts to measure (as objectively as possible) psychotherapy outcomes.
There are two primary approaches to outcomes research: (1) efficacy research and (2) effectiveness research. These terms flow from the well-known experimental design concepts of internal and external validity (Campbell et al., 1963). Efficacy research employs experimental designs that emphasize internal validity, allowing researchers to comment on causal mechanisms; effectiveness research uses experimental designs that emphasize external validity, allowing researchers to comment on generalizability of their findings.
Efficacy Research
Efficacy research involves tightly controlled experimental trials with high internal validity. Within medicine, psychology, counseling, and social work, randomized controlled trials (RCTs) are the gold standard for determining treatment efficacy. RCTs statistically compare outcomes between randomly assigned treatment and control groups. In medicine and psychiatry, the control group is usually administered an inert placebo (i.e., placebo pill). In the end, treatment is considered efficacious if the active medication relieves symptoms, on average, at a rate significantly higher than placebo. In psychotherapy research, treatment groups are compared with a waiting list, attention-placebo control group, or alternative treatment group.
To maximize researcher control over independent variables, RCTs require that participants meet specific inclusion and exclusion criteria prior to random assignment to a treatment or comparison group. This allows researchers to determine with greater certainty whether the treatment itself directly caused treatment outcomes.
In 1986, Gerald Klerman, then head of the National Institute of Mental Health, gave a keynote address to the Society for Psychotherapy Research. During his speech, he emphasized that psychotherapy should be evaluated through RCTs. He claimed:
We must come to view psychotherapy as we do aspirin. That is, each form of psychotherapy must have known ingredients, we must know what these ingredients are, they must be trainable and replicable across therapists, and they must be administered in a uniform and consistent way within a given study. (Quoted in Beutler, 2009, p. 308)
Klerman’s speech advocated for medicalizing psychotherapy. Klerman’s motivation for medicalizing psychotherapy partly reflected his awareness of heated competition for health care dollars. This is an important contextual factor. Events that ensued were an effort to place psychological interventions on par with medical interventions.
The strategy of using science to compete for health care dollars eventually coalesced into a movement within professional psychology. In 1993, Division 12 (the Society of Clinical Psychology) of the American Psychological Association (APA) formed a “Task Force on Promotion and Dissemination of Psychological Procedures.” This task force published an initial set of empirically validated treatments. To be considered empirically validated, treatments were required to be (a) manualized and (b) shown to be superior to a placebo or other treatment, or equivalent to an already established treatment in at least two “good” group design studies or in a series of single case design experiments conducted by different investigators (Chambless et al., 1998).
Division 12’s empirically validated treatments were instantly controversial. Critics protested that the process favored behavioral and cognitive behavioral treatments. Others complained that manualized treatment protocols destroyed authentic psychotherapy (Silverman, 1996). In response, Division 12 held to their procedures for identifying efficacious treatments but changed the name from empirically validated treatments to empirically supported treatments (ESTs).
Advocates of ESTs don’t view common factors in psychotherapy as “important” (Baker & McFall, 2014, p. 483). They view psychological interventions as medical procedures implemented by trained professionals. However, other researchers and practitioners complain that efficacy research outcomes do not translate well (aka generalize) to real-world clinical settings (Hoertel et al., 2021; Philips & Falkenström, 2021).
Effectiveness Research
Sternberg, Roediger, and Halpern (2007) described effectiveness studies:
An effectiveness study is one that considers the outcome of psychological treatment, as it is delivered in real-world settings. Effectiveness studies can be methodologically rigorous …, but they do not include random assignment to treatment conditions or placebo control groups. (p. 208)
Effectiveness research focuses on collecting data with external validity. This usually involves “real-world” settings. Effectiveness research can be scientifically rigorous but doesn’t involve random assignment to treatment and control conditions. Inclusion and exclusion criteria for clients to participate are less rigid and more like actual clinical practice, where clients come to therapy with a mix of different symptoms or diagnoses. Effectiveness research is sometimes referred to as “real world designs” or “pragmatic RCTs” (Remskar et al., 2024). Effectiveness research evaluates counseling and psychotherapy as practiced in the real world.
Other Research Models
Other research models also inform researchers and practitioners about therapy process and outcome. These models include survey research, single-case designs, and qualitative studies. However, based on current mental health care reimbursement practices and future trends, providers are increasingly expected to provide services consistent with findings from efficacy and effectiveness research (Cuijpers et al., 2023).
In Pursuit of Research-Supported Psychological Treatments
Procedure-oriented researchers and practitioners believe the active mechanism producing positive psychotherapy outcomes is therapy technique. Common factors proponents support the dodo bird declaration. To make matters more complex, prestigious researchers who don’t have allegiance to one side or the other typically conclude that we don’t have enough evidence to answer these difficult questions about what ingredients create change in psychotherapy (Cuijpers et al., 2019). Here’s what we know: Therapy usually works for most people. Here’s what we don’t know: What, exactly, produces positive changes.
For now, the question shouldn’t be, “Techniques or common factors?” Instead, we should be asking “How do techniques and common factors operate together to produce positive therapy outcomes?” We should also be asking, “Which approaches and techniques work most efficiently for which problems and populations?” To be broadly consistent with the research, we should combine principles and techniques from common factors and EST perspectives. We suspect that the best EST providers also use common factors, and the best common factors clinicians sometimes use empirically supported techniques.
Naming and Claiming What Works
When it comes to naming and claiming what works in psychotherapy, we have a naming problem. Every day, more research information about psychotherapy efficacy and effectiveness rolls in. As a budding clinician, you should track as much of this new research information as is reasonable. To help you navigate the language of researchers and practitioners use to describe “What works,” here’s a short roadmap to the naming and claiming of what works in psychotherapy.
When Klerman (1986) stated, “We must come to view psychotherapy as we do aspirin” his analogy was ironic. Aspirin’s mechanisms and range of effects have been and continue to be complex and sometimes mysterious (Sommers-Flanagan, 2015). Such is also the case with counseling and psychotherapy.
Language matters, and researchers and practitioners have created many ways to describe therapy effectiveness.
- D12 briefly used the phrase empirically validated psychotherapy. Given that psychotherapy outcomes vary, the word validated is generally avoided.
- In the face of criticism, D12 blinked once, renaming their procedures as empirically supported psychotherapy. ESTs are manualized and designed to treat specific mental disorders or specific client problems. If it’s not manualized and doesn’t target a disorder/problem, it’s not an EST.
- ESTs have proliferated. As of this moment (August 2025), 89 ESTs for 30 different psychological disorders and behavior problems are listed on the Division 12 website (https://div12.org/psychological-treatments/). You can search the website to find the research status of various treatments.
- To become proficient in providing an EST requires professional training. Certification may be necessary. It’s impossible to obtain training to implement all the ESTs available.
- In 2006, an APA Presidential Task Force (2006) loosened D12’s definition, shifting to a more flexible term, Evidence-Based Practice (EBP), and defining it as ‘‘the integration of the best available research with clinical expertise in the context of patient characteristics, culture, and preferences’’ (p. 273).
- In 2007, the Journal of Counseling and Development, the American Counseling Association’s flagship journal, inaugurated a new journal section, “Best Practices.” As we’ve written elsewhere, best practice has grown subjective and generic and is “often used so inconsistently that it is nearly meaningless” (Sommers-Flanagan, 2015, p. 98).
- In 2011, D12 relaunched their website, relabeling ESTs as research-supported psychological treatments (n.b., most researchers and practitioners continue to refer to ESTs instead of research-supported psychological treatments).
- As an alternative source of research updates, you can also track the prolific work of Pim Cuijpers and his research team for regular meta-analyses on psychological treatments (Cuijpers et al., 2023; Harrer et al., 2025).
- Other naming variations, all designed to convey the message that specific treatments have research support, include evidence-based treatment, evidence-supported treatment, and other phrasings that, in contrast to ESTs and APA’s evidence-based practice definition, have no formal definition.
Manuals, Fidelity, and Creativity
Manualized treatments require therapist fidelity. In psychotherapy, fidelity means exactness or faithfulness to the published procedure—meaning you follow the manual. However, in the real world, when it comes to treatment fidelity, therapist practice varies. Some therapists follow manuals to the letter. Others use the manual as an outline. Still others read the manual, put it aside, and infuse their therapeutic creativity.
A seasoned therapist (Bernard) we know recently provided a short, informal description of his application of exposure therapy to adult and child clients diagnosed with obsessive-compulsive disorder. Bernard described interactions where his adult clients sobbed with relief upon getting a diagnosis. Most manuals don’t specify how to respond to clients sobbing, so he provided empathy, support, and encouragement. Bernard described a therapy scenario where the client’s final exposure trial involved the client standing behind Bernard and holding a sharp kitchen knife at Bernard’s neck. This level of risk-taking and intimacy also isn’t in the manual—but Bernard’s client benefited from Bernard trusting him and his impulse control.
During his presentation, Bernard’s colleagues chimed in, noting that Bernard was known for eliciting boisterous laughter from anxiety-plagued children and teenagers. There’s no manual available on using humor with clients, especially youth with overwhelming obsessional anxiety. Bernard used humor anyway. Although Bernard had read the manuals, his exposure treatments were laced with empathy, creativity, real-world relevance, and humor. Much to his clients’ benefit, Bernard’s approach was far outside the manualized box (B. Balleweg, personal communication, July 14, 2025).
As Norcross and Lambert (2018) wrote: “Treatment methods are relational acts” (p. 5). The reverse is equally applicable, “Relational acts are treatment methods.” As you move into your therapeutic future, we hope you will take the more challenging path, learning how to apply BOTH the techniques AND the common factors. You might think of this—like Bernard—as practicing the science and art of psychotherapy.
**********************************
Note: This is a draft excerpt from Chapter 1 of our 4th edition, coming out in 2026. As a draft, your input is especially helpful. Please share as to whether the rabbit hole was too deep, not deep enough, just right, and anything else you’re inspired to share.
Thanks for reading!
John Sommers-Flanagan 