Over the years a number of studies have looked at the relationship between therapist competence, adherence to the principles of CT, and treatment outcome. You'd expect skilled therapists to get better results than unskilled therapists and you'd expect therapists who follow CT protocols to get better results than therapists who deviate from the protocols. However, it doesn't necessarily turn out this way.

Some studies have found that therapists who get high scores on a Cognitive Therapy rating scale (which assesses the extent to which the therapist is doing the things that are believed to be important in CT) get better results than therapists who get low scores. However, other studies have found that there was no significant relationship and yet others have found a negative relationship! This does not make sense.

Here are a few ideas that may help make sense of these contradictory results:

1. To evaluate the importance of adherence to CT principles or competence in CT, we need an adequate range of levels of adherence/competence. If everybody is competent/adherent, you get "restriction of range." As a result correlations are small and non-significant simply because of the restricted range. For example, if you examine the relationship between player's height and scoring in basketball but only have subjects who are between 6'10" and 7'2", you're likely to find little relationship between height and scoring. If you have subjects whose heights range from 4'2" to 7'2" you'll find a much stronger relationship between height and scoring. In outcome studies where therapists are well trained and well supervised, you may well get restriction of range because everyone is doing CT well. This will make it look as though competence in CT is unrelated to treatment outcome.

2. The importance of adherence/competence may vary depending on the therapist's level of experience and the complexity of the clinical situation. Consider the relationship between adherence to a recipe and quality of outcome in cooking. With inexperienced cooks, adherence to the recipe will produce better results. However, experienced cooks can deviate from the recipe and still produce good results. In fact, they may produce better results by varying the recipe. If one is following the recipe and encounters a problem, a cook who adheres rigidly to the recipe anyway may achieve a poor outcome while a cook who deviates from the recipe to correct the problem may produce much better results.

3. If the therapist is rigidly adhering to a treatment protocol without listening to the client, understanding them, and responding sensitively, they are doing bad CT (even if they are using many CT interventions). However, they might get a high adherence score. Adherence to a protocol isn't necessarily the same as applying the protocol skillfully. It is not easy to do a good job of measuring skill in applying CT (or any other therapy). The various versions of CT checklists/rating scales can do a good job of assessing how skillfully CT is being done. However, for this to happen the raters need to be skilled enough in CT to tell the difference between good CT and bad CT and the rating scale being used needs to do more than count the number of cognitive and behavioral interventions used.