PURPOSE: To assess how well clinical practice guidelines (CPGs) incorporate evidence on patient preferences compared to that on treatment effectiveness. METHODS: Our database consisted of 65 high-quality CPGs that were externally judged to be the best in their field for a wide-range of primary care issues. We adapted two instruments originally constructed to evaluate the overall quality of CPGs to specifically assess the quality of integrating information on patient preference and treatment effectiveness. We also counted words and references in each CPG associated with patient preferences and treatment effectiveness. Two reviewers independently assessed each CPG. The post-hoc validity of our quality and quantity assessment techniques was assessed. RESULTS: In our preliminary analysis, the internal consistency of our quality instruments was moderate (Cronbach's alpha 0.64 to 0.88). The outcome measures (instruments, counts) evaluating the integration of patient preference correlated highly with one another (Spearman rho = 0.644 to 0.888), as did measures assessing the incorporation of treatment effectiveness (rho = 0.256 to 0.815). Rank correlations between the reviewers' overall subjective assessment compared to the outcome measurements were high (Kendall's tau-b = 0.337 to 0.719). Based on our adapted instruments, CPGs scored significantly higher (p < 0.0001) on the quality of integrating treatment effectiveness evidence compared to patient preferences evidence (mean instrument 1 scores on a scale of 0.25 to 1.0: 0.67 vs. 0.43; mean instrument 2 scores on a scale of 0 to 1: 0.57 vs. 0.17). The average percentage of the total word count of each CPG dedicated to treatment effectiveness was 22.4% compared to 4.6% for patient preferences. The average percentage of references citing treatment effectiveness evidence was 33.8% compared to 6.2% for patient preferences. CONCLUSIONS: High quality CPGs fail to integrate data on patient preferences to the same degree as they do information on treatment effectiveness. This finding may reflect the view within the evidence-based medicine paradigm that published preference studies are of limited value, or the view that preference elicitation belongs primarily within the clinical encounter. Expanding the definition of evidence to include studies of patient preferences may increase the quality of CPGs.