Purpose: The objective of this study is to compare experts' judgments of the appropriateness of a treatment (interferon treatment for melanoma) on the basis of important attributes of this disease (thickness, ulceration, lymph node involvement and type of metastases) to a decision analytic model in which the probabilities are derived from the medical literature and from epidemiological studies.

Methods: We compare experts' judgments of the appropriateness of a treatment (interferon treatment for melanoma) on the basis of important attributes of this disease (thickness, ulceration, lymph node involvement and type of metastases) to a decision analytic model in which the probabilities are derived from the medical literature and from epidemiological studies. The comparison is based on what we call the linearity test, which examines whether appropriateness is a linear function of the epidemiological value of p2, the probability of deterioration of the patient condition if he would have received the treatment. This comparison allows for the assessment of the validity of the experts' judgments under the assumption that the decision analytic model is valid, or alternatively, the assessment of the validity of the decision analytic model under the assumption that the experts' judgments are valid.

Results: Under the former assumption the results indicate that appropriateness judgments are by and large appropriate, but raise questions regarding the appropriateness of the scales by which medical information is communicated to clinicians. Under the later assumption the results support the idea of a constant treatment effect, the idea that efficacy of a treatment is constant over various levels of severity of the disease.

Conclusions: The results verify the validity of the experts' judgments under the assumptions of the decision analytic model.