THE USE AND MISUSE OF ORTHOGONAL ARRAYS IN THE DESIGN OF CONJOINT ANALYSIS APPLICATIONS IN HEALTH

Purpose: Conjoint analysis applications in health often combine main-effects orthogonal designs and the paired-comparison question format, but such experimental designs are difficult when attributes have more than two levels. We demonstrate the deficiencies of existing approaches and introduce a novel method of design based on the design of the underlying experiment (and as a consequence, the simultaneous choice of the profile pairs) using orthogonal arrays.  

Method: We first evaluate the properties of main effects orthogonal designs for attributes with two levels using existing approaches, including “random pairing”, “constant comparator” and “fold-over designs”.  We then demonstrate the added complexities of allowing one attribute to have three levels. Finally, we demonstrate a novel method for generating an experimental design using an orthogonal array to generate pairs of profiles and explore how this relative complex method can be simplified using economic theory.  

Results: The minimum main effects orthogonal array for a 2x2x2 space is four scenarios. Paired comparisons based on random pairings or constant comparator fail to yield an orthogonal underlying design, but the use of a “fold-over” design does.  If one attribute has three levels the minimum orthogonal array is 12 scenarios, but the inability to apply the fold-over approach prevents the construction of paired comparisons.  If the three level factor is considered as a six level factor (on for each possible differing paired comparisons of the three levels), an orthogonal array of 12 scenarios can be created that allows for a fold-over design. If one assumes that the three level attribute represents a continuous and equally spaced variable, or if one excludes the direct comparison of the extreme endpoints of the factor, a minimum orthogonal array that allows paired comparisons involves 8 such pairings.  If one makes both assumptions, the number of paired comparisons can be lowered to 4.   

Conclusion: Current practice of generating random pairings from the orthogonal array or comparing the orthogonal array with a constant comparison should be avoided.  In using design of experiment methods to generate paired comparisons for conjoint analysis, one should not focus the number of attribute levels, but on the number of policy relevant comparisons of the levels of each factor are needed. Only in the case of two levels is this number the same.