REPORTING QALYS – HOW PRECISE CAN WE BE?

The Canadian Common Drug Review (CDR) assesses the cost effectiveness of new drugs seeking listing on public drug plan formularies.  Manufacturers submit economic evaluations of their drugs to CDR; submissions may include cost-utility analyses (CUAs) that report benefits in terms of quality-adjusted life-years (QALYs).  Scientific convention holds that results should be reported with the same degree of precision as they are measured.  This raises questions with respect to CDR economic submissions: Is there published guidance regarding the precision for reporting QALYs?  Are QALYs reported to the same degree of precision as measured?  How does implied precision affect the interpretation of reported QALYs?  Purpose: To review the precision of the reported QALYs in economic evaluations submitted to CDR.  Methods: A literature review was conducted to identify published references or guidelines regarding the accepted precision of QALYs.  The economic evaluations submitted to CDR in 2007 were selected for this project.  CUAs were identified and reviewed; information was abstracted, specifically on the level of precision of utility and QALY estimates, derivation of QALYs, and data sources for utilities.  The key information and issues were summarized. Results:  Of the 27 economic evaluations submitted to CDR in 2007, 14 were CUAs reporting incremental QALYs in the range of 0.0101 to 2.9.  These QALY gains were either reported to one decimal place (two submissions, n=2); two decimals (n=4); three decimals (n=5); or four decimals (n=3).  The utility values of 9 of the 14 CUAs were based on EQ-5D or HUI scores, which are typically calculated to a precision of two decimals. Discussion: Based on the review of the literature, several articles discuss the issue of minimally important differences in utility values and QALYs, in some cases noting issues of precision; however, there is no consensus on a meaningful difference in QALYs (O’Brien 1994, Dolan 1997, Roberts 2004).  Where a 0.1 difference in QALYs is considered to be the minimum increment for precision, four CDR submissions reported QALY gains that were not measurably different from zero, suggesting that reporting conventions can affect the interpretation of results.  Further investigation is required regarding minimally important differences in utility values and QALYs.