Methods: 2 stage modified Delphi Process included: a) defining 12 classes of quality criteria; b) summarizing their theoretical and empirical links to decision quality; c) drafting and testing voting items on quality indicators; and d) nominated sampling of stakeholder groups (consumers, practitioners, researchers, payers). Invitees were emailed (2 reminders) to vote on a website within 3 weeks. Respondents reviewed plain language theoretical/empirical summaries and voted on the importance of potential quality indicators using a 9 point rating scale. In the 2nd round, they voted with feedback on 83 indicators. Consensus was defined by 'equimedians' [weighted by size of group] between 7 and 9 unless at least 30% the scores were in the top and bottom terciles.

Results: 122 of 212 of invitees (58%) participated on voting panels [14 countries; 77 researchers; 21 consumers; 10 practitioners; 14 payers]. 103/122 (85%) voted in both rounds. There was consensus on 74 of 83 quality indicators (# equimedian scores: '9'=41; '8'=26; '7'=7); equimedians were identical to medians in 68/74 indicators. The number of endorsed items by criterion class were:

a) development process (9/9);

b) option information (13/13);

c) probabilities (11/13 including: using event rates with comparable denominators, time periods and scales; describing uncertainty around estimates; using multiple methods [words, numbers, diagrams]; placing probabilities in context; using mixed frames);

d) values clarification (3/3 including providing vicarious experience; asking which positive/negative features matter most; suggesting communication strategies);

e) patient stories (2/5);

f) guiding/coaching (3/3 guiding; 0/2 coaching);

g) disclosure (5/5);

h) Internet delivery (6/6);

i) balance(3/3);

k) plain language (4/6);

l) up-to-date evidence (7/7); and

m) effectiveness (8/8) including indicators focused on: 'decision process' [recognize need for decision; know available options; understand values affect decision; clear about features that matter most; discuss values with practitioner; involvement in decision making in preferred ways]; and, 'decision quality' [improves the match between the features that matter most to the informed patient and the option that is chosen].

Conclusions: Stakeholders agreed on criteria with most empirical support. Results will assist: a) developers to improve their patient decision aids; b) users and payers to judge the quality of patient decision aids; c) researchers to address gaps in research and explore implementation issues.