Purpose: Numeracy refers to the ability to perform basic arithmetic tasks necessary in daily life, and numeracy may become even more important when individuals are confronted with complex healthcare decisions. Recent research has suggested that health numeracy significantly influences treatment decision-making in hypothetical scenarios and may also influence other important health outcomes. Several measures have thus been constructed to assess health numeracy. Developed by Schwartz and colleagues (1997), the Numeracy Questionnaire (NQ) is commonly used and has been the foundation of several other measures. While many studies have employed this measure, none have thoroughly examined its psychometric properties. The purpose of this study was to analyze the NQ using Item Response Theory (IRT) methods. Method: Following informed consent procedures, three groups of participants completed the NQ and sociodemographic items: (1) women participating in an ovarian screening program (N=609), (2) a subset of their spouses (N=98), and (3) a non-screening group of age and education matched women (N=102). For each of three NQ items, a two-parameter IRT model was fit to obtain estimates of item difficulty (b) and discrimination (a) parameters. Item and total test information were assessed along the continuum of the latent numeracy construct. Results: Participants' mean age was 60 (SD=11). The majority were at least high school educated (92%), while a minority reported having at least a college degree (19%). Mean NQ score was 1.26 (SD=0.89), and proportions of participants correctly responding to each item were: 69% (item 1), 44% (item 2), and 12% (item 3). IRT analyses revealed one item with low difficulty (b₁=-1.32) and low discrimination (a₁=0.69), providing little information for assessing numeracy. Remaining items were substantially more informative (a₂=1.96, a₃=2.76), providing information about two distinct levels of numeracy (b₂=0.20, b₃=1.38). Overall, the NQ was most informative for levels of numeracy between the mean and approximately two standard deviations above. Information (and thus precision) decreased dramatically below the mean and at high levels of numeracy. Conclusions: The NQ appeared to be an informative measure only for participants with numeracy levels above the mean. To improve efficiency and precision of measurement across the continuum of numeracy, additional items should be developed to discriminate among respondents at lower levels of numeracy. Results demonstrate the potential utility of IRT analyses to improve numeracy measurement.