Purpose: In this study, we evaluated the ability of patients to obtain quality answers to health questions by searching the Web with standard search engines in comparison to using a search engine specifically designed for finding health information.

Methods: We recruited 66 adult patients in an outpatient clinic to each search the Web for answers to six representative health questions using one of three search engines (pseudo-randomized order). Two of the search engines were systems currently available on the Web: MedHunt, which specifically searches medical content, and Google, which served as a representative general purpose search engine. The third search engine was a new system designed to help patients find quality health content. During the study, we recorded all computer interactions and then assessed patients' satisfaction with the content returned by the search engine. We also asked clinicians to evaluate the quality of the Web sites that had been returned by each of the search engines for each of the questions.

Results: We found that patients preferred the newly designed health search engine to the currently available health search engine with respect to usefulness, ease-of-use, relevance, and overall satisfaction. However, most patients judged the general purpose search engine, Google, to be at least as good, if not better than both other health search engines. In addition, the overall ranking of search engines by patients at the end of the experiment placed Google at the top. From our qualitative findings, it appears that this may be due to our subjects having previous experience with Google, and not the other search engines. In addition, the Google search engine has a very simple interface. From the clinician perspective, however, the Web sites retrieved by the newly designed health search engine were rated higher on all three dimensions of quality (relevance, accuracy and trustworthiness) when compared to MedHunt, and better than Google on both accuracy and trustworthiness.

Conclusions: Although patients did not perceive an improvement in quality of information, clinician ratings of the material being returned to patients during representative Web searches of health information showed that the newly designed health search engine outperformed those currently available on dimensions of accuracy and trustworthiness. Health-specific search engines may be important in ensuring that patients obtain reliable information from the Web.