Purpose: We describe a computer-based simulation tool, MEDIC that we developed to study cognition and dynamic decision making in medical diagnosis incorporating time constraints, multiple and delayed feedback, and repeated decisions. Methods: MEDIC is an interactive tool involving presentation of symptoms, generation of diagnoses, tests of diagnoses, treatments, and outcome feedback. The simulation begins with a patient complaining of symptoms. Each patient has a different initial health level, which continues to fluctuate downward until the patient has been treated or until the patient dies. Since each patient is randomly assigned one of many fictitious diseases, participants must test for the presence of symptoms, each having a different probability of being associated with one of the diseases. Each test returns with a definitive diagnosis (absent or present) after a predetermined time delay for the test to run. The participant provides an assessment of the probability of the presence of the disease. Then, participants can either conduct more tests or administer a treatment. Feedback comprises the actual disease present, the disease the participant believed was present, and a score that represents their accuracy throughout the task. Results: We have used the simulation tool to study the process by which people improve their diagnosis and treatment with repeated patient cases. In general, performance does not always improve with experience. Individuals that perform poorly are those that do not realize the value of information, and the organization and diagnosticity of cues. There is a cost for checking all the cues in terms of significant time delays, which are rarely overcome by the benefits gained with experience. Conclusions: MEDIC allows one to study several crucial facets of complex medical decision making while also being well controlled for experimental purposes. Using MEDIC, there is a correct diagnosis for the patient, which provides both outcome and process measures of good performance. MEDIC also allows us to calculate cue diagnosticity and probability functions over the set of hypotheses that participants are explicitly considering, based on assumptions of local (bounded) rationality.