"Report Cards" for Doctors Not Always Helpful

When choosing a doctor based on increasingly popular “doctor report cards,” patients should be aware that the areas in which physicians are graded are not always clear-cut, according to researchers from the Johns Hopkins Bloomberg School of Public Health and School of Medicine. They examined the usefulness of quality indicators for asthma specialists; that is, individual aspects of patient care that can be used to “grade” physicians, such as patient satisfaction with care or use of the correct medication. The study is published in the January 2005 issue of The American Journal of Managed Care.

Albert W. Wu, MD, MPH, senior author of the study and associate professor in the Department of Health Policy and Management at the Bloomberg School of Public Health, explained that grading physicians can be difficult for a variety of reasons. “If all physicians receive the equivalent of an ‘A’ on a certain indicator, that does not help distinguish between the doctors. For some indicators, it can also be difficult to obtain reliable data from patients. And finally, surveying a doctor who doesn’t treat enough patients won’t give you an accurate picture of how the physician is performing.”

Albert W. Wu, MD, MPH

The researchers analyzed surveys collected from 2,515 asthmatic patients from 20 California-based physician groups from 1998-1999. Doctors were assessed based on six quality indicators, as reported by their patients—accessibility of asthma care for patients; self management of asthma care; the use of inhaled corticosteroids; satisfaction with asthma care; improvement in health; and the number of emergency department visits and hospitalizations attributable to asthma.

The researchers found that patients were able to give reliable answers about these indicators of the care they were getting. However, they also found that having a sufficient number of patients was key to getting useful results.

“Our findings have practical implications for managed care decision makers and organizations engaged in physician profiling for asthma or other diseases,” said I-Chan Huang, PhD, the first author of the study and postdoctoral fellow in the Bloomberg School of Public Health’s Department of Health Policy and Management. “Variability in quality indicators is a desirable attribute for performance measures. In addition, our results suggest that the reliability of profiling indicators is also an important consideration.”

According to the study authors, not all quality indicators are equal. Indicators should be something that varies doctor-to-doctor and they also need to be measurable, but without much error.

“As doctor report cards become more popular across the country, patients should know that quality of care does vary, but there are clear-cut indicators that can help them determine which doctor to go to. Our research found that, for asthma patients, the following may be useful indicators to help them select among different doctors: asthma self-management knowledge, use of inhaled steroids, satisfaction with asthma care and self-reported improvement in health status,” said Dr. Wu, who also holds an appointment at the Johns Hopkins School of Medicine.

The study authors were supported by grants from the Pacific Business Group on Health.

“Variations of Physician Group Profiling Indicators for Asthma Care” was co-authored by I-Chan Huang, PhD; Gregory B. Diette, MD, MHS; Francesca Dominici, PhD; Constantine Frangakis, PhD; and Albert W. Wu, MD, MPH.