Problem analyzing data, know your source - example Consumer Reports Hospital Ranking based on Medicare Billing

We all want to go to the best hospitals.  I read the Consumer Reports ranking on Hospitals.  What I didn't know is the ranking is based on billing information from Medicare.  What does billing information from Medicare have to do with the quality of your hospital?  The ability for a hospital to format the data according to Medicare standards could be what gives a hospital a higher ranking.  Some hospitals weren't ranked because their data did not meet medicare standards.

Hope this gets you thinking about your big data projects.

NBCnews reports on this situation.

Dr. Peter Pronovost, senior vice president for patient safety and quality at Johns Hopkins and one of the leaders in the fight to improve hospital quality, applauds the idea but says the data the report is based on is flawed. “I really applaud the Consumer Report effort to get information to consumers about complications,” he said.

“The overall concept is spot-on,” Pronovost told NBC News. “One of the concerns is they measured these complications using administrative data, which is completely understandable, but we know it’s not completely accurate.”

...

Many of the biggest and most famous hospitals aren’t listed. Consumer Reports used Medicare reporting data for its report and could only include hospitals that reported data in a certain way.

The article points out an accuracy of 25% correlation of infections base on billing information filed.

Unfortunately, he said, there’s not much better data out there yet. One of the measures – infections among patients fitted with a catheter – is only right 25 percent of the time when calculated using billing information filed to Medicare, Pronovost says.