The project aims to help Canadians understand a hospital's possible strengths and weaknesses so they can ask informed questions about their care. (Claude Vickery/CBC)

For anyone facing a serious medical problem, getting the best possible hospital care can be a matter of life and death. But in Canada, limited information about the quality of hospital care is available to the public.

Most of the statistics on what hospitals are doing and how well they do it is housed in little-known databases designed for hospital managers and policy-makers.

Patients trying to find information about hospitals often have few resources beyond their family doctor's recommendation.

CBC's Rate My Hospital report card is a Canadian first.

The new online tool grades hospitals on key performance indicators reported by hospitals and allows patients to rate their own health care experiences on a range of critical care measures.

• Why the fifth estate rated Canadian hospitals

CBC's Rate My Hospital is also a call for more accountability and transparency in our health care system.

The report includes four sections:

• Ratings of hospital performance are based on data supplied by hospitals and published by the Canadian Institute for Health Information.
• The fifth estate Hospital Survey shows initiatives to improve quality and safety. The information was collected through a unique survey of hospital administrators.
• Quick stats are a collection of basic statistics that reflect a hospitals size and performance. They include data gathered by the fifth estate about emergency room wait times and the rate of the hospital-acquired infections C.difficile and MRSA in individual hospitals.
• The patient reviews are based on ongoing feedback from Canadians who have recently been in the hospital.

The website is not meant as a definitive source of information about hospital performance. It provides general information and does not offer medical advice or recommendations for treatment.

A stay at a hospital with a good grade will not necessarily be problem-free. A hospital with a lower grade might do well on measures that CBC has not considered.

The website is designed to help Canadians understand a hospital's possible strengths and weaknesses so that they can ask their health care providers informed questions about their care.

The report card

The methodology that underpins CBC's Rate My Hospital was developed with the help of a panel of Canadian, British and American experts who are leaders in assessing hospital quality and performance.

The panel met in Toronto in November 2012 to develop an outline for the report card.

• Experts advised CBCs Rate My Hospital team

Jason Sutherland and Peter Austin, Canadian biostatisticians noted for their work in evaluating health system performance, developed the statistical methodology used to rate hospitals.

Hospital ratings are based on data collected and reported by more than 600 acute care hospitals in Canada and analyzed and published by the Canadian Institute for Health Information (CIHI). The data represents hospital performance in the fiscal years 2010-11 or 2011-12.

Your hospital may not have received a grade. That's because some hospitals do not do the procedures that CIHI data measures. Others handle too few surgeries or medical cases to accurately assess their performance.

Using the data for five important measures of patient outcomes, CBC has ranked the performance of 239 Canadian hospitals. We also provide grades for individual performance measures on a total of more than 600 hospitals.

When grading hospitals, CBC chose five key performance measures from a list of 21 indicators published by CIHI in March.

Many of the indicators reported by CIHI focus on specialties such as obstetrics and cardiac care, but not all hospitals offer the full range of specialized care.

We limited ourselves to five measures that assess the quality of general surgical and medical treatment only. Including the specialty services would have significantly reduced the number of hospitals we could rate.

We also eliminated performance measures if reliable data was unavailable for most of the hospitals we examined. For small hospitals, the grades are based on three of the five measures.

CIHI's data divides hospitals into four groups:

• Small community hospitals.
• Medium community hospitals.
• Large community hospitals.
• Teaching hospitals.

Hospitals of the same size and that treat similar types of patients are grouped together. Placing hospitals in four groups levels the playing field and makes it easier to compare them.

CIHI has also adjusted the data to account for factors that may contribute to worse outcomes, such as age, gender and whether a patient has multiple medical problems. These risk adjustments are intended to make comparisons between hospitals of the same size more valid.

CBC's ratings are based on a statistical model known as standardization. Each hospitals performance was standardized on every quality indicator.

We then used a quantity called the standard deviation to measure how the hospitals performance compares to other hospitals of the same size.

By using standardization, we have been able to factor out the variability between different measures of hospital performance. In this way, a hospital's performance can easily be compared to others.

This approach is similar to a method often used to grade exams in which test scores are standardized to calculate an average and variability.

Please note that the CBC ratings are not medical advice or recommendation.

The overall rating for each hospital is the average of its standardized scores for each of the five or three indicators used in our report.

Five grades were assigned based on how much the standardized scores or their average deviated from those of a typical hospital of the same size:

Because of the limited amount of data released by CIHI, our ratings do not take into account the inherent uncertainty in any individual measure of hospital performance. The methods hospitals use to keep track of data about quality of care vary widely across Canada. Publicly reported data does not show whether or not a hospital has underreported or wrongly identified certain events.

To account for the uncertainty, CBC would have needed access to the original data reported by hospitals. CBCs the fifth estate requested the information, but 10 of the 13 provinces and territories declined to provide it.

Prince Edward Island, Nunavut and New Brunswick provided some of the data in response to requests to the access to information and privacy coordinators at their health ministries. To ensure a level playing field, we did not calculate a special rate for these provinces using the extra data.

Data for Quebec is largely unavailable because of differences in the way hospitals in that province code information about patient outcomes. The Quebec health ministry and CIHI are just beginning to harmonize their coding systems.

The Quick Stats

Our list of hospital-acquired C. difficile and MRSA infection rates is taken from the most recent reports of provincial health ministries.

They show the annual infection rate per 10,000 in-patient days for the most recent year. The term inpatient days refers to the number of days a hospital provided services to all inpatients.

For provinces that report infection rates per 1,000 patient days, we have converted the rates to 10,000 patient days. For provinces that report infection rates several times a year, the rates are approximated by calculating the average annual rate.

Where provinces report infection rates several times a year, we have calculated the average rate for that year.

A CBC survey sent to hospitals also provided information on safety, patient friendliness and quality. (Claude Vickery/CBC)

There is no Canadian standard for detecting or reporting hospital-acquired infections. Different provinces might measure MRSA or C. difficile in different ways and hospitals might use different tests or surveillance methods to detect the superbugs.

For example, Ontario reports only MRSA infections found in the bloodstream. British Columbia counts all types of MRSA infections as well as patients who may be carrying MRSA but have not developed an infection.

Infection rates are known to vary widely between types of hospitals.

Teaching hospitals, which treat a large number of patients with complicated conditions, can expect to have a higher infection rate than smaller hospitals that treat patients with less complex problems. Our rates have not been adjusted to account for those differences.

Infection rates should not be used to compare hospitals. They are intended as a guide to identify potential problems with infection control and to help patients ask sound questions.

The fifth estate Hospital Survey

Questions on the fifth estate Hospital Survey are based on a review of current literature on best hospital practices and were formulated with the help of a wide range of health care experts.

We asked the chief executive officers of more than 600 acute care hospitals to complete the survey this January; 132 agreed to do so.

The survey included questions about 18 measures that are associated with safety, patient experience and quality of care in a hospital.

The measures we looked at include nurse staffing levels, hand hygiene compliance, the use of electronic medical records, use of safe surgical practices, the extent of palliative care services and a range of patient services designed to address psychosocial needs by accommodating family and friends.

We have not reported on results if a question yielded answers that were open to interpretation.

Our final assessment of the CBC survey includes results for 12 questions divided into three categories: Safety, patient friendliness and quality.

We set benchmarks based on national and provincial best practice standards. If there is no accepted standard, our benchmark is the average or median response, whichever is higher.

In grading the results, CBC awarded a full point, which is represented by a red check mark, for each response that met or exceeded the benchmark in question.

Safety

Here are the four criteria for the safety category and the benchmark necessary to receive a full point:

Regular, reliable pain assessment: The hospital uses a valid and reliable pain assessment tool, and patients pain levels are evaluated during every nursing shift.

Medication reconciliation on admission: The hospital must compare the medications a patient is taking when admitted to those that will be given in the hospital. It must meet or exceed the median compliance rate among hospitals the CBC assessed. The median rate is 84 per cent.

Hand hygiene compliance: Hospital staff must follow proper handwashing procedures. It must meet or exceed the median compliance rate among hospitals the CBC assessed. The median rate is 86 per cent.

Single-bed rooms: This criteria looks at the percentage of single-bed rooms in the entire facility, including acute and non-acute care beds. Hospitals must have at least as many single-bed rooms as the median rate among hospitals the CBC assessed. The median rate is 24 per cent.

Patient friendliness

Here are the four criteria in the patient friendliness category and the benchmark necessary to receive a full point:

Flexible visiting hours: The hospital must have at least some units where close family members can visit at any hour.

Cots available for family members to stay overnight: The hospital must allow family members to stay overnight and have cots available for them.

Full meal service until 6 p.m. daily: The hospital must have a canteen, cafeteria or food court that provides full meals until at least 6 p.m. seven days a week.

Conducts and reports formal patient surveys: The hospital must administer a formal patient survey to patients about their hospital experience and report the findings of the surveys internally at least once every three months.

Quality of care

Here are the four criteria in the quality of care category and the benchmarks necessary to receive a full point:

Use of electronic medical records: Industry group HIMSS grades hospitals on electronic medical record use in eight stages. To receive a check mark, the hospital must have a grade at least as high as the median grade among hospitals CBC assessed. The median grade is 3.17.

Care provided by registered nurses: The percentage of nursing care provided by RNs must meet or exceed the median rate among hospitals CBC assessed. The median rate is 70 per cent. The rate only includes staff who provide direct care in emergency rooms, inpatient areas and operating rooms and recovery. It does not include unit clerks, secretarial staff, clinical nurse specialists, nurse practitioners or educators.

Note: CBC awarded a check mark to four hospitals that did not meet the 70 per cent minimum: St. Josephs Health Care London, Hornepayne Community Hospital, North Bay Regional Health Centre and Stewart Memorial Hospital. These hospitals report that a large majority of their beds are set aside for non-acute care. Because only four hospitals fit this category, CBC did not have enough information to assess the level of care provided by RNs in those hospitals.

Interpreter services: The hospital must have trained professional interpreters available.

Hospital board discusses patient stories: The hospital board and/or quality committee must discuss patient stories related to quality of care at every meeting.

To contact the Rate My Hospital team with tips or information related to the series, please email ratemyhospital@cbc.ca.