FAQ – Measuring quality
What are quality indicators?
A quality indicator is a quantitative measure that can suggest the need for further analysis of medical quality. Unusual results are recognised as statistical anomalies that should be investigated further. A presentation of quality indicators in connection with a more in-depth analysis and improvement of the underlying treatment processes can lead to significant improvements in measured outcomes.
What are the limits of quality indicators?
Quality indicators do not perfectly reflect the clinical reality and do not constitute empirical evidence. The presentation of quality indicators alone does not lead to improvements. Improvements can only be made when these indicators are utilised by management in combination with procedures for analysing processes and eliminating suspected weaknesses.
What is IQM’s goal for using quality indicators?
The German Inpatient Quality Indicators (G-IQI) are designed to make treatment outcomes as transparent as possible and to identify areas for improvement in treatment procedures. They allow hospitals to evaluate their own actions and are intended as the impetus for a targeted review of treatment processes.
From which data are the G-IQIs calculated?
Since the introduction of the DRG system in Germany, an enormous amount of medical information (routine data) is now available in hospitals’ administrative databases. This data is collected for accounting purposes in accordance with Section 301 of the German Social Code Volume V (SGB V) (data records in accordance with Section 21 of the law on the fees for full and partial inpatient hospital services (KHEntgG)) and constitutes the starting point for calculating G-IQIs used by IQM. As this data is checked for accuracy by the attending physician, the hospital’s controlling department and the health insurance companies, the data is among the best verified in the health care system.
What is routine data?
For billing purposes, each service in the hospital is coded by case using ICD and OPS codes to represent diagnoses made and treatment procedures performed. This routine data contains medical information on every patient treated in hospital. The data can also be used to assess quality without the need to collect additional data. This routine data is carefully collected by the hospital physicians and checked by the hospital’s medical controlling department and the various cost objects, all the way up to the Medizinischen Dienst der Krankenkassen (MDK) service organisation for statutory health insurance companies. They cover 100% of inpatients and are available at short notice.
How do we comply with data protection requirements when transferring data?
The data under Section 21 KHEntgG is processed with pseudonymised case numbers. Only the hospital can recover the original case numbers after receipt of the assessments, so that they can use the inpatient record to further analyse the quality of treatment.
Are there disadvantages to using routine data to measure quality?
The indicators calculated with DRG billing data are based only on routine data collected within the DRG system. It is therefore important to ask the right questions of the routine data at hand.
Which quality indicators does IQM use?
G-IQI version 5.2 from routine data comprises more than 380 quality indicators, which are divided into 12 categories:
- Diseases of the heart
- Diseases of the nervous system, cerebral infarction
- Geriatric medicine
- Diseases of the lung
- Diseases of the abdominal organs
- Vascular surgery
- Obstetrics and gynaecology
- Diseases of the bones, joints and connective tissue
- Diseases of the urinary tract and male reproductive organs
- Skin diseases
- Complex, heterogeneous disease syndromes
- Highly specialised medicine
The indicators map a total of 60 major syndromes and indications. In addition, the G-IQI overall and the G-IQI mortality indicators map some selected lengths of stay and the rates of coverage amongst inpatients and/or deceased patients. The G-IQI are currently the most comprehensive set of indicators for monitoring and improving a key treatment outcome, namely the controllable hospital mortality rate.
The regular IQM group assessments also include 23 Patient Safety Indicators (PSI from the Agency for Healthcare Research and Quality (AHRQ)) as well as the 279 indicators for external quality assurance (EQA), which are recommended for publication.
Once a year, IQM members also receive the AOK-QSR clinic report. It provides cross-sector information about long-term outcomes.
How are new indicators developed?
In general, indicators are selected based on the following considerations:
- Frequent standard treatments (e.g. herniotomy in abdominal surgery)
- More complex but relatively common procedures (e.g. colorectal surgery)
- Specialised procedures that are of importance in the particular speciality (e.g. oesophageal surgery)
The outcome and procedure-based indicators are defined such that they are of medical interest and can be controlled through improvement of the treatment processes.
What types of indicators are there?
- Mortality targets (with and without risk adjustment)
- Observation values
- Process and complication indicators
- Volume information
- Indication and care indicators
The precise definitions of the indicators are provided in the current definition manuals.
What types of indicator are there for observed hospital mortality?
When working with quality indicators, three types of indicator tend to emerge in practice:
- Indicators for which mortality is more frequent (relatively high denominator with high numerator) with relatively homogeneous patient population (e.g. myocardial infarction)
- Indicators for which mortality is more frequent but the population is relatively inhomogeneous (e.g. death rate on ventilation > 24 h)
- Indicators for which deaths are rare (e.g. death from hip endoprostheses)
Can hospital mortality be a quality criterion?
IQM considers hospital mortality a major indicator of outcome quality; however, it is only used in indicators where mortality can typically be controlled through quality measures. Quality of outcomes is the very goal of medicine. When outcome quality can be measured, quality management can be implemented to improve processes and structures, which is the only way to achieve improvements in outcomes. A measurement of mortality is not intended to suggest that all deaths are avoidable. The mortality in the IQM indicators can be controlled by improving the quality of medical treatment, e.g. by avoiding errors and complications.
Does measuring mortality result in therapy at any price?
The question of goals and intensity of treatment at end of life is a serious and complex issue for physicians and for medicine as a whole. In making these decisions for individual patients, all IQM hospitals must be guided by the patient’s well-being, their recorded or presumed wishes and the fundamental medical, ethical and legal principles at play.
Does measuring mortality cause clinics to avoid high-risk patients?
This would only be desirable in cases where, for example, elective high-risk patients could be transferred to an institution that is better suited to the risk. Otherwise, it would be completely unacceptable and unethical for indicated and necessary interventions to be avoided just because their outcomes are measured and published.
What does risk adjustment mean?
Whenever reference values for the average hospital mortality in the Federal Republic of Germany (hospital diagnostic statistics) are available from the Federal Statistical Office, IQM’s G-IQI uses the risk of mortality of the patients treated in that particular hospital (determined by age and sex) to calculate the individual target values for the respective hospital. Separate expected values are therefore calculated per hospital and per diagnosis.
Is risk adjustment by age and gender sufficient?
It remains an open question as to whether risk adjustment by age and gender is sufficient, or whether a finer adjustment with additional consideration of comorbities is needed. Analyses (logistic regression) have shown that age is the most important predictor of mortality. The additional variance in mortality due to comorbidities is comparatively small and amounts to only 5 to 15% per indicator. The occurrence of comorbidities correlates strongly with age, which makes age a good predictor of the occurrence of comorbidities. Limiting the adjustment to age and gender prevents the adjustment from being influenced by data quality, which is a concern given the varying degrees to which comorbidities are recorded in hospitals. This type of risk adjustment is sufficient for our purpose, which is to improve the quality of care.
Why were confidence intervals not calculated?
The question of statistical significance is of secondary importance for internal quality management, which aims to improve treatment processes and structures and uses indicators primarily as triggering criteria for record analyses. If unusual indicator values are detected during process analyses such as case reviews, the resulting optimisations need to be implemented in the treatment processes regardless of whether the deviation in the indicator is significant or not.
What does an expected value mean?
The expected value indicates the mortality that would be expected for a patient group of the same age and gender distribution based on the national average. Different clinics will therefore have different expected values if there are differences in the age and gender composition of the patients they treat.
What types of target value are there?
For G-IQIs that can be risk-adjusted for age and gender, the goal of IQM clinics is to get the observed mortality in their own hospital below the value expected based on the patient population they serve. The data required to calculate the expected values comes from the Federal Statistical Office (in-depth diagnostic data for hospital patients). For G-IQIs where it is not possible to calculate expected values, the IQM target values consist of federal average values, which the Technische Universität Berlin calculates from routine data from the DRG statistics provided by the federal and regional statistical offices. IQM target values are not available for indicators whose results are used for observation or informational purposes.
What does SMR mean?
The SMR (standardised mortality ratio) is the ratio of observed to expected mortality. If the observed mortality is greater than the expected mortality, the SMR has a value greater than 1. If the observed mortality is lower than expected, the SMR is less than 1.
Why is benchmarking important for IQM?
For indications where an expected value can be calculated, the hospital’s observed mortality is compared with the expected value. This is equivalent to comparing the hospital’s mortality with the average hospital mortality in Germany. In other words, the measured outcomes are compared with the national average. Measured outcomes are not used to benchmark IQM members relative to each other.
Who else uses these indicators besides IQM?
Hospital mortality is an internationally recognised key indicator for assessing treatment outcomes. The G-IQIs are used by about 1,000 hospitals in Germany on a voluntary basis. In Switzerland (CH-IQI) and Austria (A-IQI), the G-IQIs have been adapted to the particulars of each country. CH-IQI and A-IQI are binding for all hospitals throughout the respective country and are thus used as an official component of the national quality measurement systems.