November 2016Economic evaluation ofclinical quality registriesFinal reportMonash University and Health Outcomes Australia have preparedthis report on behalf of the Australian Commission on Safety andQuality in Health Care.
Published by the Australian Commission on Safety and Quality in Health CareLevel 5, 255 Elizabeth Street, Sydney NSW 2001Phone: (02) 9126 3600Fax: (02) 9126 3613Email: [email protected]: www.safetyandquality.gov.auISBN: 978-1-925224-65-8 Commonwealth of Australia 2016All material and work produced by the Australian Commission on Safety and Quality inHealth Care is protected by Commonwealth copyright. It may be reproduced in whole or inpart for study or training purposes, subject to the inclusion of an acknowledgement of thesource.The Commission’s preference is that you attribute this publication (and any material sourcedfrom it) using the following citation:The Australian Commission on Safety and Quality in Health Care. Economicevaluation of clinical quality registries: Final report. Sydney: ACSQHC; 2016Enquiries regarding the use of this publication are welcome and can be sent [email protected]
PrefaceClinical quality registries have attracted attention in Australia and internationally as apotential means of improving patient outcomes and the safety and quality of health care.However, historically, there has been relatively little work in Australia quantifying the valueand benefits of clinical quality registries.The Australian Commission on Safety and Quality in Health Care (the Commission)engaged Health Outcomes Australia, through Monash University, to evaluate the economicimpact of five selected clinical quality registries in Australia. The Australian GovernmentDepartment of Health provided funding for the study, with part of the work also funded by theVictorian Department of Health and Human Services.The purpose of this preface, which is the work of the Commission rather than the report’sauthors, is to provide an overview of the project and how the findings may be used in future.Key pointsThe study assesses the cost-effectiveness of five Australian clinical quality registries. Usinga conservative methodology, it shows that Australian clinical quality registries have deliveredsignificant value for money when correctly implemented and sufficiently mature.The key findings of the study are: Each of the five clinical quality registries improved clinical practice at a relatively lowcost, leading to a significant net positive return on investment.The return on investment varied between clinical quality registries, with benefit-to-costratios ranging from 2:1 to 7:1.The minimum expected benefit-to-cost ratio would be 4:1 if full national coverage wereachieved by all five clinical quality registries.While the analysis shows the potential economic benefit of clinical quality registries, thestudy notes that not every clinical quality registry will be cost-effective. Problems such as lowcoverage, inadequate reporting and inadequate collection of information about patientoutcomes will limit the effect of some clinical quality registries, and their value to the healthsystem.The report also finds it is likely there are substantially more individual practitioner, culturaland system-level benefits that flow from the registries than are captured by the study, giventhe study’s focus on financial benefits and costs under very conservative assumptions.ConclusionThe Commission worked closely with the authors and sees this work as a valuable additionto the available literature on the benefits of clinical quality registries.The findings from the five case studies included in the analysis provide evidence of thepotential value of clinical quality registries, and represent the first time this sort of analysishas been conducted in the Australian context.This report will be used to support the development of a national policy context for clinicalquality registries.
Economic evaluation ofclinical quality registriesAustralian Commission on Safety and Quality inHealth CareMonash University and Health Outcomes Australia
Economic evaluation of clinical quality registries1ContentsExecutive summary . 3Conclusions . 6Background and objectives . 8Project background. 8Objectives of this report . 9Perspectives on the impact of Clinical Quality Registries . 9Approach and methodology. 12Overview of registries selected . 12Methodology for evaluation of economic impact . 131. Assessing changes in clinical outcomes and treatment costs. 132. Adjusting for confounding influences by comparing against a controlgroup . 133. Conversion to economic value . 144. Measuring against registry costs . 15Results of the economic evaluation of five case studies . 16Limitations of the approach. 17Opportunities to expand coverage . 17Conclusions . 19Appendix A – Case study summaries. 20Victorian Prostate Cancer Registry . 20Victorian State Trauma Registry . 21Australia and New Zealand Intensive Care Society Adult Patient Database . 23Australia and New Zealand Dialysis and Transplantation Registry . 25Australian Orthopaedic Association National Joint Replacement Registry . 27Appendix B – Case study details . 30Victorian Prostate Cancer Registry . 30Introduction . 30Approach Used . 32Results. 33Attribution of Benefits to the Victorian Prostate Cancer Registry . 35Benefits attribution . 36Opportunities to expand the evaluation . 37Victorian State Trauma Registry . 37
Economic evaluation of clinical quality registries2Introduction . 37Approach Used . 39Results. 40Attribution of benefits to Victorian State Trauma Registry outlierfeedback through the CRG . 42Opportunities to expand the analysis . 43Australia and New Zealand Intensive Care Society Adult Patient Database . 44Introduction . 44Approach used. 45Results. 46Attribution of benefits to the ANZICS APD . 48Opportunities to expand the analysis . 50Australia and New Zealand Dialysis and Transplantation Registry . 50Introduction . 50Approach used. 52Results. 54Attribution of Benefits to the ANZDATA registry . 56Opportunities to expand the analysis . 57Australian Orthopaedic Association National Joint Replacement Registry . 57Introduction . 57Approach used. 59Challenges . 62Other potential approaches to analysis . 63Results. 65Attribution of Benefits to the AOA National Joint Replacement Registry. 68Opportunities to expand the analysis . 70Appendix C: Glossary of main abbreviations . 71Appendix D: Support slides . 73Acknowledgements . 74
Economic evaluation of clinical quality registries3Executive summaryThis study aimed to provide an objective economic basis to support future registryinvestment, and develop and articulate a methodology for other registries to assess theirimpact and cost-effectiveness.The study focussed on financial costs and benefits and found significant net positive returnson investment for each of the registries under very conservative assumptions of attribution.Substantial benefits were measured reflecting improvements to clinical practice andoutcomes over time. These included enhanced survival, improvements in quality of life andavoided costs of treatment or hospital stay.Because the study focussed on financial benefits and costs under very conservativeassumptions, there are substantially more individual practitioner, cultural and system levelbenefits than the evaluation captures.The registries had benefits including enhanced survival for patients, improvements in qualityof life and avoided costs of treatment or hospital stay. There are broader clinical qualityregistry functions that drive continuous improvement and maintenance of safety andstandards. The link between registries and clinical trials allows rapid translation of research intopractice (most of the registries are associated with clinical trial groups where evaluationof clinical problems within the Australian health care system are investigated). As Australia moves towards re-certification of practitioners, registry data, particularlywhere it assesses patient outcomes, will be increasingly important in ensuring quality ofcare delivered by individual practitioners and their teams; clinical quality registries helpdeliver quality assurance of the clinical teams that are contributing to the data sets. The action of a clinical team contributing to a registry results in a substantial contributionto standardisation of care, with additional benefits around team collaboration, sharing ofinformation and team communication.The study conservatively evaluated five registries that have had a measurable influence onclinical practice. The analyses focussed on a selection of indicators within each registry(based mainly on data availability) not the complete set of indicators measured by eachregistry. The evaluations should be viewed as case studies showing that registries, whencorrectly implemented and sufficiently mature, have delivered significant value for money.There is likely to have been considerable clinical, societal and economic benefit driven bycontinuous improvement and changes in practices motivated by registry data and functions.However, the study presents only incremental benefits that can be attributed independentlyto each registry, rather than other influences on practice, such as guidelines, novel therapiesor newly published trials.Not every registry will be cost effective. Problems of low coverage, inadequate feedback andconstrained outcome measures still limit the impact of many registries.An internal rate of return of between 23-52% was measured in the Victorian Prostate CancerRegistry (Victorian PCR), Victorian State Trauma Registr