Summary Various industries, including health care, have adopted quality improvement (QI) to enhance practices and outcomes. As demands on the U.S. public health system continue to increase, QI strategies may play a vital role in supporting the system and improving outcomes. Therefore, public health practitioners, like leaders in other industries, are developing QI approaches for application in public health settings.
Quality improvement in public health involves systematically evaluating public health programs, practices, and policies and addressing areas that need to be improved to increase healthy outcomes. Although QI methods and techniques have only recently been applied to public health, public health systems offer a wide range of opportunities for implementing, managing, and evaluating QI efforts.
The growing field of Public Health Systems and Services Research (PHSSR) offers the potential to contribute to and support QI efforts in public health. PHSSR examines the delivery of public health services within communities as well as the outcomes that result from dynamic interactions within the public health system. By examining
the public health system, stakeholder interactions, delivery of services, and outcomes, PHSSR can inform and support the implementation of QI initiatives.
Most recently, national, state, and local levels have made notable progress in quality improvement in public health.1, 2 One initia- tive credited with achieving progress is the Multi-State Learning Collaborative (MLC). The MLC aims to inform the national accredi- tation program, incorporate quality improvement practice into pub- lic health systems, promote collaborative learning across states and partners, and expand the knowledge base in public health.
Bringing together state and local practitioners and other stakeholders in a community of practice to achieve MLC goals has yielded several best practices and lessons for public health stakeholders. However, more work is needed if QI is to become standard practice in public health—particularly in understanding health departments’ readiness for change, building the evidence base for effective public health QI practices in the context of the public health system, and examining the sustainability of successful projects, and identifying the determinants
of transformational change.
Quality Improvement in Public Health: Lessons Learned from the Multi-State Learning Collaborative
Background: AcademyHealth’s 2009 Annual Research Meeting At the 2009 Annual Research Meeting (ARM), June 28–30, in Chicago, AcademyHealth convened a panel of three experts, members of the Multi-State Learning Collaborative (MLC), to discuss their experiences in implementing quality improvement collaboratives in public health. Leslie Beitsch, M.D., J.D., associate dean for health affairs and professor of family medicine and rural health at the College of Medicine, Florida State University; Joe Kyle, director of the Office of Performance Management, South Carolina Department of Health and Environmental Con- trol; and William Riley, Ph.D., associate professor and associate dean at the School of Public Health, University of Minnesota, participated in the panel. Brenda Joly, Ph.D., assistant research professor in health policy and management at the Muskie School of Public Service, University of Southern Maine, provided an overview of the evaluation of the MLC initiative and moderated the discussion. The speakers highlighted some of the challenges and opportunities associated with implementing, managing, and evaluating quality improvement efforts in public health. This issue brief is based largely on the panel’s presentations.
Quality Improvement in Public Health: Lessons Learned from the Multi-State Learning Collaborative
Introduction The attacks of September 11, 2001, the bioterrorism threats that
followed, and the series of natural disasters that devastated parts
of Florida and the Gulf Coast exposed the long-neglected public
health infrastructure in the United States. Public outcry and elected
officials demanded redress of the inadequacies of the public health
system at all levels of government. As a result, the demands on the
public health system have continued to evolve and expand.
With increased demand has come growing expectations of
accountability. Policymakers are increasing the scrutiny with
which they measure the success of public health interventions and
make resource allocation decisions. As such, those implementing
public health programs may wish to consider the role of quality
improvement and seek opportunities for integrating, managing,
and evaluating QI efforts in public health systems.
Formally defined by the U.S. Department of Health and Human
Services Public Health Quality Forum (PHQF) in August 2008,
“Quality in public health is the degree to which policies, programs,
services, and research for the population increase desired health
outcomes and conditions in which the population can be healthy.”1
PHQF provides further guidance on the aims of public health QI,
drawing on the aims set forth for QI in patient care in the 2001
Institute of Medicine report, Crossing the Quality Chasm: A new
health system for the 21st century (see text box). The goals of QI
in public health involve systematically evaluating public health
programs, practices, and policies and addressing areas that need to
be improved in order to optimize population health.3
Processes to facilitate QI include accreditation, performance measurement, and the development of quality standards. As the core entities that make up the public health system, local and state public health departments play a vital role in carrying out those processes and advancing QI. For public health departments to encourage the adoption of QI in public health systems, they must address the existing challenges to implementing QI in public health practice and build a culture for QI. Those challenges include identification of meaningful goals and objectives, data collection limitations, lack of workforce training, insufficient research evidence, and dearth of knowledge about best practices.5 Various initiatives have addressed barriers to QI in public health.
Major Initiatives for Quality Improvement in Public Health Recently, three major initiatives have been achieving notable progress
in building QI in public health at the local, state, and national levels.
First, the U.S. Department of Health and Human Services created the
PHQF in April 2008, reflecting the federal government’s support of
and commitment to public health QI. Leading a national movement
for coordinated efforts to improve QI across all levels of the public
health system, the PHQF is charged with identifying “a set of aims for
improvement of quality in public health, a framework to guide and
standardize QI efforts, priority areas for QI in the public health system,
and a core set of quality indicators in each of the priority areas.”6
Building on the PHQF framework, QI in public health has been defined
as the use of a deliberate and defined improvement process with a
distinct management approach to ensure that health departments
consistently meet their communities’ health needs and strive to improve
the health status of their populations.7
Characteristics of Quality in Public Health Through a consensus-building process with public health system partners led by the U.S. assistant secretary for health, the Public Health Quality Forum has identified aims that characterize public health quality improvement, thereby framing and promoting consistency with the implementation of quality improvement initiatives. Public health practices across the entire system should reflect the following characteristics:4
• Population-centered–protecting and promoting healthy conditions and the health of the entire population
• Equitable–working to achieve health equity
• Proactive–formulating policies and sustainable practices in a timely manner while rapidly mobilizing to address new and emerging threats and vulnerabilities
• Health-promoting–ensuring policies and strategies that advance safe practices by providers and the population and that increase the probability of positive health behaviors and outcomes
• Risk-reducing–diminishing adverse environmental and social events by implementing policies and strategies to reduce the probability of preventable injuries and illness or other adverse outcomes
• Vigilant–intensifying practices and enacting policies to support enhancements to surveillance activities (e.g., technology, standardization, systems thinking/modeling)
• Transparent–ensuring openness in the delivery of services and practices with particular emphasis on valid, reliable, accessible, timely, and meaningful data that are readily available to stakeholders, including the public
• Effective–justifying investments by using evidence, science, and best practices to achieve optimal results in areas of greatest need
• Efficient–understanding costs and benefits of public health interventions and facilitating optimal use of resources to achieve desired outcomes
Second, the Public Health Accreditation Board (PHAB), established
in May 2007, supports the development of a voluntary national
accreditation program for state, local, territorial, and tribal public
health departments.8 The goal of the accreditation program is to
improve and protect the health of every community by advancing
the quality and performance of public health departments. Through
the process of meeting national accreditation standards, health
departments will be able to identify and implement tools and
methods needed to ensure their communities’ health and safety.
Third, the Multi-State Learning Collaborative (MLC) is a major
initiative focused on QI at the local and state levels.9 A three-phase
initiative established in 2006 and slated to conclude in 2011,
the MLC informs the national accreditation program, prepares
public health departments for accreditation, incorporates quality
improvement practice into public health systems, promotes
collaborative learning across states and partners, and expands the
knowledge base in public health.
Multi-State Learning Collaborative: Incubators for Quality Improvement In the first of the MLC’s three phases, local and state health
departments in five states served as a “real-time laboratory”
and explored the use of accreditation as a quality improvement
process.10 In the second phase, a cohort of 10 states came together
to examine best practices for teaching and implementing QI
practices at the local and state levels. The effort included QI
training and consultation, use of small in-state collaboratives,
and increased outreach to local health departments. The MLC’s
third phase expanded the project to 16 participating states and
focused on implementing public health QI activities to achieve
specific, measurable goals. By the MLC’s conclusion, the efforts of
participating states will have contributed to the development of a
national voluntary accreditation program, bolstered QI capacity,
institutionalized QI practice in health departments to prepare
them for national accreditation, and demonstrated progress on QI
goals through specific, measurable improvements.
Considerations for Implementing and Managing Quality Improvement Efforts Organizing local and state practitioners and other stakeholders
into a community of practice to achieve the MLC’s goals has
yielded several best practices and lessons for public health
stakeholders, particularly for local and state health departments
looking to engage in or improve their quality improvement
practices and obtain and uphold accreditation.11
• Creation of a community of QI practice requires a constant focus on outreach to stakeholders across all levels of the public
health system, including within the state, the collaborative,
the accreditation community, and the broader public health
• Four primary principles should guide the development of a QI collaborative and the engagement of partners: reliance on
existing work and information (e.g., data, evidence), creation
of a transparent process open and visible to all participants,
development of a participatory process to ensure that all
stakeholders have an opportunity to voice their opinions, and
achievement of consensus among stakeholders to the greatest
• States and collaboratives must select QI target areas that can be defined in standard and specific terms. They must also measure
an important aspect, result, or outcome of public health work and
then implement activities to improve performance against the
• Target areas for QI in public health may be outcome-, capacity-, or process-related. Outcome target areas include a
reduction in the incidence of vaccine-preventable disease, a
decrease in the preventable risk factors predisposing to chronic
disease, a reduction in infant mortality rates, limiting the
burden of tobacco-related illness, and a decrease in the burden
of alcohol-related disease and injury.12 Capacity and process
target areas include community health profiles, culturally
appropriate services, health improvement planning, assurance
of a competent workforce, and customer service.
• Accreditation is expected to support public health agency performance through QI. As such, preparation for accreditation
may involve the implementation of QI strategies. However,
accredited agencies and departments with limited QI experience
may benefit by building QI into domains where it might not have
• Local and state health departments interested in implementing QI should consider the development of an overall performance
improvement plan and formation of QI advisory councils and
offices. These formal structures will help health departments
address any performance gaps uncovered by accreditation or
assessment. They will also indicate the need for QI manuals,
technical assistance for QI and accreditation, tracking systems that
monitor progress, and coordination of community involvement in
Community Health Improvement Planning (CHIP).
• States should consider three effective process models and tools for implementing QI activities in public health systems: the
Plan-Do-Study-Act (PDSA) model, the Institute of Healthcare
Improvement’s (IHI) model, and “mini-collaboratives” –smaller,
Quality Improvement in Public Health: Lessons Learned from the Multi-State Learning Collaborative
Case Study: Building a QI Culture in Minnesota’s Local Public Health Departments The Minnesota Public Health Collaborative for Quality Improvement,
a partnership of the Minnesota Department of Health, Local Public
Health Association, and University of Minnesota School of Public
Health, participated in the MLC’s second phase.13 A main focus of the
collaborative was the integration of QI into public health practice by
building a QI culture in the Minnesota public health system, which
includes 75 local health departments and the Minnesota Department
of Health. Goals included building the public health workforce’s
capacity to use QI tools and methods, developing and testing a model
for using QI to improve public health practice, and creating strong
linkages between practice and academia.
Half of the state’s local health departments participated in identifying
eight target areas for QI and implementing eight corresponding
projects. Seven of the projects had positive outcomes. Survey
responses from participants about the projects indicated that efforts
to build a culture of QI were effective. For example, 75 percent
of respondents saw quality improvement as relevant to their
organizations; 60 percent “strongly agreed” that the collaborative gave
them new, useful information about QI; 72 percent intended to use
QI practices in future projects; and 79 percent rated management’s
interest in the QI project as “very supportive.” Overall, approximately
250 local, state, and university public health professionals were trained
in 10 QI methods. The Minnesota Collaborative shared the results
of the eight projects with local public health departments across the
state and documented several lessons learned and best practices for
teaching and implementing QI practices in support of a QI culture
in public health at the local and state levels. Lessons learned include
• Accurate definition of the problem and goal is an essential first step that leads to a realistic assessment of capabilities and potential
solutions and helps ensure success.
• Slight modifications to existing QI models may make the models more acceptable to those in public service settings.
• Collaboratives should tailor the pace and scope of learning to the audience’s capacity.
• Given that the incentives to participate are small, relevance and gains need to be significant.
• It is important to use evidence-based interventions when possible.
• The collaborative framework and model serve as a way of simultaneously managing a series of projects among several
entities, not just individual projects one at a time.
Case Study: South Carolina’s Application of the Institute of Healthcare Improvement’s Model for QI in Public Health The South Carolina Department of Health & Environmental
Control (SC DHEC) participated in the third phase of the MLC
and is one of two states adapting and implementing the IHI’s
Breakthrough Series model for its QI collaborative, which focused
on tobacco use and exposure to secondhand smoke.14 SC DHEC
collaborative partners came to a consensus on this QI target area,
given the strong evidence base available substantiating tobacco
interventions and the opportunity to influence development
of a new policy as DHEC clinics were beginning to implement
tobacco-use screening. For two reasons, SC DHEC employed the
First, the IHI’s model relies on the spread and adaptation of
existing knowledge to several settings to accomplish a common
aim and thus fit well with SC DHEC’s highly centralized and
integrated organizational and operational structure. The SC
DHEC public health system includes the state office and 55 clinic
sites in 46 counties organized into eight regions. Under its health
services and performance management system, the SC DHEC
continuously monitors more than 200 measures across all program
and functional areas—a transparent system that allows department
employees to track the progress of many system indicators.
Second, SC DHEC used the IHI’s model because the department’s
tobacco-use collaborative met the model’s criteria for existence of a
gap between evidence and practice (i.e., clinical practice guidelines
are effective in increasing tobacco-use cessation, yet public health
departments apply the guidelines unevenly across programs),
existence of examples of better performance, and existence of a strong
“business case” for the intervention.
Several findings and recommendations around the IHI’s model grew
out of South Carolina’s experience as follows:
• Appropriate training of staff on the QI methodology and timely technical assistance are essential to ensuring that staff have the QI
skills needed for implementation, thereby increasing their buy-in
for carrying out QI methods.
• Policy and procedure changes should be carried out with adequate time for staff to test how best to implement the changes before
final adoption. Documentation of promising practices from the
testing phase should accompany the rollout of new policies and
• Staff members need sufficient authority to function independently so they can make decisions on their own when evidence is not
strong enough to serve as a clear guide.
• Will (e.g., visible commitment, peer pressure, focus on results), ideas (e.g., focus on content), and execution (e.g., tests of change,
implementation) are essential to a collaborative’s success.
Next Steps Through their participation in the MLC, states have become more
focused on accreditation, developing expertise in QI and forming
collaboratives that suit their needs. The MLC has helped inform
stakeholders on what to do to engage in public health QI activities.
Preliminary evaluation of the MLC has surfaced information on
organizational culture, QI capacity and competencies, QI initiatives
and experiences, and alignment, integration, and spread of QI
initiatives in participating states.
Although MLC states demonstrate solid progress in implementing
QI methods to prepare health departments for accreditation,
improve agency performance, and ultimately improve the health of
their communities, more research is needed on QI in public health.
Specifically, health departments’ readiness for change and ability to
improve outcomes needs further study.15 It is also important to build
the evidence base for effective public health QI practices in the context
of the public health system and infrastructure and to examine the
sustainability of successful projects to date so that QI in public health
will become standard practice.
Endnotes 1 Public Health Quality Forum, U.S. Department of Health and Human Services.
“Consensus Statement on Quality in the Public Health System,” August 2008. Accessed October 10, 2009, from http://www.hhs.gov/ophs/initiatives/quality/quality/ phqf-consensus-statement.pdf.
2 Institute of Medicine, Committee on Quality of Health Care in America. Crossing the Quality Chasm: A new health system for the 21st century. Washington, DC: National Academy Press, 2001.
3 Public Health Quality Forum, U.S. Department of Health and Human Services. “Consensus Statement on Quality in the Public Health System,” August 2008. Accessed October 10, 2009, from http://www.hhs.gov/ophs/initiatives/quality/quality/ phqf-consensus-statement.pdf.
4 Ibid. 5 Leep, C.J. “Quality Improvement at Local Health Departments: Strategies for the
Adoption of Quality Improvement for Public Health Impact.” National Association of County and City Health Officials, January 18, 2008.
6 Public Health Quality Forum, U.S. Department of Health and Human Services. “Consensus Statement on Quality in the Public Health System,” August 2008. Accessed October 10, 2009, from http://www.hhs.gov/ophs/initiatives/quality/quality/ phqf-consensus-statement.pdf.
7 Riley, W.J. et al. Commentary “Defining Quality Improvement in Public Health.” J Public Health Management Practice, Vol. 16 No. 1, 2010, pp. 5-7. 8
8 Public Health Accreditation Board. Homepage. Accessed October 5, 2009, from http://www.phaboard.org/.
9 National public health partners collaborated to inform and support the MLC, including the American Public Health Association (APHA), Association of State and Territorial Health Officials (ASTHO), National Association of County and City Health Officials (NACCHO), National Association of Local Boards of Health (NALBOH), Public Health Foundation (PHF), Centers for Disease Control and Prevention (CDC), Robert Wood Johnson Foundation (RWJF), Public Health Informatics Institute (PHII), and Public Health Accreditation Board (PHAB). The Multi-State Learning Collaborative is managed by the National Network of Public Health Institutes (NNPHI) and the Public Health Leadership Society (PHLS).
10 Beitsch, L. “Quality Improvement: The Multi-State Learning Collaborative Approach.” Presentation at the AcademyHealth Annual Research Meeting, Chicago, IL, June 30, 2009.
11 These “lessons learned” were summarized from the presentation by Leslie Beitsch at AcademyHealth’s panel on “Quality Improvement in Public Health.” AcademyHealth Annual Research Meeting, Chicago, IL, June 30, 2009.
12 Multi-State Learning Collaborative project states developed a menu of target areas related to outcomes based on experiences of state quality improvement programs with health status measures, Healthy People 2010, MAPP Community Health Assessment, and the Community Health Status Indicators project. The states also developed a menu of target areas related to capacity/process based on state/ local leveraging of opportunities, National Public Health Performance Standards, NACCHO’s operational definition, Healthy People 2010 infrastructure chapter, Institute of Medicine 2002 report, and a cross-walk of performance gaps.
13 Minnesota case study information and “lessons learned” summarized from the presentation by William Riley at AcademyHealth’s panel on “Quality Improvement in Public Health.” AcademyHealth Annual Research Meeting, Chicago, IL, June 30, 2009.
14 South Carolina case study information and lessons learned summarized from the presentation by Joe Kyle at AcademyHealth’s panel on “Quality Improvement in Public Health.” AcademyHealth Annual Research Meeting, Chicago, IL, June 30, 2009.
15 Institute of Medicine, Committee on Quality of Health Care in America. Crossing the Quality Chasm: A new health system for the 21st century. Washington, DC: National Academy Press, 2001.
16 Preliminary evaluation information summarized from the presentation by Brenda Joly at AcademyHealth’s panel on “Quality Improvement in Public Health.” AcademyHealth Annual Research Meeting, Chicago, IL, June 30, 2009.
17 An online quality improvement survey tool was administered by external evaluators as part of an annual NACCHO survey of local health department representatives from the 16 states participating in the third phase of the Multi-State Learning Collaborative. Lead public health officials completed 80 percent of the surveys, for an average response rate of 60 percent (n = 690). Results are from the first year of the third phase of the Multi-State Learning Collaborative. Limitations of the survey include reliance on self-report of one agency representative, use of core constructs from literature that have not been tested with public health agencies, and application of a blunt instrument for assessing QI status.
Preliminary Evaluation of Quality Improvement in Local and State Public Health Agencies in the Multi- State Learning Collaborative A preliminary evaluation of QI in the local and state public health agencies participating in the Multi-State Learning Collaborative sheds light on current QI approaches and the impact of efforts to create a favorable environment for QI to flourish.16, 17 Preliminary results show that leaders are receptive to new ideas for QI, that the impetus for QI is driven by internal desire, and that staff share agency data for performance improvement, all pointing to agencies’ successes in developing a QI culture within their organizations. However, the assessment of QI capacity and competency shows that, while public health leaders and staff participate in training on QI methods, staff at all levels are not participating in QI efforts. Data on QI initiatives and experience indicate that more than half of public health agencies have implemented a formal process to improve the performance of a specific service or program, process, or outcome and that almost half of public health agencies have implemented at least one formal QI project. Results for alignment and spread suggest that public health departments/systems need to increase the availability of data to evaluate quality of services. In addition, QI needs to be integrated into the way staff members work, and programs need to increase their adoption of QI ideas.