“ While it may seem like an overwhelming process to some, engagement of diverse partners and utilization of key indicators, will help understand if an impact is being made. ”
AUTHOR: Sandra Melstad, SLM Consulting, LLC
Are you making an impact? Knowing the answer to this question can help assess the value and effectiveness of programs and initiatives focused on improving population health, as well as adjust the program or initiatives to make an impact. Is it important to emphasize the importance and value of using evidence-based findings to guide decision-making. Utilization of evidence-based findings provides a platform to guide development and implementation of programs and initiatives, highlighting what works to address issues in specific populations, such as health equity, childhood obesity, skin cancer prevention in youth, or access to healthy foods in rural communities. However, additional data and information is necessary to evaluate the impact and effectiveness of those programs and initiatives. Indicators provide that additional data and information to help determine if a program or initiative is being implemented as expected and outcomes are being achieved (Centers for Disease Control and Prevention, 2016).
What is an indicator? An indicator is a “specific, observable and measurable characteristic that can be used to show changes or progress a program is making toward achieving a specific outcome” (UN Women, n.d.). An indicator can be quantitative or qualitative measure that describes community conditions, issues, etc., such as health status, economics, social or cultural conditions, or that measure components of a program or initiative, including input, process, and outcomes. Indicators help answer the question; “How are we doing regarding the conditions we care about?” Results-Based Accountability, focused on how to do the work, also suggests assessing impact from indicators by asking, “How well did we do it?”, “How much did we do?”, and “Is anyone better off?” (Results Based Accountability, n.d.).
As research continues to highlight addressing social, economic, and environmental factors is key to improving population health, it is important to collect indicator data on these factors that contribute health outcomes. For example, a program focused on understanding the high percentage of rural community residents without access to healthy foods, could help understand the impact the program or initiative is having by gathering data on indicators such as, the percentage of people who live in a food desert, food insecurity rates, poverty rate, percentage of households without a motor vehicle, and the number of farmer’s markets available. These indicators help answer the question of “why is there poor food access in this community”? Just collecting information on lack of grocery stores is not enough to explain the issue.
Why should you use indicators? According to the Community Tool Box (2018), “community-level indicators provide bottom-line evidence of the impact of the program or initiative, help determine effects of key components of the program or initiative, help push issues to the forefront of the public agenda and show that positive results can help secure more support for the program or initiative”. Specifically, community health indicators provide an objective method to measure the data, progress towards a goal, describe the long-term effect of the efforts, determine the effects of key components of the program or initiative, and increases support for policy, systems, and environment changes important to address population health.
How are indicators determined? Indicators or specifically health indicator data is available from multiple local, state, and national sources, as well as may be collected from primary data sources. Indicators should reflect the goal of the program or initiative. Reviewing evidence-based indicators is one method to understanding the overall health of a community and understanding the long-term effect of implementing initiatives to address community health. However, communities may find that the recommended indicators may not be useful to support the desired outcomes. Characteristics of good indicators include: meaningful, relevant, and actionable; valid and accurate; outcome-oriented; measures that available; stable, timely, and reliable; possible to collect; and sensitive to the program or initiative. Steps to identify indicators include:
1. Establish a framework for determining what indicators you want to collect data on. Work collectively with partners and key stakeholders involved in the program or initiative to determine a focus area, answering the question: “What themes, issues or goals are most important for measuring the current and future conditions of the health of the community?”
2. Identify and select indicators that meet the established focus area, issues, and goals. It is important to include key community groups, partners, and data providers integral to the program or initiative to ensure a comprehensive process and obtain engagement from stakeholders. Review other indicator projects and/or rely upon experts to help select appropriate measures. Identify indicators from existing local, state, and national sources that embody the goal of the program or initiative and are predictive, then determine if additional indicators need to be developed in order to assess whether the program or initiative is making an impact. While the number of indicators that can be identified can be wide-ranging, it is important to identify core indicators that truly measure impact. Essentially, quality versus quantity.
3. Compile and Summarize the Data. It is important to keep track of the indicators through data monitoring system, such as a data dashboard (e.g. Black Hills Knowledge Network) or simply an Excel file, to track change over time. In addition, consider opportunities to summarize findings using different data visualization methods beyond the tried and true report, such as infographics, graphs, or informational briefs. Consider the audience when summarizing findings and ensure that they can effectively interpret and utilize the findings as they are intended. Resources such as Tableau , Community Commons, or GIS Data Visualizations from the Institute for Health Metrics and Evaluation are just some of the many tools available to support summarization of data.
4. Disseminate the Data. Inform and educate a variety of audiences, key to the program and initiative, including the community-at-large, local media, and stakeholders. Use the data as a resource to encourage public support and highlight the positive results of the program or initiative. Afterall, you likely have enough door stops. The data does no good if it is not shared.
Recently I had the opportunity to learn and engage with experts across the world, who are focused on using data, indicators, and data dashboards to support communities to achieve impact, including the Healthy Cities Livable Group in Australia, the community indicator project, Peg, in Winnipeg, Canada, and the Heart of New Ulm Project in New Ulm, Minnesota. The common theme throughout this learning opportunity was to utilize cross-sector partners and data to drive action. Data and partnerships are powerful factors in improving where people live, work, learn, and play. One key point gathered from this conference is that it is important to consider identifying indicators that describe the good things happening in communities rather always focusing on what is bad…”What is the community doing well?” It is also important to share data with partners and organization, albeit through data sharing agreements or otherwise, but just consider how much impact can be made by sharing and collaborating. While it may seem like an overwhelming process to some, engagement of diverse partners and utilization of key indicators, will help understand if an impact is being made. So, are you making an impact?
References
1. Center for Community Health and Development. (2018). Community Tool Box, Gathering and Using Community-Level Indicators. University of Kansas. Retrieved from https://ctb.ku.edu/en/table-of-contents/evaluate/evaluate-community-initiatives/community-level-indicators/main
2. Black Hills Knowledge Network. (2018). Retrieved from https://www.blackhillsknowledgenetwork.org/
3. Institute for Health Metrics and Evaluation. (2018). University of Washington. Retrieved from http://www.healthdata.org/
4. Results-Based Accountability. (n.d.). Implementation Guide, A Comprehensive Resource for the RBA/OBA Community. Retrieved from http://raguide.org/
5. United Nationals Entity for Gender Equality and the Empowerment of Women. (2012). Indicators. Retrieved from http://www.endvawnow.org/en/articles/336-indicators.html
6. Centers for Disease Control and Prevention, Program Performance and Evaluation Office. (2016, December 2). Indicators. Retrieved from https://www.cdc.gov/eval/indicators/index.htm