Measuring Community Programs and Policies in the Healthy Communities Study

Abstract

Childhood obesity is a challenging public health issue facing communities throughout the U.S. Local efforts are believed to be essential to assuring environments that support physical activity and healthy food/beverage consumption among children and their families. However, little is known about how broadly and intensively communities are implementing combinations of programs and policies that address childhood nutrition, physical activity, and weight control. The Healthy Communities Study is a nationwide scientific study in diverse communities to identify characteristics of communities and programs that may be associated with childhood obesity. Data collection occurred in 2013–2015; data analysis will be completed in 2016. As part of the Healthy Communities Study, researchers designed a measurement system to assess the number and scope of community programs and policies and to examine possible associations between calculated “intensity” scores for these programs and policies and behavioral and outcome measures related to healthy weight among children. This report describes the protocol used to capture and code instances of community programs and policies, to characterize attributes of community programs and policies related to study hypotheses, and to calculate the intensity of combinations of community programs and policies (i.e., using the attributes of change strategy, duration, and reach). Childhood obesity is a challenging public health issue facing communities throughout the U.S. Local efforts are believed to be essential to assuring environments that support physical activity and healthy food/beverage consumption among children and their families. However, little is known about how broadly and intensively communities are implementing combinations of programs and policies that address childhood nutrition, physical activity, and weight control. The Healthy Communities Study is a nationwide scientific study in diverse communities to identify characteristics of communities and programs that may be associated with childhood obesity. Data collection occurred in 2013–2015; data analysis will be completed in 2016. As part of the Healthy Communities Study, researchers designed a measurement system to assess the number and scope of community programs and policies and to examine possible associations between calculated “intensity” scores for these programs and policies and behavioral and outcome measures related to healthy weight among children. This report describes the protocol used to capture and code instances of community programs and policies, to characterize attributes of community programs and policies related to study hypotheses, and to calculate the intensity of combinations of community programs and policies (i.e., using the attributes of change strategy, duration, and reach). IntroductionChildhood obesity is a critical public health challenge facing communities.1IOMPreventing Childhood Obesity: Health in the Balance. The National Academies Press, Washington, DC2005Google Scholar, 2Ogden C.L. Carroll M.D. Kit B.K. Flegal K.M. Prevalence of childhood and adult obesity in the United States, 2011-2012.JAMA. 2014; 311: 806-814https://doi.org/10.1001/jama.2014.732Crossref PubMed Scopus (6209) Google Scholar, 3National Center for Health StatisticsHealth, United States, 2011: With Special Features on Socioeconomic Status and Health. DHHS, Hyattsville, MD2012Google Scholar To accelerate progress in obesity prevention, communities throughout the U.S. have engaged—to varying degrees—in creating environments that support healthy nutrition, physical activity, and healthy weight.4IOMAccelerating Progress in Obesity Prevention: Solving the Weight of the Nation. The National Academies Press, Washington, DC2012Google Scholar These activities include efforts to:1increase healthy food/beverage choices (e.g., through programs to improve access to healthy foods in stores and school lunches); and2increase physical activity (e.g., through changes in bike lanes, walking trails, and school policies to assure more time for youth to be physically active).4IOMAccelerating Progress in Obesity Prevention: Solving the Weight of the Nation. The National Academies Press, Washington, DC2012Google Scholar, 5Collie-Akers V.L. Fawcett S.B. Preventing childhood obesity through collaborative public health action in communities.in: Jelalian E. Steele R. Handbook of Child and Adolescent Obesity. Springer Publishers, New York2008Google ScholarYet little is known about the scope and intensity of efforts by communities to implement multiple programs and policies relevant to childhood obesity. To accelerate health promotion efforts, an understanding of how community efforts occur under real-world conditions is needed, as well as learning whether combinations of community programs and policies (CPPs) are associated with children’s diet and physical activity behaviors and healthy weight outcomes.6IOMEvaluating Obesity Prevention Efforts: A Plan for Measuring Progress. The National Academies Press, Washington, DC2013Google ScholarAcquiring this information requires a measurement system for documenting the implementation of multiple CPPs, in different sectors, over time.6IOMEvaluating Obesity Prevention Efforts: A Plan for Measuring Progress. The National Academies Press, Washington, DC2013Google Scholar Because not all CPPs are equally influential, even under optimal implementation conditions, it also requires accounting for differential levels of influence and estimating the intervention strength of CPPs, singly and in combination.7Collie-Akers V.L. Fawcett S.B. Schultz J.A. Measuring progress of collaborative action in a community health effort.Rev Panam Salud Publica. 2013; 34: 422-428PubMed Google ScholarThis report outlines the community measurement system used in the Healthy Communities Study (HCS). The HCS was initiated by the NIH, and this protocol is part of a research design developed by Battelle Memorial Institute, NIH, and university partners. The goal of the HCS is to examine associations between characteristics of CPPs and diet, physical activity, and BMI among children. The overall research design and other components are described more fully in other papers in this supplement. This protocol was used to capture and characterize CPPs occurring in 130 diverse communities throughout the U.S. This report describes the community measurement methods used in the HCS.MethodsContext for Protocol DevelopmentA core research question in the HCS is whether an association exists between CPPs and outcomes related to childhood obesity. The study required a systematic method7Collie-Akers V.L. Fawcett S.B. Schultz J.A. Measuring progress of collaborative action in a community health effort.Rev Panam Salud Publica. 2013; 34: 422-428PubMed Google Scholar to capture, code, characterize, and calculate the intensity of community efforts to prevent childhood obesity. The Work Group for Community Health and Development at the University of Kansas (KU Work Group) led the development of the community measurement protocol based on its prior experience in community measurement.7Collie-Akers V.L. Fawcett S.B. Schultz J.A. Measuring progress of collaborative action in a community health effort.Rev Panam Salud Publica. 2013; 34: 422-428PubMed Google Scholar, 8Fawcett S.B. Sterling T. Paine-Andrews A.L. et al.Evaluating community efforts to prevent cardiovascular diseases. CDC, National Center for Chronic Disease Prevention and Health Promotion, Atlanta, GA1995Google Scholar, 9Fawcett S.B. Schultz J.A. Using the Community Tool Box’s online documentation system to support participatory evaluation of community health initiatives.in: Minkler M. Wallerstein N. Community-based Participatory Research for Health. Jossey-Bass, San Francisco2008Google Scholar, 10Collie-Akers V.L. Schultz J.A. Carson V.L. Fawcett S.B. Evaluating mobilization of neighborhood and faith organizations to reduce risk for health disparities within a REACH 2010 coalition in Kansas City, Missouri.Health Promot Pract. 2009; 10: 118S-127Shttps://doi.org/10.1177/1524839908331271Crossref PubMed Scopus (20) Google ScholarA Community Measurement Subcommittee of the HCS was formed to oversee development of the protocol. Members included representatives from NIH, CDC, Battelle Memorial Institute, and the university partners on the study. The aim of this community measurement approach was to identify CPPs and characterize them sufficiently to explore the main hypotheses of the study; those focused on determining if specific combinations of CPPs are associated with BMI, diet, and physical activity among children.Overview and Development of ProtocolDrawing on community measurement methods developed by the KU Work Group,7Collie-Akers V.L. Fawcett S.B. Schultz J.A. Measuring progress of collaborative action in a community health effort.Rev Panam Salud Publica. 2013; 34: 422-428PubMed Google Scholar, 8Fawcett S.B. Sterling T. Paine-Andrews A.L. et al.Evaluating community efforts to prevent cardiovascular diseases. CDC, National Center for Chronic Disease Prevention and Health Promotion, Atlanta, GA1995Google Scholar the protocol was designed to capture and characterize the multiple and varied community programs and policies emerging over time in diverse communities.For this study, CPPs were defined as the presence of a program, policy, or other activity in the community related to improved nutrition, increased physical activity, or weight control (preventing childhood obesity). Data collection occurred in 2013–2015; data analysis will be completed in 2016. Retrospective collection of CPP data allowed for CPPs to be documented for a longer study period (2003–2015). Types of CPPs included:1.programs (e.g., nutrition program in a youth organization);2.policies (e.g., new physical activity requirement in a school district); and3.environmental changes (e.g., expanded bike path).To be coded as an instance of a CPP, the activity had to be related to the goals of the study, occur during the study period, directly or indirectly address children aged 4–15 years, and be implemented in or affect the high school catchment area (that defined the community).Two primary methods were used to capture instances of CPPs. Initially, community liaisons (CLs) conducted interviews with key informants (KIs; mean, 12 per community) drawn from multiple sectors. KIs were individuals identified as having knowledge of community efforts in their organizations and other settings and sectors (e.g., schools, government). Priority roles of KIs from targeted sectors included school principals, leads of health coalitions, health department and hospital representatives, parks and recreation staff, and staff of other governmental and community-based organizations. Then, document abstraction was used to capture instances of CPPs from archived reports, web searches, and other written sources identified through searches and information from KIs.Not all community programs and policies are likely to have an equivalent effect on health behaviors and related obesity outcomes, even in an ideal implementation scenario. Moreover, the implementation of each program and policy is likely to vary across communities, as a result of differences in local infrastructure, funding, staff availability, and other factors.“Intensity” scoring provides a way to take the potential influence of these activities into account and to estimate the “dose” of intervention actually delivered in each community. This scoring approach also allows for an examination of the association between different amounts and kinds of CPPs with key outcome variables (i.e., dietary intake, physical activity, and BMI) at different times in the study period. In a complex system, an intensity score for a given CPP should take into account multiple attributes, such as reach or change strategy, as well as real-world circumstances that influence associations between programs and policies and changes in childhood obesity.Accordingly, the team defined the construct of intensity7Collie-Akers V.L. Fawcett S.B. Schultz J.A. Measuring progress of collaborative action in a community health effort.Rev Panam Salud Publica. 2013; 34: 422-428PubMed Google Scholar (of CPPs) to reflect three characteristics:1.behavioral intervention strategy (e.g., modifying access was weighted higher than providing information);2.duration (e.g., an ongoing policy was weighted higher than a 1-day event); and3.reach (e.g., a program reaching 20% of the population was rated higher than one reaching 2%).7Collie-Akers V.L. Fawcett S.B. Schultz J.A. Measuring progress of collaborative action in a community health effort.Rev Panam Salud Publica. 2013; 34: 422-428PubMed Google Scholar, 11Cheadle A. Rauzon S. Spring R. et al.Kaiser Permanente’s community health initiative in northern California: evaluation findings and lessons learned.Am J Health Promot. 2012; 27: e59-e68https://doi.org/10.4278/ajhp.111222-QUAN-462Crossref PubMed Scopus (17) Google Scholar, 12Glasgow R. Vogt T. Boles S. Evaluating the public health impact of health promotion interventions: the RE-AIM framework.Am J Public Health. 1999; 89: 1322-1327https://doi.org/10.2105/AJPH.89.9.1322Crossref PubMed Scopus (3317) Google Scholar, 13Glasgow R.E. Klesges L.M. Dzewaltowski D.A. Estabrooks P.A. Vogt T.M. Evaluating the impact of health promotion programs: using the RE-AIM framework to form summary measures for decision making involving complex issues.Health Educ Res. 2006; 21: 688-694https://doi.org/10.1093/her/cyl081Crossref PubMed Scopus (358) Google ScholarTo capture the dynamic unfolding of comprehensive community efforts over time, a composite intensity score was computed for each community for each year of the study.The community measurement protocol was approved by the HCS Executive Committee, IRBs of Battelle Memorial Institute and the University of Kansas, the Observational Study Monitoring Board, and the Office of Management and Budget.Steps in Implementing the Community Measurement ProtocolA four-step protocol was implemented.1. Capture community programs and policies using KI interviews and document abstractionCLs were trained by the KU team to use the community measurement protocol to conduct structured interviews with KIs; the training included workshops, feedback on performance in scoring, and practice to mastery. CLs reviewed community information sources to locate potential KIs within the specific priority roles (e.g., school principal, staff in parks and recreation). Using phone calls and written communications, they communicated the types of information sought and confirmed whether the person was appropriate before requesting an interview time via phone or in person. They also requested any key documents (e.g., final reports, reports to funders) that could help identify relevant a structured KI CLs about instances and attributes of CPPs that occurred in or the defined community over a period. The interview protocol sought to and of specific CPPs that the implemented during the study of the activity (e.g., year of of each CPP (e.g., strategy behavioral was on information on the of CPPs implemented in each community and their information to examine the primary study for CPPs (e.g., specific programs were implemented by and information about This information was used by researchers to help characterize CPPs the KI for more specific and these were by CPPs, the KI interview also sought information to help characterize that and community and were through a Management Management by document abstraction was used to capture other instances of CPPs and to help or information in KI Research staff and reviewed documents (e.g., program reports, reports to funders) and for information about community The information was used to capture CPPs that were using a and scoring instances of community programs and of activities through KI interviews and document abstraction was conducted by CLs and Battelle staff, with training and support by the KU team based on prior V.L. Fawcett S.B. Schultz J.A. Measuring progress of collaborative action in a community health effort.Rev Panam Salud Publica. 2013; 34: 422-428PubMed Google Scholar, 8Fawcett S.B. Sterling T. Paine-Andrews A.L. et al.Evaluating community efforts to prevent cardiovascular diseases. CDC, National Center for Chronic Disease Prevention and Health Promotion, Atlanta, GA1995Google Scholar, 9Fawcett S.B. Schultz J.A. Using the Community Tool Box’s online documentation system to support participatory evaluation of community health initiatives.in: Minkler M. Wallerstein N. Community-based Participatory Research for Health. Jossey-Bass, San Francisco2008Google Scholar for systematic scoring included a with and and scoring that defined an instance of a a program or policy to be as a CPP, it had to all of the occurred (e.g., was not was a program, policy, or other change to the (e.g., walking that in the community during the study was related to nutrition, physical activity, or weight of childhood targeted or children aged 4–15 occurred in or children in the defined Battelle team each documented activity as an instance of a The KU team data by scoring a of identified CPPs. The used was an of levels of and of community programs and policies for key and instances of CPPs were for attributes used in intensity scoring as well as other HCS hypotheses (e.g., behavioral and of (e.g., in specific during the KI interview to help characterize CPPs. to characterize the behavioral by the CPP, CLs were the key behavioral of the community program or behaviors of children were to These were by for those for behavioral related to physical activity included walking or school and to physical among characterize CPPs for intensity scoring, each CPP was coded for three specific intervention strategy used (i.e., providing information and and modifying and or modifying policies and (i.e., of the as a occurring more than or and3.reach (i.e., or the priority population was in or the program or was a based on its strength for the V.L. Fawcett S.B. Schultz J.A. Measuring progress of collaborative action in a community health effort.Rev Panam Salud Publica. 2013; 34: 422-428PubMed Google intensity scores for community programs and the scores through the of CPPs described the study team rated each on a of to and the to a intensity V.L. Fawcett S.B. Schultz J.A. Measuring progress of collaborative action in a community health effort.Rev Panam Salud Publica. 2013; 34: 422-428PubMed Google Scholar for each program and were for all CPPs in for each year of the study describes the and for the potential impact of a The across all three were to a overall intensity Using this the CPPs documented in the HCS could in score from and of influence on to and of The used to calculate intensity scores the for Community and Using V.L. Fawcett S.B. Schultz J.A. Measuring progress of collaborative action in a community health effort.Rev Panam Salud Publica. 2013; 34: 422-428PubMed Google and weight for intensity for intensity of documented community of behavioral intervention policies and and for a community or walking and information and support for physical an program for physical (i.e., throughout the year of the new bike more than during that program that over the local health or of the population to to the policy having an impact on all of the population to the policy implemented at a in the of the population to the program than community in a new data for intensity scoring will be computed using a that all will of that aim to capture the complex among provides of CPPs in communities, by primary goal of and intensity score methods described of for Community and policy intervention strategy walking to and and an to school about how to healthy among children information and intensity in a new of Community were used to the of data using the community measurement protocol. the KU team training for Battelle staff and CLs in and instances of CPPs. the KU team conducted of KI interviews and those for with the protocol. The score required for all was with the CLs than the score and of the HCS and reviewed reports about the of KIs across sectors and the of of the the Battelle and KU staff for scoring documented activities as CPPs. each used of the activity and to key to whether each CPP all for the KU team also all of the variables used in the intensity score to assure and by the KIs and community measurement protocol was designed for the a study with the primary aim of whether is an association between CPPs and behaviors and healthy weight outcomes of children in the overall goal of the community measurement system was to identify CPPs related to healthy nutrition and physical activity of children and to characterize these sufficiently to estimate their scores were computed for each CPP, and for all CPPs in a over a study period. a the protocol used KI interviews and document well as systematic and of intensity scores for overall community study of this for community implementing the protocol required support given the time, and other required to more than interviews with KIs in 130 and assuring performance of CLs required training in the protocol and and feedback for time and for KIs and CLs it to a capture of CPPs that occurred during the study period. on the protocol was and to information about CPPs in the KI is likely a with of a data from communities that KIs could community as as is it is that it is systematic and also be KIs that implementation of CPPs had the of a of the of implementation is a of this data was challenging it required and of information for and of multiple programs and policies into CPPs. also required data from different KIs and document into of activities for and HCS community measurement protocol a number of it a key in prior reports on childhood Childhood Obesity: Health in the Balance. The National Academies Press, Washington, DC2005Google Scholar, Progress in Obesity Prevention: Solving the Weight of the Nation. The National Academies Press, Washington, DC2012Google Scholar, Obesity Prevention Efforts: A Plan for Measuring Progress. The National Academies Press, Washington, DC2013Google Scholar that the to and the web of on physical activity and nutrition that may affect outcomes related to childhood obesity. This community measurement V.L. Fawcett S.B. Schultz J.A. Measuring progress of collaborative action in a community health effort.Rev Panam Salud Publica. 2013; 34: 422-428PubMed Google Scholar a and approach for and the multiple and varied childhood obesity by local communities over intensity score data from the community measurement system, with behavioral and outcome will of possible associations between community efforts and related change and healthy weight and measurement across communities will for of such this intensity scoring approach could be for the potential impact of comprehensive efforts to prevent childhood as well as for specific components of community efforts with this community measurement protocol the of community efforts to prevent childhood obesity in a diverse of U.S. data will be to a understanding of how a and implementation of combinations of programs and policies may to improved health for all its children. IntroductionChildhood obesity is a critical public health challenge facing communities.1IOMPreventing Childhood Obesity: Health in the Balance. The National Academies Press, Washington, DC2005Google Scholar, 2Ogden C.L. Carroll M.D. Kit B.K. Flegal K.M. Prevalence of childhood and adult obesity in the United States, 2011-2012.JAMA. 2014; 311: 806-814https://doi.org/10.1001/jama.2014.732Crossref PubMed Scopus (6209) Google Scholar, 3National Center for Health StatisticsHealth, United States, 2011: With Special Features on Socioeconomic Status and Health. DHHS, Hyattsville, MD2012Google Scholar To accelerate progress in obesity prevention, communities throughout the U.S. have engaged—to varying degrees—in creating environments that support healthy nutrition, physical activity, and healthy weight.4IOMAccelerating Progress in Obesity Prevention: Solving the Weight of the Nation. The National Academies Press, Washington, DC2012Google Scholar These activities include efforts to:1increase healthy food/beverage choices (e.g., through programs to improve access to healthy foods in stores and school lunches); and2increase physical activity (e.g., through changes in bike lanes, walking trails, and school policies to assure more time for youth to be physically active).4IOMAccelerating Progress in Obesity Prevention: Solving the Weight of the Nation. The National Academies Press, Washington, DC2012Google Scholar, 5Collie-Akers V.L. Fawcett S.B. Preventing childhood obesity through collaborative public health action in communities.in: Jelalian E. Steele R. Handbook of Child and Adolescent Obesity. Springer Publishers, New York2008Google ScholarYet little is known about the scope and intensity of efforts by communities to implement multiple programs and policies relevant to childhood obesity. To accelerate health promotion efforts, an understanding of how community efforts occur under real-world conditions is needed, as well as learning whether combinations of community programs and policies (CPPs) are associated with children’s diet and physical activity behaviors and healthy weight outcomes.6IOMEvaluating Obesity Prevention Efforts: A Plan for Measuring Progress. The National Academies Press, Washington, DC2013Google ScholarAcquiring this information requires a measurement system for documenting the implementation of multiple CPPs, in different sectors, over time.6IOMEvaluating Obesity Prevention Efforts: A Plan for Measuring Progress. The National Academies Press, Washington, DC2013Google Scholar Because not all CPPs are equally influential, even under optimal implementation conditions, it also requires accounting for differential levels of influence and estimating the intervention strength of CPPs, singly and in combination.7Collie-Akers V.L. Fawcett S.B. Schultz J.A. Measuring progress of collaborative action in a community health effort.Rev Panam Salud Publica. 2013; 34: 422-428PubMed Google ScholarThis report outlines the community measurement system used in the Healthy Communities Study (HCS). The HCS was initiated by the NIH, and this protocol is part of a research design developed by Battelle Memorial Institute, NIH, and university partners. The goal of the HCS is to examine associations between characteristics of CPPs and diet, physical activity, and BMI among children. The overall research design and other components are described more fully in other papers in this supplement. This protocol was used to capture and characterize CPPs occurring in 130 diverse communities throughout the U.S. This report describes the community measurement methods used in the