Background
Quality audit tools are the cornerstone for understanding and quantifying the impact of healthy eating and physical activity (HEPA) supportive policies.1 Audit tools  are frequently used as a method to evaluate policy intervention effectiveness (i.e. policy impact evaluation) and as a way to track changes in the environment over time.2  For audit tools to be useful in informing decision makers about an intervention’s effectiveness they must demonstrate the ability to detect changes in the policy environment characteristics.2,3 Yet in the majority of cases, such tools are rarely evaluated for how well they capture changes in policy environment characteristics. The Healthy Afterschool Activity and Nutrition Documentation (HAAND) instrument is a newly developed audit tool consisting of two sub-indices - the Healthy Afterschool Program Index for Physical Activity (HAPI-PA) and for Nutrition (HAPI-N). Each of the HAPI-PA and HAPI-N indices consist of 7 domains (i.e. polices, training, child involvement, evaluation, curriculum, screen time or access to vending machines, scheduling of activity or quality of snack served). The HAAND instrument is designed to quantify the physical activity and nutrition environment within afterschool programs. Although the validity and reliability of the HAAND instrument has been established.4 To date the responsiveness of the HAAND to changes in the policy environment and practice has not yet been established.

Objectives
The purpose of this study is to examine the responsiveness of the HAAND instrument to changes in the policy environment characteristics following implementation of strategies aimed at helping afterschool programs meet HEPA policies.

Methods
Twenty afterschool programs across South Carolina serving over 1700 children (5-12 years old) participated in a group randomized controlled trial with 10 intervention and 10 control sites.  The intervention, STEPs (Strategies To Enhance Practice), focused on intentional programming of healthy eating and physical activity in each afterschool programs’ daily schedule, and included a grocery store partnership to reduce price barriers to purchasing FV, professional development training to promote physical activity to develop core physical activity competencies, as well as ongoing technical support/assistance. Responsiveness to change of the HAAND instrument was assessed using HAAND baseline data (Spring 2013) and post intervention year 1 follow up data (Spring 2014). HAAND data was collected during a single day’s visit and consisted of an interview with the ASP site leader, review of existing documents, and observation of the scheduled physical activities. HAAND scores were computed and policy environment characteristics changes were calculated as difference between baseline and post intervention (year 1 - baseline). Wilcoxon sign-rank tests were used to examine the difference between intervention and control group.

Results
Over the entire intervention group the total HAAND median scores (sum of HAPI-PA & N scales) showed a significant increase of 16 points (p = 0.001,) whereas the control group total HAAND median score increased only 1.0 point (p = 0.623) from baseline to year 1 follow up. The intervention group median (SD) baseline for total HAPI-PA was 9.5 (3.5), and increased to 13.5 (1.6) by post-assessment (p = 0.002).  For the HAPI-N, the intervention group median (SD) baseline was 6.5 (3.9) and improved to 21.0 (3.1) by post-assessment (p = 0.001). In comparison, the HAAND scores showed non- significant changes between baseline and year 1 follow up in the control group. With total HAPI-PA median (SD) scores showing a slight decrease from 9.0 (2.4) to 8.5 (3.4) (p = 0.500) and the HAPI-N scores median (SD) also showed a slight decrease from 8.0 (4.2) to 7.5 (4.7) (p = 0.6367) from baseline to year 1 follow up.

Conclusions
The HAAND instrument can detect changes to the policy environment characteristics of afterschool programs. Additional studies should examine the responsiveness to change of the HAAND instrument in comparison to other tools designed to evaluate HEPA policy environment and practices in the afterschool program setting.

Implications
The ability to detect policy changes and identify effective policies in a timely and efficient manner is increasingly important for policy makers. The HAAND instrument is capable of detecting changes in afterschool programs’ policy and practice environment. This makes the HAAND instrument a valuable tool that serves as an outcome or process measure to evaluate policy and practice changes over time.

References

1. Saelens BE, Glanz K. Work group I: Measures of the food and physical activity environment: instruments. Am J Prev Med 2009;36(4 Suppl):S166-70.
2. Kelly CM, Hoehner CM, Baker EA, Brennan Ramirez LK, Brownson RC. Promoting physical activity in communities: Approaches for successful evaluation of programs and policies. Evaluation and Program Planning 2006;29(3):280-292.
3. Story M, Giles-Corti B, Yaroch AL, Cummins S, Frank LD, Huang TT, et al. Work group IV: Future directions for measures of the food and physical activity environments. Am J Prev Med 2009;36(4 Suppl):S182-8.
4. Ajja R, Beets MW, Huberty J, Kaczynski AT, Ward DS. The healthy afterschool activity and nutrition documentation instrument. Am J Prev Med 2012;43(3):263-71.