Background:
Daily physical activities such as walking to school can help reduce health burdens among children, by enabling the development of life-long active lifestyle habits. Toward this end, a method to systematically, comprehensively, and objectively assess the safety and walkability of built environments at and around schools is necessary to identify the barriers and facilitators specific to each school’s unique setting and design characteristics. Environmental audit instruments have served as effective measurement tools for this purpose. While several audit instruments have been developed for assessing streets, parks and trails, to our knowledge, no instrument has been designed to capture the school environment including the school ground and the nearby streets.
Objectives:
This paper introduces a school audit instrument that includes three sub-audits: (a) street segment audit, (b) school site audit and (c) map audit. It presents the conceptual basis and the development process of this instrument, a training manual and protocol; and discusses the methods and results of the reliability assessments. Like other audits, this audit focuses on the modifiable attributes of the built environment that can be improved and managed through policy and environmental interventions.
Methods:
The development of school audit instrument was carried out for two years (2008-2009) and the development process involved five steps: (1) developing the initial instrument based on the review of existing audit tools and relevant literature; (2) pretesting on one urban, one suburban and one rural school; (3) pilot testing on one urban school; (4) developing the final instrument, training manual and protocol; and (5) conducting reliability tests on the final instrument.
The three main constructs (Origins-Destinations, Routes, and Areas) of the Behavioral Model of Environment BME were used to conceptualize and organize the audit items. Several additional assessments were also carried out during the first step, including comparisons in accuracy between drive-through vs. walking audit, field vs. digital method, and checklist vs. map audit format.
The reliability tests were conducted by two trained auditors from different backgrounds. A total of 12 schools were selected from the larger central Texas region. Two schools were randomly sampled from each stratum determined by median income and community settings: high vs. low income; and urban (Houston and Austin), suburban (Bryan and College Station) and rural (4 micropolitan and non-core counties) settings. For the training protocol development, additional auditors (7 auditors total) from diverse background and gender/ethnic groups were involved.
The reliability tests included (a) inter-rater, (b) test-retest, and (c) peak hour vs. off-peak hour reliability assessments. Kappa statistics were used for categorical audit items and ICC (Intra-class correlation coefficient) used for continuous and likert-type items. ICC and Kappa values usually range between 0 and 1, with 1 considered a perfect agreement.
Results:
Walking audit, field method, and the combination of both the checklist and map audit formats yielded most accurate results. Final audit items incorporated items from the existing street audits and new/re-designed items such as the design of drop-off/pick-up areas, sidewalk and crosswalk connectivity to the school buildings, shade conditions, and barriers and safety hazards. Also importantly, pedestrians and children have more intimate relationships with their surrounding environments, as compared to drivers and adults. Therefore, to achieve the level of details and precisions needed for these target groups, important route-related variables such as exact locations of crosswalks, sidewalks, bike lanes, buffers, drainage ditches, barriers within the sidewalk, etc., were incorporated into a series of map audits. Each map audit page was prepared with a background aerial photo image and mapping instructions, which allow auditors to draw and locate each map audit item directly onto the aerial photo, thus providing more precise spatial references.
For the inter-rater reliability test, the average of overall Kappa was 0.839 (considered outstanding) and ICC was 0.602 (moderate). For the test-retest reliability, average Kappa was 0.903 (outstanding) and ICC was 0.774 (good). The agreement between the peak (during the drop-off and pick-up yours) and off-peak measurements was 0.801 (good). Rural schools showed most consistent results in the peak-off peak and test-retest assessments. Inter-rater rests showed somewhat different results with urban schools showing the highest ICC and rural schools showing the highest Kappa values.
Conclusions:
This School Audit Instrument is a tool that can provide effective and efficient assessments of street and school site environments, focusing on those attributes related to children’s active transportation to school. The instrument’s three components of Street Audit, School Site Audit and Map Audit cover multi-faceted environmental factors comprehensively and with sufficient spatial and observational details. Most items achieved moderate to high levels of reliabilities from the testing that involved schools with different income levels and community settings. Therefore with proper training, this audit can be used to provide reliable information about the existing built environmental conditions around schools, for research, intervention, and policy-support purposes.
Support:
This work was partially supported by Robert Wood Johnson Foundation grants.
