Presentation at the 2005 Active Living Research Annual Conference
There is mounting evidence that physical environmental attributes of neighborhoods, such as access to community trails and paths, are positively related to participation in physical activity. However, little is known about how specific physical characteristics of trails/paths may influence use. Furthermore, there is a distinct
need for reliable instruments to measure trail/path characteristics.
The objective of this study was to develop and test the inter-observer reliability of the Path Environment Audit Tool (PEAT), a computer-based instrument for measuring physical characteristics of community trails/paths.
Setting. This study was conducted in Massachusetts and included two rail-trails, one linear park, one conservation area with a path loop, and two parks with path circuits located in urban, suburban and rural settings. Study sites were required to minimally support walking and secondarily, other linear activities (e.g., cycling).
Instrument development. Three approaches were used to identify potentially relevant characteristics of trails/paths to include in PEAT: a review of the parks and recreation and planning literatures; intercept surveys with users at two sites; and input from a transdisciplinary team that spanned parks and recreation, landscape architecture, geography and public health. Items were developed in three content areas: design features (e.g., slope; 11 items with sub-items), amenities (e.g., presence of lighting; 16 items with sub-items), and maintenance/aesthetics (e.g., graffiti; 7 items). In addition, PEAT included a 6-item module for roads that intersected trails (e.g., presence of crosswalks, overall safety rating). PEAT items were scaled either as dichotomous (e.g., presence/absence of amenities) or on a Likert or ordinal scale.
Data collection. Prior to conducting observations with PEAT, geographic information systems (GIS) data were collected on trails with a high-level global positioning system (GPS) unit. GIS trail data were used to create a functional unit of observation for PEAT; trail segments, approximately 400 meters in length. These data were also used to produce detailed maps of PEAT trail segments that guided observations. Two observers unfamiliar with the study received two days of office and field-based training using a manual with PEAT item definitions. All observations were conducted during June 2004. Both observers used a tablet PC to conduct observations, either during a different part of the day or on separate days. A total of 229 PEAT trail segments (including 44 intersecting road segments) were audited by each observer.
Kappa coefficients (k) were generated for dichotomous items. For ordinal and Likert-scaled items, intraclass correlation coefficients (ICCs) were estimated. Observed agreement (%) was also calculated.
Preliminary findings for primary items are reported. For 13 dichotomous trail amenity items k ranged from 0.49 (presence of pay phone on trail segment) to 0.92 (presence of services, such as a snack bar). Using Landis and Koch's adjectival ratings, the coefficient for eight items was either "substantial" or "almost perfect"
(> 0.60). Reliability varied for presence of mass transit stations or stops (k = 0.70), commercial destinations adjacent to a trail segment (k = 0.49), and cultural/civic destinations near a segment (k = 0.38). Overall, k-values for trail design features (yes/no items) were not as high as the values obtained for amenities. For six items k ranged from 0.23 ("fair") for presence of a shoulder to 0.69 for presence of a gate/bollard. ICCs for trail surface condition, slope, site distance, and enclosure/vegetative cover by a trail segment were 0.50, 0.65, 0.55 and 0.31, respectively. For seven maintenance/aesthetics items, which were rated on a 4-point scale from "none" to "a lot," observed agreement was the following: glass = 87%; litter = 49%; graffiti = 51%; vandalism = 85%; odor = 74%; noise = 59%; animal droppings = 89%. For the five dichotomous items used to rate attributes of intersecting road segments, four had k > 0.85 (presence of curb cuts, crosswalks, raised crosswalks, and pedestrian crossing signals), while an item for presence of signs/signals for vehicular traffic had only "fair" agreement (0.40). Reliability for a Likert rating of intersection safety was moderate (ICC = 0.50).
The majority of primary items in the PEAT tool have a moderate to high level of inter-observer reliability. Generally, reliability was higher for trail amenity items and lower for more subjective items related to maintenance/aesthetics (e.g., glass). Although further refinements of PEAT may be needed, our results suggest that PEAT shows promise as a reliable tool for researchers and practitioners. In further reliability assessments, we will use the ecometric approach of Raudenbush and Sampson to determine the extent to which a smaller number of trail segments can be audited to assess a particular trail characteristic with adequate levels of precision.