Background: The study’s purpose is to describe the development and evaluate the reliability (inter-observer agreement) and validity (rater agreement with a gold standard) of a direct observation instrument to assess park characteristics that may be related to physical activity.

Methods: A direct observation instrument of 181 items was developed based on a conceptual model consisting of the following domains: features, condition, access, esthetics, and safety. Fifteen pairs of observers were trained and sent to two parks simultaneously to assess two Target Areas each.

Results: Overall domain reliability was 86.9%, and overall geographic area reliability was 87.5%. Overall domain validity was 78.7% and overall geographic area validity was 81.5%.

Conclusions: Inter-rater reliability and validity were generally good, although validity was slightly lower than reliability. Objective items showed the highest reliability and validity. Items that are time-sensitive may need to be measured on multiple occasions, while items asking for subjective responses may require more supervised practice.