We examine whether specific types of people are more sensitive to the built environment when making a decision to walk or engage in other physical activity. Over 700 participants from 36 environmentally diverse, but equivalent-sized neighborhoods or focus areas responded to a survey, kept a travel diary, and wore an accelerometer for seven days. Subgroups defined by demographic and socioeconomic variables, as well as self reported health and weight status demonstrate that most subgroups of people walk more for transportation in high density areas. However, only the less healthy walked more overall in high density areas after controlling for sociodemographic characteristics and physical activity was remarkably similar among the groups and across different kinds of environments.  While environmental interventions may not increase physical activity population wide, some populations – including some for whom interventions may be important such as the less healthy and the unemployed or retired – are more affected by these neighborhood environmental characteristics.