August 08, 2005

paul gronke on 08 Aug 2005

b!x,

A few reactions. The survey needs to vary the format of the questions. First, the questions are at risk of positivity bias (also sometimes called "yea-saying"). Most of the questions have the same response categories and are always in the same direction (e.g. if you like the city or like a service, you always say "satisfied" or "very good").

Typically, you try to vary questions so a respondent has to express disagreement in order to provide a "positive" evaluation (for instance, "I think city streets are too dirty" agree/disagree).

The consequence is that this survey is likely to make Portland citizens seem more satisfied with city services than they might actually be.

Second, I'd recommend varying the type of question as well, don't make all of these "likert" (agree/disagree) format.

Next, I don't understand the reliability report in the Appendix. It reports a 38% response rate (5682/15000) but then the standard error is based on N=3442.

I think it's very important to include the percentage of don't knows, even if you don't use them in calculating the marginals. Some of these items are pretty obscure, and you have to know how many people are simply checking "I don't know" before you base any policy decisions on the results.

Third, I'd add some add'l demographics. Most importantly, I don't see anything asking how long someone has lived in the City. Seems quite important if we're asking a battery of evaluations of city services.

Lastly, I also see this as an opportunity to learn about other things, but not ONI or other relatively obscure city services. Instead, why don't we ask overall evaluative questions about the effectiveness of Portland government (e.g. Thinking about the individuals who hold these positions, would you say you have a lot of confidence, a little confidence ... in Portland City Council / Mult County Govt); how people feel about the schools; whether they ahve considered moving out of Portland; how positive they feel about the future.

I know these are potentially volatile topics, but they are central to many of the debates we are having about Charter reform, about the visioning process, about why we need things like campaign reform. So why not collect the info when we have a chance?

Anne Dufay on 08 Aug 2005

I appreciate Paul G's substansive review and follow-up questions on this audit.

However, I must confess, I read the thing for its humor.

I mean, I've read, seriously, many many previous reports on these same issues, in these same neighborhoods. (Anyone remember the SW Plan? -- that'll tell you something about how Multnomah residents feel about the "smoothness" of their streets -- not that any real resident of rutted, potholed, sidewalk and storm-sewer "deficient" SW or outer SE would actually use such a precious term to describe their aggravation...) Plus, I've been to them... Sssso, I actually already know about it...

But, oh, the "smoooottthhhhness" of the streets... where are you Jon Stewart? Portland's got some material for you now...

Anne Dufay on 10 Aug 2005

I should add, my comments are my personal opinion, not in any way a reflection of how the NA community feels about this audit.

Many probably are thrilled to finally have some recognition of what they've been saying for so long -- that individual neighborhoods are different.

Me, I just consider incuring the expenses of an audit, to answer the "question" of whether there are differences in folk's concerns from neighborhood to neighborhood, a collosal waste of time and money. I think we can safely put that "question" in the same basket as the "question" of whether evolution is science. Or, closer to home, whether public school teachers get sick more often than city clerks...

I would support a city-wide audit, not to answer the "question" of whether neighborhoods are different, but rather to identify the varying issues neighborhoods face, and then assign responsibilities for fixing them. I'd really become a fan if there was a timeline for responding -- like "we commit to fix 3 out of the top 10 problems facing each neighborhood within the next 4 years." (I know, I know, dream on)

paul gronke on 10 Aug 2005

Anne,

I have a sneaky agenda here, but not so sneaky since I've posted about it elsewhere. I think Portland desperately needs a district based election system for City Council. And relatedly, I think the Eastside/outer SE is badly underrepresented under the current citywide elections system.

So my hopes are that a city wide survey like this will reveal precisely the kinds of divides that I think are apparent.

And yes, I also giggled a bit at the "smoothness" question. Hmmm.. man, Powell avenue is SMOOTH! Yesiree! SMOOTH! I love those smooth streets.

I think probably just asking about road maintainance is good enough.

Gary Blackmer on 10 Aug 2005

We do plan on adding questions related to ONI to our survey but if you look at our annual Service Efforts and Accomplishments report, each bureau is a major effort, including a review of goals, revenues, staffing, workload, and operational performance measures. This goes well beyond developing questions.

This year we chose to develop a chapter on PDC and we intend to develop the ONI chapter next year, after the Mayor has had an opportunity to shape its mission and objectives. Ramping up to survey at the neighborhood association level was a major undertaking as well.

I think the survey results will let bureaus see where the most dissatisfaction is among the neighborhoods and work with that community to identify the specific problem and solutions. I believe it will encourage a dialogue within the neighborhoods as well. This can better inform the officers of a neighborhood association about the real priorities of their neighbors.

Professor Gronke's points are well-taken and I will convey them to my staff, however there are other authorities on polling who support our methods. We have conducted considerable research on methodology and a principle difference from adhoc polling is that we have been asking the same questions for 15 years so we are always looking at the relative change from year to year, which corrects for any skewed results.

It is a satisfaction survey related to specific City services so questions about politicians and their decisions would be more appropriately answered through political polling and elections. We have a limited amount of space and participant attention and we need to keep the survey as lean and focussed as possible.

By the way, smoothness is a much more specific and meaningful term to drivers than "road maintenance."

Frank Dufay on 11 Aug 2005

By the way, smoothness is a much more specific and meaningful term to drivers than "road maintenance."

SE Powell's repavement is under way. In the end it will be "smoother" but the problem is no one asked the neighborhoods through which is passes how to make it better.

Our neighborhood association, Hosford-Abernethy, asked the Oregon Department of Transportation (Powell is a state highway) to put off this repaving until we and the other impacted neighborhoods had some input into what kind of improvements we wanted to see. The street isn't just an arterial for cars...it impacts bicyclists, pedestrians, local business accessibility, and storm water management. It impacts our local streets and transit service.

ODOT said they'd do a study, and get our input, after they finished spending millions repaving it. In this case, "smoother" is not a good way of saying "better." In fact, sometimes we even build bumps in our streets to make them better.

Frank Dufay on 11 Aug 2005

we intend to develop the ONI chapter next year, after the Mayor has had an opportunity to shape its mission and objectives.

Hmmm....sorta like Powell Boulevard. We'll ask folks about ONI after we've shaped its mission and objective.

Mark Sieber on 11 Aug 2005

I do find it encouraging that the Auditor is looking at neighborhood differences and priorities, since Portland's small, at-large City Council is structurally incapable of responding to needs across the entire city in anywhere near real-time. (Not to say I favor the ills of a ward system: Portland has an opportunity to really be inventive during charter review...)

That said, the resources spent are duplicative: between the late 1980s and the present, the Planning Bureau facilitated or evaluated approximately 50 neighborhood and community plans, spanning much of the city (see the Bureau of Planning website documents list at http://www.portlandonline.com/planning/index.cfm?c=34248). There are also a number of self-generated neighborhood plans yet to go through the city's adoption process, due to budget constraints.

While some are incomplete, and others marred by politics, a reasonable (if not always successful) effort was made to include community comment--and in later years, active particpation--in the creation of these plans. "Vision" portions of the documents alone outline all the issues noted in the Auditor's report, and more. These issues are still in the forefront of ongoing discussion and action in neighborhood associations, business associations and community groups.

I was involved as a volunteer in SW Portland community planning from 1994 to 2000, and chaired the Southwest Community Plan Task Force in 2000: a great deal of thought and effort was put into creating individual neighborhood plans in most of the sixteen SW neighborhoods, each with a vision, goals & objectives, and actions to remedy identified lacks and needs--and this is only one of the plans in the archive.

If pulled off the shelf and properly reviewed and collated, this wealth of community knowledge could form the basis from which to add new, updated information. This kind of audit would be a great service.

Updated in coordination with the existing community governance model (the current neighborhood 'system'), the Mayor's project to expand community governance, and his Visioning project, this would provide Portland's leaders and public servants the wherewithall to understand the city's sense of itself and its hopes for a better present and future--in all it's diverse and delicious variety.

Anne Dufay on 11 Aug 2005

Does anyone remember the survey/mapping work that Mark Helfand and his Multnomah NA team did a few years ago? As I recall they went pretty much to every residense in the NA boundaries, and got pretty close to 100% participation. As I recall the focus was on transportation and infrastructure issues and the survey quite well designed, and informative. (I say, as I recall, because memory is a tricksy beast :-))

It would be interesting to read it again, and see if I am remembering it correctly, and, (if the auditor's survey captured any of the same issues) if anything has changed.

Anne Dufay on 11 Aug 2005

"road maintenance" is indeed a better descriptor of the problem than "smoothness". Smoothness, in addition to its underwhelming power as a descriptor of the rutted, (often just dirt and gravel) roads of wide swaths of SE and SW, does not get at the real issue - which is not that of course we all know that roads in certain areas of town are "unsmooth"... but that the city will not be responsible to bring the roads in these specific areas of the city up to the standards of, oh say, SOWA...

Further, "smoothness" does not get at the equally important issues of adequate storm drainage systems attached to the roads (or, not...) and adequate sidewalks, also attached to the roads, or, not...

So I nominate "concerns about inadequate/non-existent roadway infrastructure, construction and maintenance" as the descriptor we are looking for.

I am sure, if you ask the right questions, you'll get very insightful and educated responses that might help the city better understand the real issues the citizens and their NA's are struggling with.

In my old neighborhood of SW (Hayhurst) folks could tell you a bookful about the problems associated with their inadequate roadway infrastructure and the city's regulations that leave them - the citizens, entirely on the hook. They took a "crash" course a few years ago.

One morning my neighbor, across the street and two houses down, got up to his buzzing alarm clock, went in to his kitchen, made coffee, poured himself a cup -- and the hillside came down on his house. His bedroom was filled with mud in just minutes. (Had he hit “snooze” and slept another 15 minutes, he would have been dead.)

Moments later he was standing in our street, in his bathrobe, holding a cup of coffee, as the rest of his house filled with mud.

The problems, as we learned over the coming months, were that there was no storm water system on the, un-maintained, road above our neighborhood, that the city would not do anything to repair the damage or stabilize the hill (off the hook because the road is sub-standard and therefore – not the city’s responsibility… (Though they did come in and survey to determine just which citizens property's abutted the rights-of-way on the street where the slide originated.)

Then the city kindly identified the culprits upon whom we were to affix responsibility -- a little old lady in a tiny little old house which was her only asset, and a middle-aged woman, with another old house under some big trees, and four dogs, her only assets. Funny story about her: she’d led the neighborhood citizens who had taken their objection to the variance needed to build that house all the way to city council 20 years before. The neighbors concern? The instability of the hillside the house was to be built into. The builder’s response? The neighbors were being NIMBY. The Council ruled? Go ahead and build it. Ironic, eh?)

It would have cost over 1 million to repair the damage to the house and make the hill safe. As you might guess, even “liquidating” the combined assets of the two woman above, would not have come close… Neighbors went to the city, got nowhere ("sorry, nothing we can do". We learned all the in's and out's -- our "ins", the city's "outs" related to the city's street maintenance rules.) Neighbors called on the neighborhood association for help – there was a huge meeting, the neighborhood association said they would try and talk to the city, were told, "sorry, nothing we can do".)

Anyway, too long a story for this venue, suffice it to say, there was no happy ending…

But a lot of people got very savvy about the issues. And, "smoothness" just doesn't get at it.

allehseya on 11 Aug 2005

"If pulled off the shelf and properly reviewed and collated, this wealth of community knowledge could form the basis from which to add new, updated information. This kind of audit would be a great service"

I'm very interested in seeing this "Southwest Community Plan Task Force in 2000" plan that you speak of. Most especially as it relates to outlining the process of how it created the networking / connecting of the 16 neighborhoods with (as you stated): "... a vision, goals & objectives, and actions to remedy identified lacks and needs--and this is only one of the plans in the archive."

Do I access it through ONI, you, or ? (you can feel free to email me at PDXArts@aol.com)

paul gronke on 11 Aug 2005

Gary,

It does help that your survey is conducted over many years--this allows you to track changes. But you still may have a level problem, if positivity bias is overestimating the satisfaction levels. As you are aware, comparing results year to year does nothing to test for positivity bias. The bias is easy to test for and fix if it seems evident.

I didn't mean to make too light of the smoothness question, I know formulating these questions is hard. Just trying to keep things lighthearted. Along with Anne, I found it a rather odd phrase.

And of course I knew you can't ask the kinds of questions I suggested. This doesn't stop me from wishing does it? I do think, though, that overall "mood" or "state of things" items are relevant to satisfaction with the City and its services. I'd still urge you to consider some broad attitudinal measures. I'd be happy to suggest some off line if you'd like.

Anne Dufay on 17 Aug 2005

Paul writes>>>I didn't mean to make too light of the smoothness question, I know formulating these questions is hard. Just trying to keep things lighthearted. Along with Anne, I found it a rather odd phrase.

This reminded me of a recent "Economist" article on why we find things funny, or not :-)
http://www.economist.com/displaystory.cfm?story_id=4246393

Describing the incongruity theory: (snip) "One description of how laughter is provoked is the incongruity theory developed by Victor Raskin of Purdue University and Salvatore Attardo of Youngstown State University, both in America. This theory says that all written jokes and many other humorous situations are based on an incongruity—something that is not quite right. In many jokes, the teller sets up the story with this incongruity present and the punch line then resolves it, in a way people do not expect. Alternatively, the very last words of the story may introduce the absurdity and leave the listeners with the task of reconciling it. For instance, many people find it funny that a conference on humour could take place in Germany."

(Personally, I found the term "smoothness" to be very incongruent with my experience of the roadways in question and that may indeed be why it made me laugh.)

or, from another theory, another excerpt: "Indeed, humour is one way of dealing with the fact that humans are “excrement-producing poets and imperfect lovers”, says Appletree Rodden of the University of Tuebingen. He sees religion and humour as different, and perhaps competing, ways for people to accept death and the general unsatisfactoriness of the world."

Interesting article. Seemed very appropos to this discussion :-)

B!X -- I don't think many residents know what ONI is. Even those engaged at the level of neighborhood association membership (even board service) often have no clue. So I think that would be a tough question, as I'm not sure that enough folks know enough to speak to it in more than a "guess the multiple choice answer" kind of way.

The One True b!X on 17 Aug 2005

I don't think many residents know what ONI is.

So their answer would be, "I don't know what ONI is." That might seem to be a useful thing to know.

And, actually, the point isn't to just have a couple of quesitons about ONI -- what I was after by asking whether or not ONI would be included in surveys was questions about the neighborhood system, ranging from ONI itself, to a resident's respecitve NA. Who knows these entities exist, who knows what they do, who has an opinion on what they do?

Anne Dufay on 18 Aug 2005

B!X writes>>>what I was after by asking whether or not ONI would be included in surveys was questions about the neighborhood system, ranging from ONI itself, to a resident's respecitve NA. Who knows these entities exist, who knows what they do, who has an opinion on what they do?

Ah. That I think would be useful information. I'd read that study with interest. :-)

Did you get your city survey yesterday? on 18 Aug 2005

My household was one of approximately 500K selected to participate in this year's 2005 Citizen Survey - and I'll have my completed survey in the mail later this afternoon. I can anticipate what kind of responses local PDX bloggers might...