Skip to content

LETTER: Survey accurate?

To the editor: There is some euphoria (and head scratching) surrounding the citizen satisfaction survey recently presented to council.


To the editor:
There is some euphoria (and head scratching) surrounding the citizen satisfaction survey recently presented to council.
The results represent, some say, a tour de force denying credibility to a “vocal minority removed from what is actually going on in this city.”
Wow! Let’s have a look. First, a word about opinion surveys.
Governments, for their part, provide services on the basis of the information they receive. Citizens, for their part, demand a greater say in what services are provided as well more power respecting how these decisions are made.
These twin objectives – good information and public participation can be addressed with opinion surveys. It is sound public policy, but not easy.
Mistakes, when made, are commonly identified in the data interpretation stage. We will concentrate there. We are told a silent majority roared in response to this question: “Please tell me how satisfied you are with the overall level and quality of services provided by the City of Thunder Bay? Would you say you are very satisfied, somewhat satisfied, not very satisfied or not at all satisfied?”
Nineteen per cent responded very satisfied, 70 per cent responded somewhat satisfied, seven per cent responded not very satisfied, while three per cent were not at all satisfied.
Here is the interpretation from City Hall: “The 2013 City of Thunder Bay Citizen Satisfaction Survey shows the vast majority of residents are satisfied with the services and quality of life Thunder Bay offers.”
In this context, vast satisfaction does not distinguish between “very satisfied” and “somewhat satisfied;” it just adds the totals – and that is problematic.
“Somewhat” according to the Oxford dictionary means “to some extent but not very.” Or, as noted by Floyd Fowler, noted authority on survey design and evaluation, the difficulty with “somewhat satisfied” is its negative connotation – almost certainly less positive than generally satisfied.
It is a value less than satisfied but greater than not satisfied at all.
In other words, it records a level of dissatisfaction, not satisfaction. Satisfied “to some extent” is the best you can make of that 70 per cent response – hardly an endorsement, ringing or otherwise, but it would be honest.
If we allow manipulation of the results (spin) to allow that “somewhat satisfied” be combined with “very satisfied” to accept city hall’s “vast majority,” then at the very least we must also allow for an alternate analysis, that “somewhat satisfied” be added to “not very satisfied” and “not satisfied at all” to conclude: “80 per cent of residents are less than satisfied with city services.”
It would be more accurate than city hall’s position because “somewhat satisfied” is an inferior value to “satisfied.”
This would indeed constitute a roar, but not one to brag about.
It may be asked: “Isn’t somewhat satisfied good enough?”
Consider this. It seems in many surveys with limited categories people tend toward the positive end of the scale – a bias surveys are supposed to mitigate and in many cases do by offering respondents a graduated scale of one to 10.
Perhaps people were being polite – not happy with services, but accepting that they are stuck with them or unwilling to be negative with a friendly stranger over the phone.
It doesn’t matter.
“Somewhat” doesn’t take you to satisfied, and may in any event be a generous evaluation reflecting respondents demonstrated tendencies to rate higher on limited category scales.
Imagine, less than satisfied being a generous rating.
The real tragedy here is that we have lost sight of what the whole exercise was about.
Taxpayers’ money was spent on a survey (a public policy tool) to provide useful information to council and empower citizens.
It is instead being misused as a political tool, a kind of propaganda that has led at least one media outlet to take administration’s word for it, and reflect on the survey as a thumping endorsement of city government.
The silent majority has spoken. Is anyone there to listen?

William Olesky,
Thunder Bay





push icon
Be the first to read breaking stories. Enable push notifications on your device. Disable anytime.
No thanks