Sharing stuff I've learned, and things I've thought about...

Ever the optimist am I…

I’ve been somewhat bemused by the internal fight going on in the ranks of the Republican Party. I don’t know which side to root for.

The outlying Tea Partiers are working hard toward driving the last nail into the coffin of the GOP. This would be a great outcome to my way of thinking. After all, the party has had a serious case of the stupids for over 30 years. Time was, the only saving value of the GOP was to serve as a balance to the seriously stupids on the Democratic side of the aisle, but the Democrats have, since the Clinton administration, become much more centrist and have done a good job of marginalizing the wackos, so who needs the Republicans? If the GOP went the way of the dodo, the Democrats would have a chance to split into a pair of parties with different approaches to addressing a commonly recognized set of national issues. On the other hand, it would be nice to see some sanity come to the Republicans, and make it possible for me to either vote Republican occasionally, or at least give some credence to their arguments on how to address an issue.

I’m good with it either way, but a recent article in the LA Times gives hope that the GOP just might succeed in re-inventing itself and will survive to serve another day, and we might actually improve our nation rather than just squabble.



If your boss asks you to gin up a survey, or if you or your organization is thinking of doing a survey, this post just might help – anyway, I  hope it does…


I can’t remember the last time I was presented with a well-designed survey. I’m pretty sure that most surveys are designed by someone who has no experience with surveys, and have probably had the task delegated to them by someone that has provided some pretty vague direction.

Designing a good survey isn’t difficult from a technical standpoint, but it can be a bit labor-intensive. There is a lot of thinking that needs to be done before you log in to Survey Monkey and start entering the survey questions. The principles of good survey design are pretty basic, and reflect ideas that are usually conveyed in any formal management or supervisory education, and doubly so for people who have any formal training in behavioral science research.

The advent of “free” online tools such as Survey Monkey have greatly increased the incidence of online surveys, and greatly reduced their quality. Many businesses and organizations do surveys with little or no preparation or thought because it is so easy to do so, and can give managers the illusion that they are being proactive and innovative when in fact they are collecting random data points that is being disguised as information.

The following is a good outline for the process of designing an effective survey.

Define the survey goals

What do you hope to learn from the survey? I’m talking about the “Big Questions” here, not the specific questions that will appear on the survey. What do you hope to know about the surveyed population after the survey is completed and analyzed that you don’t know now?

This can often be initially defined in one or more brainstorming sessions, but it’s important to make it clear that this is literally a “fishing” expedition, and that you are not yet at the point of designing the survey. The reason for this important caveat is that once you have collected ideas about what kinds of things people in the room want to know, you must take a big step back and look at each item of interest and determine which of them might result in actionable information. It’s really important to separate the wheat from the chaff here – to distinguish information about the surveyed population that simply satisfies curiosity from information that can influence your organization’s future actions. This is not to say that things learned that are not actionable are not of value – if we’re surveying our customers or clients, it’s often important to arrive at a deeper understanding of who they are and how they think. However, a shorter more focused survey is always better than a longer and more meandering survey, and it may be better to conduct multiple surveys rather than try to cram everything into one; longer surveys have a much higher abandonment rate which will reduce the size of the surveyed population and skew the results.

Another dimension to examine is the time frame for the actionability of the question. Some learnings may be actionable in the short term (like attitudes regarding customer service practices) while others may have a much more distant horizon (new product development). This dimension too should suggest splitting the surveys so that each serves a specific and coherent set of goals.

Formulate the questions

Many of the worst surveys I’ve seen have obviously been designed around one of the five basic types of response format:

Yes/No or True/False
Unordered “multiple choice”
Ordered multiple choice or “ranking”
Continuous scale (sometimes mistakenly referred to as a “Likert” scale)
Open-ended question

This is driven in part by a tendency to look ahead to the way the results of the survey are to be presented. While an important part of the survey results analysis, this should not dictate the design of the survey.

Instead, the goals of the survey (now that they are clear and coherent) should drive formulation of a series of questions that will elicit responses that will hopefully achieve the goal of the survey. This too can be done in a brainstorming session, as you can never have too many candidate questions. The session should then compare, contrast, evaluate and rank the questions that meet a high standard of clarity and specificity. Particular attention should be paid to questions that may be duplicates – are they asking the same question but simply worded differently, or are there really multiple questions here if each is re-worded to be more specific?

Determine appropriate response formats

Only when the list of questions has been winnowed down to what is believed to be those that will be most effective are you ready to make decisions about the question formats.

Each question should be evaluated as to which response format will accurately capture the responses. For example, care should be taken to ensure that a Yes/No format isn’t associated with a question that can elicit a response of “maybe” or “sometimes” or “it depends”. By far, the yes/no format (or it’s close cousins agree/disagree or true/false) is the one that I see most over-used. Yes, it is very easy to analyze the results, but despite the way some people like to think, we do not live in a black-and-white world. Shades of gray always predominate, so it’s critically important to verify that if you pose a yes/no question, that everyone responding will be able to quickly answer the question one way or the other. I know that I will quickly abandon a survey I’m taking when I encounter the 2nd or 3rd yes/no question that I feel is forcing me to make a response that is not accurate.

If the questions as stated can all be answered by a single format, then you can proceed with setting up the survey. However, it’s more likely that questions will be phrased in a way that suggests different formats for different questions. While it is preferable if the survey minimizes the number of different response formats, and we would like to have the entire survey use a single response format, there is no reason that this has to be the case.  Each question should be examined to determine if it can be re-worded to fit a different response format without sacrificing the accuracy of the response. If the majority of the questions fit one response format as originally stated, then only the remainder need be examined to see if they can be modified to accommodate one of the more frequent response formats.

While a survey where all the questions use the same response format is easier to summarize into a nice graph of some kind, forcing the questions into an inappropriate format is going to compromise the quality of your results. You can still do a good job of analysis with any kind of response format, even multiple response formats within a single survey. Analysis is not that more difficult, and while the presentation of the analysis may be more complex, it can still be just as effective.

And now, the rest of the story…

If you follow the procedure I’ve outlined above, you’re going to conduct a survey that is far ahead of 90% of the surveys currently being run. This is not the whole story, however, because you need to think carefully about the other details of running an effective survey. How are you going to collect it? Web-based surveys can be easy to create and distribute and collect, and can be very cost-effective. In some cases a web survey is the only practical collection method, but is it appropriate for the audience and the subject matter? If not, it could be so inappropriate as to negate the value of your carefully crafted survey questions. How are your respondents going to be selected? If the respondents volunteer, what kind of self-selection bias are you introducing? Can you ask for additional (e.g. demographic) respondent information to correct for a self-selection bias, or otherwise ensure that the respondents are representative of your target population? If your survey is being done in person, will any bias be introduced by the person administering the survey?

These are just some of the things that you need to think about before you conduct a survey, but doing so gets you closer to providing information that you can actually use to make decisions and plans for your organization.

Packaged foods

I’ve often repeated the common advice that to eat healthy and manage weight it’s a good idea to avoid packaged foods. This is a point that Michael Pollan makes in his very good little book on nutrition (Food Rules). Pollan voices this in the rules “Don’t eat anything your grandmother wouldn’t recognize as food” and “Don’t eat any packaged food with more than 5 ingredients on the label”. The real point here is to avoid all the chemicals and extracts that are added to make packaged foods more palatable and to extend their shelf life.

The downside to this would seem to be that we have to give up the convenience of packaged foods; not something to be minimized as busy as we all are!

However, if you shop carefully and read labels, you’ll find that we’re not completely on our own here.

I’ve found several items whose list of ingredients include nothing that you wouldn’t throw into the pot if you were making them yourself. They’re very good, and can form the basis of a simple quick meal without sacrificing good eating habits.

Pacific Roasted Red Pepper & Tomato Soup

Really, really good creamy tomato soup, with a nice red pepper flavor added. Very low in calories – 110 per 1 cup serving, so you can have a very large 2-cup serving garnished with some soda crackers or with a nice chunk of bread and easily stay below 400 calories for the meal

Pacific Natural Foods All Natural Vegetable Lentil and Roasted Red Pepper Soup

Another very hearty soup – good flavor, not at all bland. Easily spiced up with your favorite hot sauce like Tabasco (if you’re in the mood for Creole) or your favorite Mexican-style sauce or a dusting of ground chipotle pepper . It’s also good served over rice with either a warm tortilla or some corn chips. Again, very low in calories @ 150 per 1 cup serving, so you can easily use this as the primary element of a delicious meal.

Tasty Bite Madras Lentils

These are great when you’re in the mood for some Indian food – only 150 calories per serving, and they come in 2-serving pouches. I use the entire pouch for a single serving as I make this as part of the main dish by serving over a cup of steamed rice. Spice it up with some garam masala, curry powder, vindaloo seasoning or just some Sriracha sauce!

All three of these products are available in most neighborhood grocery stores, and are often available at Costco in 6-pack cartons – the tomato soup usually in 32-ounce cartons, the lentil soup in 16-oz cartons, and the Madras Lentils in 2-serving pouches.

I’m going to be trying more of the Pacific Natural Foods products, as they seem to do a good job of providing excellent flavor without sacrificing good nutrition!