Surveys and the Human Mind: Review on CXL’s Growth Marketing Course
Here’s what I’ve learned so far from CXL’s Growth Marketing Course to conduct better qualitative market research.
I’ve been doing surveys for years, but there are specific tactics, strategies, and pitfalls I should probably consider for the next time.
Surveys allow us to understand attitudes towards our company, stay aware of our competitors’ actions, evaluate new products, and understand user behavior.
What I learned is that we need to have a strong why behind all of our survey designs. If we lose our objective (the why), we end up designing surveys that gather irrelevant data.
Start with why
Let me bring a part of Simon Sinek’s description for his book, Start With Why:
People like Martin Luther King Jr., Steve Jobs, and the Wright Brothers had little in common, but they all started with WHY. They realized that people won’t truly buy into a product, service, movement, or idea until they understand the WHY behind it.
I think the “Why” is the most important part of any research. If you don’t doubt the reasons or driving and influential factors on a phenomenon, you can’t start the research.
There’s an analysis technique called “Five whys”, as Wikipedia mentions:
Five whys (or 5 whys) is an iterative interrogative technique used to explore the cause-and-effect relationships underlying a particular problem. The primary goal of the technique is to determine the root cause of a defect or problem by repeating the question “Why?”. Each answer forms the basis of the next question. The “five” in the name derives from an anecdotal observation on the number of iterations needed to resolve the problem.
Next, when moving to design the surveys, we need to:
- Use questions that are closed-ended
These are defined as below by Wikipedia:
A closed-ended question refers to any question for which a researcher provides research participants with options from which to choose a response. Closed-ended questions are sometimes phrased as a statement that requires a response.
A closed-ended question contrasts with an open-ended question because it can not easily be answered with specific data.
2. Do not ask leading questions
Our questions should not encourage the answer we want. We should not inject any positive elements to lead the answers into what we want. We want to know the truth, and any false confirmation will corrupt our survey.
3. Do Not Mix behavior and attitude questions
Attitude is different from behavior. Attitude is something internal, and it forms our perception of the world. It’s based on experience, reflects what we think or feel, and is defined by the way we perceive things. It’s a settled way of thinking or feeling about something. It’s a mental and emotional entity that heresies in or characterizes a person. You need to interact with a person so you can understand their attitude.
Behavior is the outcome of our internal processes. It implies the actions towards others. It’s based on the situation, and it’s an outward expression of attitude. It reflects what you do and is defined by social norms. It’s observable, which means we can use it as evidence without any direct interactions with the person doing it, we only need to observe carefully.
We need to carefully understand these two and don’t mix them in our surveys.
4. Do not conduct long surveys
We as humans have very limited energy to spend. Taking part in a survey requires energy and focus, therefore, we should be careful not to make our respondents fatigued. A very good survey may only take 5 minutes, and that’s an average time. If you see you’re getting very similar answers from a single user, it’s a sign of fatigue.
He may want to end his suffering!
So, in order to eliminate the risk, we need to make our surveys as short as possible. But, watch out for not removing your essential questions. You need answers, right?
5. Don’t let the users learn what you want to get
If you design your surveys in a way that makes users aware of your intentions, it may lead them to answer in a way that makes them useless. You must not lead them to choose any answers. They must remain neutral.
Survey Design and Cognitive Biases
Order Bias or Order Effects Bias
This occurs when respondents favor choosing objects because of their position in a list or a sequence. When it happens, it affects your survey quality, because an irrelevant factor is influencing the way your target audience is behaving.
How can we combat order bias?
- You can overcome it by pre-testing your surveys with a small group of friends or colleagues. You can see patterns, and if they’re choosing in an unnatural way.
- You can eliminate order bias by showing randomized items to each respondent. By randomizing the choices to each respondent, you ensure you have unbiased responses.
- Another way is to flip or inverse the answer choices.
Reading the Room
If you are an agency and you’re conducting the survey for a client company, you may tell them what they want to hear. This is sweet for them, and can potentially strengthen your relationship for a short amount of time before they get broke.
Always write your questions in simple terms that anyone can understand. Avoid using technical jargon and complex sentences.
I think we need to be aware of all forms of cognitive bias in order to get clear and unbiased answers from our audiences. People tend to behave in predictable but irrational patterns, which are discussed thoroughly in fields such as behavioral economics. Amos Tversky and Daniel Kahneman have done lots of research on different kinds of cognitive bias, which can help us identify and eliminate misleading online behavior in our qualitative research.
Stay tuned as I’m still studying the growth marketing course in CXL, and I’ll have more updates in the coming weeks!
In the meantime, you can check out my previous articles, and you can check out CXL for more interesting insights relevant to what I just covered.
