How you ask matters: The wording of your research question affects your analysis

Writing a research question for your study can be a daunting task. The appropriate statistical language must be used for each question in order to determine the type of hypothesis test you can run. Therefore, how you ask your research question is the foundation for the rest of your analysis! The first step of the process is to determine what the goal of your research will be. Are you attempting to look at differences in means, or are you just looking to see if two variables are related to one another? Once you have determined what your ultimate research goal is, you can move on to writing the research questions in statistical language — AKA lining up the research question/hypothesis with the type of statistical test that you want to run.

There are three types of statistical research questions, each with its own associated analysis. These three types are descriptive, comparative, and relational. This blog will discuss all three and the appropriate statistical analysis for each.

First, there are descriptive research questions. These types of questions seek to simply describe a situation, and do not include any hypothesis testing. For these questions, means and standard deviations or frequencies and percentages would be calculated, depending on the level of measurement of the variables.

For example, imagine that you are a researcher interested in the leadership style of ice cream shop employers and the job satisfaction of their employees. Descriptive research questions for this topic could be:

• What is the leadership style of ice cream shop employers?
• What is the job satisfaction of ice cream shop employees?

Next is a comparative research question. These types of questions are used to compare variables or groups to assess differences between them. By using the word difference in your research question, you can create a comparative research question. Comparative analyses that seek to assess differences include the t-test and the analysis of variance (ANOVA) family of tests. A question of this type could be:

• Is there a significant difference in the job satisfaction of ice cream shop employees who have employers with different leadership styles?

Lastly, there are relational research questions. These types of questions seek to assess the relationship between two or more variables or groups. This type of question could be phrased in two different ways:

• Does the leadership style of ice cream shop employers predict job satisfaction of ice cream shop employees?
• Is there a correlation between leadership style of ice cream shop employers and job satisfaction of ice cream shop employees?

Notice that here, there are two different relational questions. One uses the word predict while the other uses the word correlation. There are several “buzzwords” in quantitative research that indicate very specific analyses, including predict, correlation, difference, relationship, positive, negative, and more. The use of the word predict will always indicate the use of a regression analysis. However, the use of the word correlate indicates the use of a simple correlation analysis (Note that this is different from describing your research design as correlational—which simply implies that you are assessing a relationship).

Alternatively, you can use the word relationship instead of correlate or predict. By using the word relationship instead, you have created a relational research question that is more general than using the words predict or correlate, leaving you with more options when it comes down to cementing your data analysis.

Finally, it is important to note the use of the terms positive and negative within a research question. These terms are known as augmenting words, or words that can be added to your research question if you want to be specific with the hypothesized direction of the relationship between variables. These should only be used when there is significant evidence in the literature to support your research question and associated hypotheses. This is because using these terms limits your ability to reject your null hypothesis. For example, if you asked:

• Is there a significant positive correlation between age and the job satisfaction of ice cream shop employees?
Your null hypothesis would be:
• There is no significant positive correlation between age and the job satisfaction of ice cream shop employees.

This would mean that if you ran your results and found that there is a significant negative correlation between the two variables, you found a significant result, but you still could not reject your null hypothesis. This is because you predicted a significant positive correlation but did not find a positive relationship. Therefore, words such as positive or negative should only be used when you want to make a specific directional hypothesis and there is substantial evidence in the literature to guide your hypothesis.

Overall, how you word your research question affects the inferences and conclusions you can form from the data—for example, if you only ask a descriptive question, you cannot assess if there are any differences between groups. Alternatively, if you ask a comparative question, you would not be able to determine if there are predictive relationships between your variables. Therefore, your entire dissertation should be carefully planned and orchestrated, even down to the specific wording of your research questions!