Statistical Analysis

In earlier tutorials, we talked about quantitative and qualitative research studies, and those discussions were pretty well focused on what to do after you’ve decided to take on a qualitative or quantitative approach. What if you haven’t decided yet? How do you decide?

This is a really common reason that we are called upon to provide consulting for dissertation research, so to help clarify factors involved in this decision, I’ll explain in this video how you might use your research gap to pick an approach. Then we’ll spend a lot more time talking about situations where you decide that maybe doing both would be ideal — that’s called a mixed methods study, and as you might think, they’re wonderfully comprehensive.

Let’s consider an example of a research gap that we might approach from either a quantitative or qualitative analysis angle. For example, let’s say that the research literature is unclear on how moral reasoning relates to bullying in the workplace.

Because there are lessons to be learned from examining the statistical relationships between moral reasoning levels and perpetration of bullying in the workplace, and also from exploring workers’ perceptions of how different levels of moral reasoning seem to tie in with bullying behaviors they have experienced at work, we could easily take this study in a quantitative or qualitative direction.

This is the type of research gap that might leave you wondering, should I do qualitative, quantitative…or maybe mixed methods?

There are advantages to each type of approach, which can make it difficult to decide. It will definitely help to plan your dissertation approach mindfully at this point by thinking through this question more deeply, so let’s first talk about each quantitative and qualitative research studies.

Each has very specific advantages and disadvantages, and each “approaches” research differently (hence the term “approaches”!).

The first way in which they’re different is where they place focus. As discussed in the qualitative methods video, for qualitative research studies, they’re largely concerned with phenomena, such as the implementation of an anti-bullying program or therapy for first responders experiencing secondary trauma.

These are things that people might participate in or experience, and their perspectives and interpretations of these experiences. Of course, it’s possible for these to be part of quantitative studies, but they look different.

Specifically, these become quantitative when they themselves or something related to them are measured numerically or in objective terms — such as the period before implementation of the anti-bullying program and the period after implementation (where you have two possibilities — before and after) — and then subjected to statistical analysis. For secondary trauma, it might be burnout among therapists doing this work, or perhaps it’s PTSD symptomology in the first responders receiving a therapeutic intervention. All of these can be measured numerically and, in some cases, with survey instruments that have already been developed. In this way, they’re variables. In providing dissertation help to hundreds of clients, we’ve seen a number of instruments! You might be surprised about the constructs and contexts addressed by these research tools.

Because qualitative and quantitative studies are focused differently, they also proceed differently — they engage in analysis in opposite directions, actually. Let’s stick to our quantitative examples for just a moment — because they start with a sense of those variables (and clearly defined — what’s called “operationalized”), they’re usually deductive in nature. That is, they start with the conclusions — things they they hypothesize are true — and then move to determine if those hypotheses can be confirmed with statistical analysis.

So in a dissertation, they might test whether the implementation of an anti-bullying program helped to reduce referral rates or reduced levels of reported victimization among students. They could test whether a specific therapeutic intervention (perhaps as compared with another) reduced PTSD symptomology for first responders.

Qualitative research, on the other hand, has no idea what’s there, at least it has no idea before it starts. In contrast to quantitative studies, qualitative ones are looking simply for what is there. They don’t know anything about the implementation of the anti-bullying program, but they may want to know how it went for teachers — how fully they felt it was implemented, what challenges accompanied that implementation, and how students responded. In this way, qualitative research moves inductively, starting with the phenomenon itself as expressed through data (interviews, focus groups, observations, and others) and moving to those general conclusions only as a result of the investigation, not in advance.

In terms of qualitative analysis, looking for what is there can be difficult, and that’s definitely the case when you consider the complexity of multi-step qualitative analysis protocols. We’re one of only a few, if not the only, companies that provide comprehensive assistance with all of qualitative methodology and analysis.

It’s hard to find qualitative data to analyze that is just “there” in the same way that you can for a quantitative study in certain fields. Primary data collection is a concern, then, and you’ll see above that qualitative studies often engage much more generally with people’s perceptions and experiences. It can do this in a couple of different ways, but the end result in either case is that it uses much more specific data collection methods than do quantitative studies. Interviews, questionnaires, focus groups, observations, and document analysis are especially common forms, as they all help to better understand people’s thoughts and experiences.

Because quantitative studies move deductively and with pre-defined constructs or variables, they often require instruments that validly and reliably measure those variables. This provides its own challenges, and talking with a statistician or consultant to walk through these statistical concerns can be helpful.

That’s because, even though it sounds as though decisions make themselves here, it’s often that your committee members have very specific ideas of their own. Because of our consulting experience with so many universities, especially the major online ones, we can help you work within those expectations that are part of the dissertation process at each school, providing assistance to point the way forward through your quantitative research.

Okay, so now for mixed methods. What if you alight on a focus for your study, and it seems like neither option is really complete as it is? Perhaps you’d like to know what people think about a particular issue, but you’d like to fully understand the nature of that issue first. Or maybe you’d like to figure out some of the problems or benefits that are part of a phenomenon and then test those specific elements more rigorously.

This is where a mixed methods study can be ideal, and it’s often an appropriate choice for an investigation in that it provides a more robust set of conclusions than would be possible with just one design on its own.

That’s because it can focus on both variables and phenomena, move both deductively and inductively, and then with both interviews (for instance) and survey instruments.

Now, a mixed methods study won’t always be the answer — and they can be a lot of work! — but in situations where there are benefits to both qualitative and quantitative studies, mixed methods can help you take advantage of all of those benefits in your research. In these cases, the qualitative and quantitative parts of your dissertation help together to answer your research questions.

If you’re thinking about setting up a mixed methods study, but you’re not really sure how to go about doing this or even if it’s a good idea for your dissertation, we can help you make this decision wisely by thinking through all of the pros and cons of using this approach. We can assist you in a variety of ways to plan out this very complex form of research, and we also know how challenging it can be to gain approval for mixed methods studies with your committee.

These types of studies are so complicated that you can expect to put in loads of effort to achieve approval, and that’s where I think you’ll see how valuable and helpful our unlimited revisions are! You can count on us to stick with you as you work toward approval of your proposal based on a mixed methods design, and we can also help with your dissertation when it comes for analysis — but, more on that later.

Within mixed methods, there are two central designs that determine how the qualitative and quantitative “arms” of the study fit together.

Parallel convergent, and explanatory sequential.

Parallel convergent studies are those where you collect your qualitative and quantitative data independently and at the same time in your study. Consider the example of the therapeutic intervention for the treatment of PTSD symptoms in first responders. Perhaps you want to examine the treatment outcomes of a particular therapeutic modality using a PTSD symptom survey and also through in-depth interviews with first responders experiencing these symptoms.

In a parallel convergent mixed methods study, you would obtain permission for use of the survey instrument and also develop an interview guide about treatment experiences at the outset of your study. Then, your participants would complete the surveys before and after treatment, and participate in interviews following treatment. You would then analyze the quantitative and qualitative data separately, but then compare the results of these analyses as a final stage of analysis to examine how the different sets of findings complement, explain, or contradict what was found in the other form of data.

The explanatory sequential mixed methods approach, however, sequences your quantitative and qualitative data collection in such a way that allows one phase of data collection to inform the next.

This can either place the qualitative or quantitative phase of the study first, depending upon your overall aims. Indeed, in our work as dissertation consultants, we’ve seen quite similar studies conducted each way. Let’s consider the above example again. If you were interested in developing a new PTSD instrument that was specifically designed for first responders, you would like begin with a qualitative analysis phase in which your interviews with first responders sensitized you to issues that might be used to develop survey items for PTSD symptoms that are more relevant to this group of individuals. This qualitative data collection would then feed into the quantitative phase, in which you conduct statistical analysis to validate the new survey instrument.

On the other hand, you might approach such a study from a quantitative stance first, by evaluating first responders’ responsiveness to a specific therapeutic treatment modality. Then, if you were using an explanatory sequential approach, you might identify those rare participants who were nonresponsive to the therapy, and then develop an interview protocol to further exploration into the possible reasons for their poorer outcomes.

As you can see, the outcomes of the first phase in a sequential approach directly influence data collection in the next phase, which sets these studies apart from those with convergent designs.

Now, let’s back up a little and talk about the first research gap I discussed in this video — check out our video on topic development for more on finding a research gap — and see how we might move from here. It would be really nice to explore managers’ and employees’ experiences with bullying in the workplace to discover how they interpret these as relating to moral reasoning, and it would also be really nice to conduct a correlational study for your dissertation to help investigate how different scores on a moral reasoning instrument relate to perpetuation of bullying. So in this way, a mixed methods approach might actually be perfect!

As you can see, the interviews with people about their thoughts and interpretations might be helpful in explaining or illuminating any statistical relationships we see between moral reasoning and bullying.

We’ll just need to select a design based on how we plan to get the most from the two “arms” of our study. In this case, we would need to decide if collecting both forms of data would be doable in one shot, or if it would be important to have participants complete one form such as the surveys first, and then decide on a direction for your qualitative interviews. Again, it’s often that, for your dissertation, both are possible. It helps to always think here with your end goals in mind.

One thing I should mention here is that, while you have ideas about what would be ideal, chairs and methodologists on your committee often do, too. With our huge amount of experience with the major online universities, we can definitely help to navigate their tough standards for setting up a mixed methods dissertation or thesis.

And, I’ll also note that with dissertation research, you might consider the potential IRB challenges of sequential explanatory design, as these involve proposing two rounds of data collection, with the subsequent round being dependent on the outcomes of the first one — see our IRB video for more information on this!

If this is your design of choice, however, we can certainly help you to put this together and will stick with you to ensure it is approved all the way through final dissertation editing. Because of this commitment, revisions are the name of the game. When we help with our projects, we include revisions at no additional charge, straight to approval!

For many other studies, though, a parallel convergent mixed methods design would be better, and that might be the case for your study.

We’d love to talk about your dissertation with you, and please know that you can call or email anytime for help here. We can help you sort through the many decisions — first if a mixed methods approach might be best, and then what design would be ideal for your study.

As I’m sure you can see at this point, conducting a mixed methods study can be very rewarding — and, it can provide you with a wealth of data that will support multiple research article submissions once you graduate. But, it is truly like conducting two studies in one, and it can be a time consuming and extremely challenging process.

With our solid grounding in all things quantitative, as well as our specialization in qualitative research, we can most certainly assist in development of a mixed methods study that will be sure to impress, and that you can really be proud to call your own!

Thanks so much!