top of page
Search
  • Writer's pictureCarlos Mauricio Díaz Nissen

Quanti Vs. Quali: What is it that really matters?



Let's start this short essay with an important anecdote...


During 2009 and 2010 I was working in my Psychology thesis. I created a game for covert evaluation of cognitive skills. It was an evaluation battery of cognitive skills dressed as a game (and developed with the budget of a student) which allowed me to get data of how children solved problems and make decisions as a process and not as a result.


One of the mini-games of the battery was a labyrinth. The labyrinth had different variations, depending on which particular skills or cognitive bias I was evaluating in that specific task. There was one where I told the child 'go to the end of the labyrinth using the shortest path', nothing more. The shortest path although not obviously visible, was easy to discern. Children did mostly well, a few did bad, but that's not the point.


I was applying the battery to this little girl, when she came to this labyrinth. She went (almost) all the way to the end of the labyrinth using the long path, but two steps near the end she turned around, went to the start line, took the other path, and finished the task.



As you can imagine, I was very puzzled (to be honest, I thought she just wanted to play for a longer time) so I asked her: 'why did you do that?' (that question was not part of the protocol of evaluation for the battery. Remember I was testing the effectiveness of the battery, not how well the children did). To what she replied: 'Because this [pointing to the shortest road] was the shortest road'.


Then it struck me! This girl did something fantastic! She went through the task and almost solved it, then she realised she had done something wrong and corrected her mistake, even though it took her double the time to finish the task. In other words, she did better than any of the children that solved the labyrinth wrong (although not as well as those who did it right in the first trial).


When I looked at her data, she was an outlier. She had double the steps and double the time than any other child in that trial. So, in a normal situation her data would be put aside or we might thing the data as a bias, the person didn't understand the premise of the task, was fooling around, or had a mental problem. We could have also said that there were problems with the data collection at this point. None of these answers would have been true.



And this happening to me almost at the end of my studies, struck me as one of the most important learning experiences I've had as a researcher: The Quanti vs. Quali fight is one of the biggest foolishness we find in current Social Science and Humanities. Data without context is nothing, numbers don't tell the stories, we use the numbers to tell the stories. We cannot generalise from a qualitative story, but in the same way, we cannot say that every single answer from 1000 different people can be interpreted in the same way.


Let's retake the example from my thesis. If I were a researcher looking only at numbers, I would have done one of two things, either omit their data (drop the outliers), or inquire more (for example, applying an IQ test). The second option is actually very interesting, and there is a whole discipline —differential psychology— dedicated to study differences. However, when we have data from 1000 people, making interviews to maybe 50 outliers is not possible. So, as it turns out, this data is often just dropped.



Don't get me wrong, I am not advocating for going solely for the qualitative aspect of things. Every person is a world, and we can easily get lost in the world of qualia without a systematic map.


So back then, thanks to this little girl, it struck me that the Mixed-Methods was the best way to analyse data. We could access the generalities of the population via Quanti, but giving meaning to the data not only based on our scientific narrative of ascetic outsiders, but accounting for the particular Quali reasons behind the behaviour of the individual.


As I grew up and learned more about research, I grew more convinced that the fight between Quanti and Quali inside the Social Sciences and Humanities was very silly. Not only this, but some people actually take that fight to a totalitarian field; some scientists not acknowledging the difficulty and validity of —good— qualitative research on the one side, and some saying that the statistics erase the subjectivity on the other side. Even sometimes journals or article reviewers do not admit a specific type of analyses, with no well grounded reasons for this.


Even more, this fight is growing stronger. On the Quanti side, now that we started using more mathematical models for 'understanding' behaviour, and with the advent of big data it's almost like we can data-mine the subjectivity out of numbers. On the Quali side we have that people are more interested to know the why of certain behaviours, ignoring important population patterns in favour of a few anecdotal experiences.



The discussion is long, and this is by no means a state of the art on the discussion. This is a very informal essay on the subject, an antipasto for a discussion if you would call it like that.


However, in my personal experience, born from the interaction with one of my 'research subjects', a small girl playing a game I might tell you that currently my answer is not even Mixed-Methods.


Use the method that allows you to address your research question, regardless of your prejudices. If you want to know if there's a group difference for an intervention, use a T-test. If you want to know why it was effective in some people and not in others, go for semi-structured interviews. If you want to understand other possible influences in the intervention, interview the participants, make them fill some questionnaires, and apply some psychological tests.



Don't handcuff yourself to paradigms, use them to understand the world better, not to close your mind. Think about your question, think about the best way to address it, think about which type of data will facilitate answering that question in a meaningful way. Let your question, not your method, guide your research.

53 views0 comments
Post: Blog2_Post
bottom of page