I am reviewing my latest article which I wrote together with my PhD thesis supervisors. You can find the article here: https://rdcu.be/cjMnI
My PhD in Learning Sciences was focused on Scientific Reasoning and Argumentation. Being the person I am, I choose to study Scientific Reasoning and Argumentation in Games. So at the start of my PhD I was reading a lot about Scientific Reasoning when something struck me. The concept was far from well defined!
Although we (PhD students) were working under a very specific framework proposed by Fischer et al. (2014), I noticed that there were big inconsistencies across the literature not only in the use of the term, which some times was interchanged with that of Scientific Thinking, but also the definition, characterisation (this is the characteristics that make the concept unique), and operationalisation (this is the way we measure so that we ensure that other scientists are measuring the same when conducting experiments) were highly irregular across disciplines and inside disciplines.
So a research question was born. Are these two concepts the same? And, if they are not, what are their characteristics? How can we operationalise them so that better research can be carried in the future?
So I started a quest to solve my questions, and for that I analysed 166 texts dating from 1900 to 2015. For this I designed a specific protocol to record my findings during my reading, aiming to categorise different definitions, characteristics, and operationalisation of the concepts.
During my research I found different things. For instance that thinking and reasoning are not the same thing, they are epistemologically different. While thinking is a broad and comprehensive cognitive skill, reasoning is focused and more specific to certain activities. This should be reflected in the concepts of scientific reasoning and scientific thinking.
As it turns out, this ended up being half-truth, not because it was half-lie, but because the epistemological consistency of the concepts were shifting all the time according to the discipline studying the concept, and sometimes even within the same discipline.
We found, for instance, that just within Psychology the terms Scientific Reasoning and Scientific Thinking were used as synonyms one-third of the time (one-third of the literature revised). We also found that some scientists were defining Scientific Reasoning as others defined Scientific Thinking and vice-versa.
This is important, because when there is not a standard definition for a concept, it is not easy to define the characteristics of the phenomenon studied. This also makes it difficult for different scientists to measure the same thing when trying to account for a phenomenon.
Regarding this last part, we also found that, although some scientists defined (for example) Scientific Reasoning in a certain way, they used a tool (standardised test) which used a different definition or characteristics to their proposed definition, so their results were difficult to interpret.
Moreover, we found that some scientists (even in experimental studies) did not operationalise or characterise the concept they were studying. Instead, they simply took measures and made inferences based on the affordances of a particular tool they were using for measuring.
Amongst the different definitions of Scientific Reasoning we found very important the role of hypothetico-deductive reasoning as a psychological skill involving observation, questioning, hypotheses creation, hypothesis falsification. Another important link to Scientific Reasoning is the scientific method, as a method or series of steps followed to create and evaluate hypotheses. Even though both seem similar, they are different in the sense that one is a cognitive skill that can be used in everyday life, while the other is a series of standardised steps scientists use when conducting experiments.
In relation to Scientific Thinking, we found the early definitions found in the beginning of the 1900s texts, as well as definitions based on Philosophy of Science to be the best epistemologically speaking. This definition sees Scientific Thinking as the set of different scientific mindsets which emerge in a certain culture at a specific time. That is, whenever we talk about The Age of Enlightenment or The Industrial Revolution we are talking about Scientific Thinking
Regarding the best definitions I particularly found concerning Scientific Reasoning, was the definition made by Fischer et al. (2014) which is one of the most updated ones, it is epistemologically founded, accounts for interdisciplinary views, and rather than diversifying aims to unify different theoretical postures. Regarding Scientific Thinking I think the works of Harre (2004) and Popper (1966) are a good framework for understanding Scientific Thinking as a socio-historical process of development of scientific theories.
The main takeaway of the article is to point out that in order to create sturdy and valid science it is not enough to use some standard tools or to quote authors talking about a specific topic. We need to make our science more standard, we need to use terms consistently within (and sometimes between) disciplines, we need transparency when defining, characterising, and operationalising our terms, so we can ensure that we all are measuring the same thing. If we take the time to do this, replicating findings (particularly in disciplines such as Psychology), communicating, and interpreting results will be easier.
The best foundation for a sound and strong science lies in the rigour of definition, use, characterisation, and operationalisation of their concepts. If we don't get to that, we will be referring to and measuring different things while thinking we are talking about the same thing. This can not be achieved from one day to another, it is a constant construction, and the idea is to polish it more every time, not to make it more diffuse with every piece of research.