Tuesday 20-02-2024 - 11:39

Image: author’s own. Existing research papers become the unit of analysis and building blocks of literature reviews, but they are not all comparable or compatible

 

Caroline Kocel, DTA Future Socieites Researcher, Anglia Ruskin University  

Literature reviews are foundational to academic research since knowledge is a cumulative and co-operative effort - we stand on the shoulders of giants. Before you get going and find yourself swirling in the currents of google scholar, it’s worth taking time to learn a little about different kinds of literature reviews to have a clearer sense of what it is you are – and are not – planning to do. Talk to your supervisor about what kind of literature review you’re planning to make sure your expectations are shared.  

Here I consider what is common among the different kinds of reviews before looking at different kinds of literature reviews and what they are for; scoping, narrative, descriptive, integrative, systematic and realist. I then give a personal critique on the conventions of literature reviews which reproduce multiple biases.

 

No matter what you are studying, literature reviews take time. Use yours wisely.

 

Source: Narrative vs systematic vs scoping review | What’s the difference? Research Masterminds video

 

What’s the Same in Different Kinds of Literature Review?

There are some characteristics which hold true whichever kind of literature review you are conducting.

• You need a clearly articulated research question to conduct a quality literature review.

But don’t get hung up on the question being the right one for the next few years, it may change and that’s ok.

• Garbage in = garbage out

Within the literature review, write ups of previous research become the data. How will you assess their quality? What information are you looking for in each paper?

• Transparency and reproducibility

Keep a note of your search parameters, dates, and findings - how will you justify your decisions? Can others reproduce your strategy?

• Find the gap

A ’factual’ gap is a piece of information that is missing; a ‘perspective’ gap is where an existing problem is looked at from a different angle to provide fresh insight; a ‘connection’ gap brings together findings from different disciplines.

 

Different Reviews

Scoping review

A scoping review is an early exploration of what is out there already to identify and analyse knowledge gaps. It should be as comprehensive as possible so grey literature could well be included. A transparent – often peer-reviewed – search strategy is used, with defined exclusion criteria to remove studies which don’t match the research question. It can be analysed using content or thematic analysis. The outcome of a scoping review is usually a recommendation for future research. Sounds great? Done properly it can take a whole team a long time.

 

Narrative review

A narrative (or ‘traditional’) literature review aims to summarise or synthesise what has been written on a particular topic, but has less formalized review and data extraction techniques than a scoping review. This does not seek cumulative knowledge from what is reviewed but rather provides an overview of what has come before. This approach is useful when researching topics that have been conceptualised differently by different disciplines.

 

“A narrative literature review is valuable…when one is attempting to link together many studies on different topics, either for purposes of reinterpretation or interconnection.” Baumeister and Leary.

 

Narrative reviews have been criticised for being unstructured in approach or lacking rigour but guidance exists on how to go about a structured narrative review and avoid the common pitfalls associated with this kind of review.

 

Descriptive or mapping review

Descriptive reviews are used to reveal any patterns in a particular topic with regard to propositions, theories, methodologies or findings. Paré and Kitsiou (2017) state that descriptive reviews follow a systematic and transparent procedure (in contrast to narrative reviews). By including information such as research methods, date of publication and strength of research outcomes, a frequency analysis can be performed to get quantitative results and reveal trends. ......

 

Integrative or critical review

In contrast to the narrative review, a critical review seeks not only to assess and synthesise the literature on a research topic, but to critique it in a way that allows new theoretical frameworks and perspectives to emerge. It could be used when looking at mature topics to critically review the existing knowledge base, or to reveal innovative theoretical models for newly emerging topics. It aims to inform others about the weaknesses of prior research and provide direction for future research.

 

Systematic review

Developed within the field of medicine, systematic reviews have been developed as a way to synthesise the findings of research on a particular topic in a way that is systematic, transparent and reproducible. Often, the research methods and expected outcomes are shared publicly with a register such as PROSPERO. Once all evidence has been collected on the basis of pre-defined inclusion and exclusion criteria, they are analysed usually using statistical methods. In cases where the results of similar studies are available, a meta-analysis may be used to calculate an overall effect size. Systematic reviews often seek to identify causation. PRISMA – the Preferred Reporting Items for Systematic Reviews and Meta-Anlyses - is a set of guidelines aiming to improve reporting in these kinds of studies.

 

Realist review

An alternative to positivist systematic reviews and a reaction from criticisms of them which seek to answer the simplistic question “what works?”. A realist review seeks to unpack the complexity of "what is it about this intervention that works, for whom, in what circumstances, in what respects and why?". Take a look at RAMESES publication standards for guidance. 

 

“As a theory-building approach, a realist review usually stats by articulating likely underlying mechanisms and then scrutinises available evidence to find out whether and where these mechanisms are applicable” (Shepperd et al., 2009).

 

Source: https://www.ncbi.nlm.nih.gov/books/NBK481583/table/c9.t1/?report=objectonly

 

Troubles with Literature Reviews

Biases inherent in the current ecology of academic publications are reproduced through standard literature reviews. Gender, racial, and other biases are well-documented – how / can we challenge them? The majority of research deemed high impact is written in the English language and though most literature reviews acknowledge this limitation, don’t the six billion non-English speaking people of this world deserve a little more? ‘Negative’ results, such as ‘results suggest no effect’ are less likely to be published, yet under-reporting of negative results feeds into bias in meta-analysis, which can skew findings of systematic reviews. Real-life consequences are that a specific medical intervention is considered more effective than it actually is. Finally, research exists to understand and improve the world around us. Although knowledge is not only produced or transmitted through academia, ‘grey literature’ (published outside of the academic peer-reviewed process) is often excluded. Few doubt the power of media to sway public opinion or policy on a particular issue, so how can we understand what’s going on by excluding it?

 

I haven’t met a PhD student yet who doesn’t understand the importance of a literature review. We know that reading and writing are huge components of the research and thesis and there is a strong push to get going. But given the competing demands of your PhD – and your life as a whole – make time before you start to reflect on what kind of review you are doing and why. What information are you looking for and not looking for, and why? Document what you are doing from the start so that later you can retrace your steps and justify the decisions you made. This will allow you to see the mistakes you made and paths that you chose not to go down or which didn’t lead somewhere useful. You will be on your literature review journey until the day you’ve defended your work, so learn from others and think before and after you read.

 

 

 

 

References and Further Reading

 

Avery, D. R., B, D. K., Dumas, T. L., George, E., Joshi, A., Loyd, D.L., van Knippenberg, D., Wang, M., & Xu, H. (Howie). (2002). Racial Biases in the Publication Process: Exploring Expressions and Solutions. Journal of Management, 48(1), 7-16. https://doi.org/10.1177/01492063211030561

 

Baumeister R.F., Leary M.R. Writing narrative literature reviews. Review of General Psychology. 1997; 1(3):311-320. Available here.   

 

Lokman I. Meho; Gender gap among highly cited researchers, 2014–2021. Quantitative Science Studies 2022; 3 (4): 1003–1023. doi: https://doi.org/10.1162/qss_a_00218

 

Mlinarić A, Horvat M, Šupak Smolčič V. Dealing with the positive publication bias: Why you should really publish your negative results. Biochem Med (Zagreb). 2017 Oct 15;27(3):030201. Doi: 10.11613/BM.2017.030201. PMID: 29180912; PMCID: PMC5696751.  

 

Munn, Z., Peters, M.D.J., Stern, C. et al. Systematic review or scoping review? Guidance for authors when choosing between a systematic or scoping review approach. BMC Med Res Methodology 18, 143 (2018). https://doi.org/10.1186/s12874-018-0611-x.

 

Paré, G. and Kitsiou, S., 2017. Methods for literature reviews. In: F. Lau and C. Kuziemsky, eds. Handbook of ehealth evaluation: an evidence-based approach. Victoria, BC: University of Victoria. Ch.9. Available at: https://www.ncbi.nlm.nih.gov/books/NBK481583/

 

Snyder, H., 2019. Literature review as a research methodology: an overview and guidelines. Journal of Business Research, 104(2019), pp.333-339. https://doi.org/10.1016/j.jbusres.2019.07.039. 

 

Wong, G., Greenhalgh, T., Westhorp, G. et al. RAMESES publication standards: realist syntheses. BMC Med 11, 21 (2013). https://doi.org/10.1186/1741-7015-11-21