One of my papers is going to be a systematic review, so for the past few weeks I have been busy searching and selecting articles. Of course I encountered some problems, because it would not be a PhD if everything went right the first time! Saskia thought it might be interesting if I share some of my experiences with you guys.
Search terms
So I don’t know about you guys, but the subject of my research has many synonyms. While I usually refer to my topic as being “intermediate assessment” other researchers have talked about frequent testing, continuous testing, or I don’t know how many other combinations. The problem with topics that have a lot of synonyms is that you can miss some very relevant articles, just because, for example, you only searched for testing and not for exams.
To avoid this I re-read some articles I have used in my research before to see what terms they used. Another smart strategy to define your search terms is looking for other reviews about the same subject and see which search terms they use. Of course if you do find other reviews, make sure that yours is an interesting addition!
After going through some articles I read before and thinking of common synonyms myself I decided on a list of six words that could all signify assessment and seven terms that could indicate some form of intermediateness.
Searching
So you’ve decided on search terms, now it is time to search! There are so many different search engines and I do not pretend to know them all, but I have used the ones that are most common in the field of educational science, which are ERIC, PsycINFO and Web of Science. Your professor or librarian may know which are most suitable for your field.
Fun fact. When I asked Web of Science to search for a combination of my 13 search terms, it found almost FIVE.HUNDRED.THOUSAND. results. Even after refining by saying I only wanted articles related to education and educational research, there were still over 8800 relevant articles. As you can imagine this is too much to go through, so I decided to switch to search term pairs. This means I had to do several searches and for now I have only combined my seven adjectives with “assessment”. In the future I may expand my search into the other synonyms for assessment, but for now I’ve decided to go and see how many articles I can select from this.
Selecting
After running a search you are left with a long list of articles that you need to check for relevance. To make efficient use of your time this first check for selection is usually done based on the title and abstract of an article. Of course this is where you run into problems if titles and abstracts are not informative enough for you to make an informed decision. For these doubtful cases you may need to read the full article.
Furthermore, selecting is a human task, meaning that it is not totally objective. I supplied my daily supervisor with one list of search results and my criteria for selection to check my process. Whereas I selected 33 articles, he only selected 24 and of course he selected some articles that I didn’t and vice versa. So now we need to sit together and see why we chose such different articles.
If you’ve selected articles you need to download the full text for reading. Fortunately, I found 260 full text articles, out of 281 selected search results.
Reading
After selecting the articles you can start reading and extracting the relevant information for your review. This is what I am starting to do today, so I cannot provide any tips or experiences yet, but maybe you guys have some top tips for me? For example how I am going to survive reading 260 articles in the coming months!!
0
0
0
Hi Indira, yes doing systematic reviews has quite a lot of challenges involved! Indeed, picking the proper search terms and using several databases is very relevant to do so. Before you begin reading the selected articles and extract data from them, you need to find out what you want to know, or to extract from them. that’ll follow from your research question(s). What I do, is developing some format with things I want to know, for example the participants (who/how many etc.) and data collection strategies, and of course their results. After reading these 260 articles (think of a good way to store all the extracted data!, Word, Excell, SPSS…), there are many ways of analyzing this data; and here are a few reading suggestions: (1) Dixon-Woods, M., Agarwal, S., Jones, D., Young, B., & Sutton, A. (2005). Synthesising qualitative and quantitative evidence: A review of possible methods a review of possible methods. Journal of Health Services Research & Policy, 10(1), 45–53. (2) Heyvaert, M., Maes, B., & Onghena, P. (2011). Mixed methods research synthesis: definition, framework, and potential. Quality & Quantity, 47(2), 659–676. doi:10.1007/s11135-011-9538-6 (3) Thomas, J., & Harden, A. (2008). Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Medical Research Methodology, 8, 45. doi:10.1186/1471-2288-8-45
Hi Marieke,
Yes I’ve been working on an extraction table to really structure what I get from the articles! I’ll discuss it with my supervisors later today and see if they think I have all the important points covered.
Thanks for the resources, I’ll look into them!