Archives for : March 2016

Doing a systematic review  2

One of my papers is going to be a systematic review, so for the past few weeks I have been busy searching and selecting articles. Of course I encountered some problems, because it would not be a PhD if everything went right the first time! Saskia thought it might be interesting if I share some of my experiences with you guys.

 

Search terms

So I don’t know about you guys, but the subject of my research has many synonyms. While I usually refer to my topic as being “intermediate assessment” other researchers have talked about frequent testing, continuous testing, or I don’t know how many other combinations. The problem with topics that have a lot of synonyms is that you can miss some very relevant articles, just because, for example, you only searched for testing and not for exams.

To avoid this I re-read some articles I have used in my research before to see what terms they used. Another smart strategy to define your search terms is looking for other reviews about the same subject and see which search terms they use. Of course if you do find other reviews, make sure that yours is an interesting addition!

After going through some articles I read before and thinking of common synonyms myself I decided on a list of six words that could all signify assessment and seven terms that could indicate some form of intermediateness.

 

Digital illustration of magnifying glass

Searching

So you’ve decided on search terms, now it is time to search! There are so many different search engines and I do not pretend to know them all, but I have used the ones that are most common in the field of educational science, which are ERIC, PsycINFO and Web  of Science. Your professor or librarian may know which are most suitable for your field.

Fun fact. When I asked Web of Science to search for a combination of my 13 search terms, it found almost FIVE.HUNDRED.THOUSAND. results. Even after refining by saying I only wanted articles related to education and educational research, there were still over 8800 relevant articles. As you can imagine this is too much to go through, so I decided to switch to search term pairs. This means I had to do several searches and for now I have only combined my seven adjectives with “assessment”. In the future I may expand my search into the other synonyms for assessment, but for now I’ve decided to go and see how many articles I can select from this.

 

 

Selecting

After running a search you are left with a long list of articles that you need to check for relevance. To make efficient use of your time this first check for selection is usually done based on the title and abstract of an article. Of course this is where you run into problems if titles and abstracts are not informative enough for you to make an informed decision. For these doubtful cases you may need to read the full article.

Furthermore, selecting is a human task, meaning that it is not totally objective. I supplied my daily supervisor with one list of search results and my criteria for selection to check my process. Whereas I selected 33 articles, he only selected 24 and of course he selected some articles that I didn’t and vice versa. So now we need to sit together and see why we chose such different articles.

If you’ve selected articles you need to download the full text for reading. Fortunately, I found 260 full text articles, out of 281 selected search results.

 

required-reading

 

 

 

 

 

 

 

 

 

Reading

After selecting the articles you can start reading and extracting the relevant information for your review. This is what I am starting to do today, so I cannot provide any tips or experiences yet, but maybe you guys have some top tips for me? For example how I am going to survive reading 260 articles in the coming months!!

PhD: Prepare for humdrum Drag?  0

Although many of my friends keep asking me “How do you do this [PhD]?!” (and I keep thinking: Doing what? I just started and I ‘still’ like it…), this post won’t be about self-discipline, self-management, self-reflection, multi-tasking, meta-cognition or another kind of a PhD students’ conditional characteristic.* I enjoyed reading my colleague’s blog on being a PhD student (https://researchblog.iclon.nl/planning-everything-reading-writing-still/) and I could probably write another ten pages about this topic. However, what I chose to share with you today are my experiences with working in an inter-university research project so far.

It has been only 4,5 months since I started my PhD at ICLON. My PhD track is, just like many other PhD tracks, comparable to a (lonely) life on a private island. However, what makes my project less lonely (or less independent), are the people involved in this project; stakeholders so to say. As your research develops, more and more people will get involved. Probably every PhD student will get to the point of receiving feedback from various people with different perspectives, different goals in mind, or at least different pathways leading towards a certain goal. The starting point of my research project is: 3 universities, 3 interdependent projects, and 9 interdependent researchers. Yes: N-i-n-e. I thought it would be nice to tell you a little bit about my who-what-were-when-dilemmas a.k.a. Prepare for humdrum Drag (PhD). Note (!!!): This is not a manual including Do’s and Don’ts but simply a description of my thoughts. It also isn’t a genuine portrayal of reality but rather an exaggerated sarcastic one.

 

  • WHO’s the lucky one? Collaboration. I find it an intriguing phenomena as its process is dependent upon so many different factors. Among others: Expectation management. The projects we’re working on is interdependent in the sense that we’re dependent on each other’s data. This is a good thing, from a sharing-interests-perspective, as a smooth collection of data is close to everyone’s heart. However, from a responsibility-perspective it’s a little bit more complicated. Who’s doing what, where, and when? In other words: Who feels responsible? Me. For everything. Duh, that’s why I’m a PhD student (did somebody say control freak?).

 

are-you-a-control-freak

 

  • WHAT are you talking about? Something can make perfect sense to me and make no sense to others at the same time. It has only been 4,5 months since I started and I already get the feeling of being a puzzled researcher in a world of my own. What I mean to say is; Research is a process in one’s head. Before concrete products appear (such as a theoretical framework, concrete measurement instruments, or even black on white results), we think, read, rethink, reread, etcetera. As a result, meetings so far exist of brain-storming, weighting pros and cons, reformulating ideas, etcetera. In other words, the challenge is to translate our personal thinking process of, let’s say four weeks, to a meeting of two hours (with 8 other researchers also having their own thoughts).

 

justin-bieber-what-do-you-mean

I finally got to understand Bieber

 

  • WHERE?  A picture can say a thousand words…

 

1

Where to save files & notes?  

 

2

Where to synchronize latest versions to?

 

  • WHEN: To mail or not to mail? I think this type of humdrum drag is comparable to the Dutch saying “He shot all his gunpowder” (Hij heeft al zijn kruit verschoten). On a normal working day several issues will cross your mind that you would like to discuss with others. Will you decide to wait and think about it yourself for a couple of hours or days more? Or will you mail your colleagues and ask for their input? In the latter case, be aware that 1) This may result in a day of mailing back and forth without actually moving forward (my personal record is 21 mails of 7 people within 2 hours on 1 specific topic), or 2) Your colleagues will be fed up with your e-mails and won’t respond to your e-mails a second time.

 

* I realize that, in the end, I actually did write about it…

 

 

 

 

 

 

A Pirate’s Treasure Trove  0

Last time, I was thrilled to announce that my data had arrived. Now, as I am looking though the piles and piles of questionnaires, new questions have risen in my mind:

  • Why would someone cut out squares from the pages of their questionnaire?
  • I said: “tick the boxes in black or blue pen”. Not: “tick them in red, orange, yellow, lightblue or barely visible pencil”!
  • Why do these kids leave so many questions unanswered? I didn’t put that question mark in for nothing!
  • Why does a significant portion of the student population think my picture on the announcement letter ought to have a beard?

But the most important thing I discovered was that not all students fill in the questionnaire seriously. Of course, that could have been expected, but naïve as I was, I thought  it wouldn’t be so bad. While scanning in the hardcopy questionnaires into the computer that can read the answers for me, I came across a group of 6 questionnaires that were all filled out in a zigzag pattern. Obviously, this had been a group of silly teenagers who were sitting together and had come up with the idea of transforming the questionnaire into a zigzag artwork. Unfortunately, I could not use these questionnaires anymore. From then on, before I scanned the questionnaires, I manually looked through all of them to discover any other potential jokers. While doing this, I also discovered some other nice (and some not so nice, and some even very rude) artworks of students on the questionnaires. I made a selection of the nice ones as a keepsake.20151217_143010

“Good luck with your research, stranger”. Well thank you, stranger!

20151218_151800

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Nice hat!

20160122_103338

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

“This makes me cranky”

Collage

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

An assembly of some nice artworks.

20151119_163555

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

A Harry Potter-with-a-beard look and a mathematical problem.

20150917_153019

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Some students thought I should be a pirate instead of a researcher. Time for a career swap?

 

It is a little like snail mail: students responded to my “letter”. Although not very extensive or polite sometimes, it was a peculiar and surprising form of communication. I have enjoyed looking for the most creative outbursts on the paper.

Which one is your favourite? And have you ever experienced students drawing on your questionnaire or picture – and if so, did they also think you were meant to be a pirate?

Until next time mateys, ARRR!

Pirate treasure

 

 

 

 

 

 

 

 

 

 

 

 

Well, this pirate is going to search for more treasures in her data!

Image source: blogs.disney.com