ASReview: AI-assistance for article screening  0

 

If you’re a researcher, you probably have conducted literature reviews or will do so in the future. Depending on your keywords, a search in online databases easily results in several hundred or even thousands of hits. One of the most time consuming steps is screening all those titles and abstracts to determine which articles may be of interest for your review. Isn’t there a way to speed up this process? Yes, there is!

 

Recently, we started a literature review about the use of Open Educational Resources (OER) in K12-education. We came across a great tool for article screening. In this blog, we will introduce this tool and share our experiences.

 

Training the system

Researchers at the University of Utrecht have developed an open source and free screening tool to help researchers go through the enormous digital pile of papers: ASReview LAB (see www.asreview.nl). In a 2-minute introduction video, you can learn how it works. Basically, the programs helps you to systematically review your documents faster than you could ever do on your own by, as they put it, “combining machine learning with your expertise, while giving you full control of the actual decisions”. We just had to try that!

 

First, we made sure the papers we’ve found all included a title and an abstract as that is what we would use to screen them on relevance. It was very easy to import our RIS file (from Zotero in our case, but can be from any reference management system) with all the hits from our search query. Then it was time to teach ASReview! We provided the system with a selection of relevant and irrelevant articles which it uses to identify potential matches, thus expediting the screening process. Following the guidelines provided in the ASReview documentation, we utilized the default settings of the AI model.

 

The researcher as the oracle

Once the system was trained, the screening phase could start. At each stage, we evaluated whether a document was relevant or not, providing notes to justify the decisions. In cases of uncertainty, where the abstract alone was not sufficient to make a judgment, we referred to the full text of the article. With each decision, ASReview adapts its learning model to ensure that as many relevant papers are shuffled to the top of the stack. That’s why it is important to make the ‘right’ decision. We worked in the ‘Oracle Mode’ (other modes are possible as well, but for reviews this is the best) which makes the researcher ‘the oracle’. ASReview describes the relevance of taking your time to make decisions: “If you are in doubt about your decision, take your time as you are the oracle. Based on your input, a new model will be trained, and you do not want to confuse the prediction mode.” (ASReview, 2023). So make sure that you carefully formulate your research questions and inclusion criteria before beginning to screen the articles. This helps to decide if an article might be of interest or not.

 

To avoid endless manual screening (which is kind of the point of using this tool), it was recommended to formulate a stop rule. To formulate our stop rule we made use of the recommendations provided by ASReview and Van de Schoot et al. (2021). According to our rule, screening would cease once at least 33% of the documents were reviewed AND ASReview presented 25 consecutive irrelevant items. This approach helped prevent exhaustive screening while maintaining rigor and reliability. A tip for maintaining focus is to spend a limited amount of time per day screening articles (for example a maximum of two hours a day). Throughout the screening process, ASReview’s dashboard provided a visual overview of progress and decisions made.

 

In total, 460 items were excluded by the system, while 324 were manually screened, with 173 rejected for various reasons. These reasons ranged from focusing on specific educational technologies to addressing broader educational issues beyond the scope of the study. To ensure the reliability of the screening process, a second researcher independently assessed a random sample of 10% of the documents.

 

 

 

Combining AI and human judgment

After completing the screening process, it is very easy to download a file with an overview of all the decisions made, including both relevant and irrelevant articles. The dashboard and the output files help you in reporting why certain articles were excluded from the review. Notably, the PRISMA model already accommodates for articles excluded through AI. So, in conclusion, ASReview offers a powerful solution for streamlining the literature review process, leveraging AI to expedite screening while maintaining the integrity of the review. It combines the efficiency of AI with human judgment, saving you time – something welcomed by all.

Lysanne Post & Marjon Baas

 

 

Read more:

Share This Post!Share on LinkedInTweet about this on TwitterShare on FacebookEmail this to someone

What is needed to be inclusive in teaching and research?  0

Source: unpslash: Braedon McLeod

Photo: unsplashed; Braedon McLeod

 

This month, the 10th Diversity & Inclusion Symposium took place, organized by the Diversity & Inclusion Expertise Office and the Faculty of Archaeology. For the small ICLON delegation that attended, the event highlighted how the questions, challenges and opportunities we face are not dissimilar to those experienced by colleagues elsewhere in our organization.

 

After 10 years of D&I policy, it has become clear that addressing equity, diversity and inclusion in meaningful and impactful way remains challenging. The symposium’s plenary speakers highlighted how the university is a place not just for research and knowledge sharing, but also where students and staff must learn to navigate complex and conflicting conversations. At ICLON, this is a topic that has been very much on our mind lately, as researchers, but also as teacher educators and trainers, and as an organization more broadly. How can ICLON research keep addressing these challenges? And what aspects of research and education should be emphasized in in order to contribute to an inclusive society?

 

Untold stories and questioning

The theme of the symposium was “Untold Stories.” In her opening keynote, Dr Valika Smeulders from the Rijksmuseum demonstrated how the museum navigates complex conversations effectively using heritage and fragile pasts. She explained about breaking existing frameworks and dominant narratives through multi-perspectivity and personal stories. In times of polarization, heritage can function to facilitate an open dialogue but also be a trigger for a heated debate.

 

This notion underpinned our recent research published in a history education journal. Collaborating with the Rijksmuseum van Oudheden, we developed a training for history teachers on addressing sensitive topics. Using concrete heritage objects and varied questions, teachers created space for students to share their perspectives and tell their stories. Following the intervention, teachers felt better-equipped to navigate such conversations in their classrooms, as observed in lessons addressing contentious issues like “Zwarte Piet”. Students and teachers were stimulated to ask questions. Certain questions can ‘heat up’ cooled down issues and hot topics can be ‘cooled down’ by questioning and not focusing on ‘the right answer’.

Photo: unsplashed Kind and Curious

 

Dealing with discomfort and doubt

Maintaining such dialogue and continuing to question can be difficult. In a workshop at the same symposium, by Ruben Treurniet from Civinc, participants engaged with each other anonymously using a tool that connects individuals with differing views. Through an online chat session, we exchanged thoughts on statements like “Debates about academic freedom should also involve the possibility of defending the right to not be inclusive.” A slight majority disagreed with this statement. The app encouraged us to ask each other questions, and provided an intriguing opportunity to converse with someone outside of one’s usual ’bubble’.

 

These anonymous discussions can foster some form of connection, and can be a useful tool in developing mutual understanding. In our professional context, however, we do not generally communicate through anonymous chat, but through face-to-face encounters, with their accompanying tone, body language and emotional load. Conversations on controversial topics can become tense and confrontational, and can actually reinforce relationships of power and dominance. Explicitly expressing feelings, doubt and judgments can also be also daunting for educators and researchers expected to exude authority, or who are anxious about repercussions if they do not maintain a ‘neutral’ standpoint. However, it is important that we, as researchers and educators, demonstrate the art of doubt and model how to deal with uncertainty.

 

Interdisciplinarity and positionality

Finally, it may be beneficial to revisit certain cooled-down topics to practice interdisciplinary thinking and multi-perspectivity. A historical perspective, as shown by Valika Smeulders, can offer various narratives, demonstrating how history is a construct accommodating diverse viewpoints. An issue that is ‘hot’ in the present could be ‘normal’ in the past and vice versa. Looking beyond your own time and discipline can be inspiring and helpful. Collaborating across disciplines broadens perspectives while requiring us to clarify our own viewpoint through questioning and being questioned. At the moment, this principle is being applied in ongoing research with history and biology teacher trainees.Source: unpslash: Braedon McLeod

 

Other current projects at ICLON are exploring culturally sensitive teaching, linguistic diversity, approaches to inclusion, and teacher, student teacher and teacher educator perspectives on equality, equity and social justice. These sensitive areas of research can create vulnerable situations for participants and researchers alike. They demand researchers’ critical awareness of their positionality, grappling with their values and giving space to non-dominant perspectives, while also contributing to authoritative knowledge and relevant practical applications.

 

Perhaps interdisciplinary and positionality could be a theme for a future symposium, bridging the diverse perspectives, experiences and expertise at ICLON and the university more widely. We could show what ICLON can offer regarding questioning, dealing with discomfort and interdisciplinarity, and open space for further dialogue at our university.

 

Acknowledgments

Many thanks to the Diversity and Inclusion Expertise Office and the Faculty of Archeology for organizing this symposium and thanks to Dr. Tessa Mearns for her helpful comments while writing this text.

 

Reference

Logtenberg, A., Savenije, G., de Bruijn, P., Epping, T., & Goijens, G. (2024). Teaching sensitive topics: Training history teachers in collaboration with the museum. Historical Encounters, 11(1), 43-59. https://doi.org/10.52289/hej11.104

 

Share This Post!Share on LinkedInTweet about this on TwitterShare on FacebookEmail this to someone

The essential role of educational sciences for innovations in university teaching  0

Recently NWO release a new funding call for educational innovation projects, labelled “Scholarship of teaching and Learning”. This is an interesting funding opportunity for academics who would like to strengthen their teaching. Academic teachers can apply for funds to put their innovative teaching ideas into practice. And indeed this is a good opportunity to get your funding for those teaching ideas you have been waiting to implement. This also is the time to re-think your teaching and teaching ideas and put them to the test.

Read more

Share This Post!Share on LinkedInTweet about this on TwitterShare on FacebookEmail this to someone

The pedagogy of hope in times of crises  0

Photo: Akil Mazumber. on Pexels

Last week I visited Germany for my study around the climate crisis and the issue of hope. One German student said: Can’t we as teachers just tell students to become vegetarians to save the planet? 

What do you think, would it be a solution, or wise, to tell students what to eat, drink, vote, do or think in order to bring about change? I mean, shouldn’t we do  something as we know that studies about the effects of climate change on young people reveal that pessimism, guilt, hopelessness and fear are common in the new generation?

 

Bringing about change in times of the many present day crises with all the doom stories and anxiety is an interesting, yet challenging research topic. Interestingly,  precisely in the midst of complex crises, those who provide education have a crucial role: to make the new generation appear to the world as powerful and innovative (Arendt, 1958). To not reinforce fear or impose what to do or think, but have the new generation discover from hope that a different future is possible and that even a crisis includes profound problems-though complex and intractable- for which solutions can be found. A focus on hope is key!

 

Hope as a construct has received attention from many different angles, such as psychology, theology, philosophy and recently even famous primatologist and anthropologist Jane Goodall (2021). Yet, although many authors endorse the need and importance of hope, to date there has been little innovation in the ways in which hope can have a practical impact and lead to change, let alone in education. In my research project, hope has been incorporated into a pedagogy of hope. It holds several powerful design principles for a pedagogy of hope stemming from pilots in teacher education institutes in both the Netherlands and Germany and is now tested in the context of the climate crisis. Around this climate crisis, pre-service teachers are known to feel very committed to teaching the topic, but at the same time concerned and anxious about the climate themselves and ignorant in how to provide hopeful and effective teaching about the climate crisis in their secondary school internship classes (Bean, 2016).

 

The pedagogy of hope was implemented in a Dutch and German teacher education institute. The preliminary outcomes show that participants were able to formulate specific intentions that are both directed toward hope for the climate as well as easy to implement in their actual teaching in secondary education. Also, many intentions show to be action-oriented and participants often used their creativity to find non-traditional ways of conveying climate hope. We also found hindrances for teaching hopefully, such as not enough time, curriculum coverage and a lacking attention in textbooks for climate change and climate hope. Also, the different opinions that others could have could make it a controversial issue to teach in school.

 

On to the next steps!

 

Michiel Dam, researcher at ICLON, LTA teaching fellow

Hoopvol klimaatonderwijs – Universiteit Leiden

Vijf nieuwe Teaching Fellows benoemd – Universiteit Leiden

 

References

Arendt, H. (1958). The crisis in education. In H. Arendt (Ed.), Between the past and present. (pp. 101-124). Leuven/Apeldoorn: Garant.

Bean, H. J. (2016). Pre-Service Teachers and Climate Change: A Stalemate? Australian Journal of Teacher Education, 41(4).

Goodall, J. & Abrams, D. (2021). The Book of Hope: A Survival Guide for Trying Times. Celadon books: New York city, New York.

Share This Post!Share on LinkedInTweet about this on TwitterShare on FacebookEmail this to someone

The Negative Consequences of Multiple Choice Testing for Student Learning  0

Exam” by albertogp123 is licensed under CC BY 2.0.

 

Multiple choice (MC) testing is generally viewed as an efficient way of assessing student knowledge. Up until today it has been a very popular assessment format in higher education. Especially in courses where a large number of students are enrolled, the MC format is often used as it allows for fast and reliable grading. Not surprisingly, as an educational consultant and teacher trainer much of my work has revolved around supporting teachers in creating and/or improving their MC assessments. Throughout the years, I have taught numerous training sessions on Testing and Assessment for the University Teacher Qualification at Leiden University. On the one hand these training sessions are designed to teach best practices, but at the same time the sessions are also designed to cater to teacher needs. As such, a large part of the sessions is focused on giving teachers instructions and tips on how to create good MC questions. To be sure, I have always managed to squeeze in some discussion on the downsides and many limitations of MC testing as well. But still… It always kept me feeling a bit uneasy. In giving the instructions that the program compelled me to, I might have inadvertently been endorsing this practice more than I would have wanted. Thus, this blogpost will be as much repentance as it is a cautionary exposition about some of the negative consequences that MC testing can have on student learning.

 

There are multiple reasons for why MC exams could be considered as detrimental for student learning. For instance, one often heard criticism is that the recognition-based MC exam format will often result in students preparing for exams in a superficial way. Furthermore, one could argue that the ecological validity of MC exams is low and not representative of real-world situations. Also, the MC test format is by design not suitable for assessing higher levels of learning. These kind of objections are well-known and they have also received considerable attention in the University teacher Qualification courses on testing and assessment taught at Leiden University. I am not going to reiterate them extensively in this blogpost. Instead, I will discuss one particularly negative consequence of MC testing that I think is often neglected: the misinformation effect.

 

The misinformation effect

Before we consider the misinformation effect in the context of MC testing, we will first take a step back and consider some general research on the workings of human memory and how misinformation can result in misremembering. One of the first general demonstrations of the misinformation effect was provided by Loftus & Palmer (1974). In Experiment 2 of their seminal study, participants watched a short video clip of a car accident. After watching the video, participants were asked to give an estimate of the speed of the cars that were involved in the accident. Half of the participants were asked to estimate the speed of the cars when they smashed into each other, while the other half of participants estimated the speed for when the cars hit each other. The subtle change of the verb used in the question resulted in a difference in the reported speed: Participants estimated the speed to have been higher when they were in the smashed condition. More importantly, one week after giving the speed estimates, participants returned and were asked to indicate whether they remembered seeing broken glass in the video. Interestingly, participants in the smashed condition were much more likely to report having seen broken glass even though there was none to be seen in the video.

 

The results from the Loftus & Palmer study are often cited in the context of the reliability of eye-witness testimonies (and the effects that leading questions can have on misremembering). More  importantly, the results are also taken as evidence in support for the idea that human memory is reconstructive in nature. During the retrieval of information from memory we reconstruct what we have previously experienced. When previously exposed to some form of misinformation, the process of reconstruction can result in substantial misremembering of previous experiences.

 

The misinformation effect in the context of MC questions

In the Loftus & Palmer (1974) study, the degree to which participants were exposed to misinformation was rather subtle (i.e., a small change of verb in the leading question). However, if we now consider the situation of an MC exam, the degree of exposure to misinformation seems much more extreme. A typical MC question will often have four alternatives for students to choose from of which the majority (usually three) is incorrect. Thus, by using MC exams, we are intentionally exposing our students to misinformation. MC exams are designed to do just that. Surely, you could argue that the negative consequences of MC exams might be less severe, because students are aware that they are being exposed to misinformation. They are going into the exam expecting this. However, in preparation of the exam, the teacher has also taken careful consideration of phrasing erroneous answers in such a way that they are plausible. Teachers are instructed to formulate alternatives that students are likely to mistakenly select as the correct one. By exposing students to misinformation in the context of MC exams, teachers might very well be sacrificing student learning for the sake of fast and reliable grading.

 

In a later study by Roediger & Marsh (2005)  the consequences of MC testing on student learning was investigated. In their experiment, participants studied short prose passages (or not) and were subsequently tested on the materials (or not) using MC questions with a number of alternatives ranging from 2 – 6. One week later participants returned and received an open-ended short answer test. Going into the test, participants were also given explicit instructions not to guess. First of all, the results on the 1-week test showed that the consequences of MC testing were not all bad: Taking a MC test increased the retention of (correct) information. This finding, also referred to as the testing effect, is well-established in the literature and has often been replicated across different test formats and settings (e.g., Rowland, 2014). On the other hand, however, being exposed to misinformation in the MC test, also increased the production of erroneous answers on the 1-week short answer test. The degree to which participants produced erroneous (MC) answers tended to increase as the number of alternatives of the MC test increased. Note that this was the case even though participants had received explicit instructions not to guess on the short answer test. Clearly, the misinformation effect is not just relevant in the context of eye witness testimonies, but also in the context of assessment in higher education. MC exams can have an adverse effect on student learning in the sense that students can mistakenly recall incorrect answer options at a later point in time. Later research (Butler & Roediger, 2008) has shown that the misinformation effect as a result of MC testing can be reduced by giving students direct feedback (either after each individual question or after taking an entire test). However, in my experience, summative MC exams in higher education usually don’t provide immediate feedback to students. In the absence of corrective feedback, students might stay under the impression that their erroneous responses on a test were correct.

 

To end on a positive note, there are promising alternatives for MC exams that teachers are exploring. For instance, at the Leiden University Medical Centre (LUMC) some teachers have started using Very Short Answer Questions (VSAQs) on exams as a substitute for MC questions. Among others, dr. Alexandra Langers (Leiden University Teaching Fellow), and her PhD student Elise van Wijk have started investigating the consequences of VSAQ exam format. VSAQs require students to generate short (1 – 4 word) answers to exam questions. Compared to MC questions, VSAQs require retrieval of correct answers rather than simple recognition and as such these type of questions can be more conducive for student learning. Because answers are short, VSAQs will still allow for some degree of automatic scoring (for some predetermined “correct” responses). This can keep grading time acceptable even for teachers with large classes. Some of the findings of the VSAQ research project have recently been published in an article in PLOS ONE. Replicating previous findings (Sam et al., 2018), van Wijk et al., (2023) demonstrate that VSAQ exams can have added benefits over MC tests in terms of higher reliability and discriminability. In addition, van Wijk at al. found that the average grading time per individual VSAQ was around two minutes. This seems very acceptable considering the cohort in the study consisted of more than 300 students. Hopefully, initiatives like the one at LUMC will pave the way for other teachers to start using assessment types that can be more supportive of student learning.

 

References

Butler, A. C., Roediger, H. L. (2008). Feedback enhances the positive effects and reduces the negative effects of multiple-choice testing. Memory & Cognition, 36, 604–616. https://doi.org/10.3758/MC.36.3.604

 

Loftus, E. F., & Palmer, J. C. (1974). Reconstruction of automobile destruction: An example of the interaction between language and memory. Journal of Verbal Learning & Verbal Behavior, 13, 585–589. https://doi.org/10.1016/S0022-5371(74)80011-3

 

Roediger, H. L., & Marsh, E. J. (2005). The positive and negative consequences of multiple-choice testing. Journal of experimental psychology. Learning, memory, and cognition, 31, 1155–1159. https://doi.org/10.1037/0278-7393.31.5.1155

 

Rowland C. A. (2014). The effect of testing versus restudy on retention: a meta-analytic review of the testing effect. Psychological bulletin, 140, 1432–1463. https://doi.org/10.1037/a0037559

 

Sam, A. H., Field, S. M., Collares, C .F., van der Vleuten, C. P. M., Wass, V. J., Melville, C., Harris, J., & Meeran, K. (2018), Very-short-answer questions: reliability, discrimination and acceptability. Med Educ, 52, 447-455. https://doi.org/10.1111/medu.13504

 

van Wijk, E. V., Janse, R. J., Ruijter, B. N., Rohling J. H. T., van der Kraan J., Crobach, S., de Jonge, M., de Beaufort, A. J., Dekker, F. W., Langers, A. M. J. (2023). Use of very short answer questions compared to multiple choice questions in undergraduate medical students: An external validation study. PLOS ONE, 18, e0288558. https://doi.org/10.1371/journal.pone.0288558

Share This Post!Share on LinkedInTweet about this on TwitterShare on FacebookEmail this to someone

Rethinking Teacher Quality in University: Embracing Teacher Agency  2

Max Kusters & Arjen de Vetten

Introduction

How often have you referred to a teacher as ‘good’ or ‘bad’ in your school career? When assessing teacher quality, we often struggle with subjective judgments and varying criteria. Is a teacher considered good because they explain well? Or because the teacher’s students obtain high scores? Perhaps it depends on positive evaluations from students. In this research blog, we aim to redefine the way we assess the quality of university educators and propose a shift toward embracing teacher agency. We argue that educators should be seen as experts in their field, who can not only meet the needs of students but also foster innovation. Educators’ willingness to take responsibility and contribute to institutional progress can significantly foster a transformative educational environment. In this regard, educators transcend their traditional role as mere providers of education and instead become facilitators of educational innovation and development.

Shortcoming of current methods

The current methods of assessing the quality of university educators have been widely criticized. For instance, several studies raise concerns about the interpretation and usefulness of Student Evaluation of Teaching (SET) ratings. These studies revealed that SET ratings were significantly influenced by students’ perceptions of their educators, thereby calling into question the validity of this specific assessment tool (see Shevlin et al., 2000 and Spooren et al., 2013 for examples).

There are also educational concerns, for example, that current assessment methods do not contribute to educators’ professional development. Among other things, assessment methods are often criticized for providing little or no constructive feedback. Without this, educators may find it difficult to improve their teaching methods or address weaknesses. Moreover, critics argue that current assessments often fail to take into account the teaching context, such as subject matter, class size, level (bachelor’s or master’s), and student diversity and background. Each of these factors can significantly affect teaching methods and outcomes and should be considered when assessing educators. Moreover, current assessment methods neglect broader purposes of teaching, such as the value of mentorship and creating an inclusive learning environment.

In some institutions, however, there is already a focus on a more holistic approach that integrates different sources of feedback, such as peer evaluations and self-reflection, to gain a more accurate understanding of teacher effectiveness, for example during the University Teaching Qualification track. The ability to reflect on what works and what does not work, and to understand why, is invaluable to teacher quality. Therefore, universities play a crucial role in promoting these skills, as these skills must be recognized and valued by policy makers by reflecting them in assessments. A change to holistic assessment of educators emphasizes effective teaching and the long-term effect educators can have on student growth. Educators need to actively pursue their own development and make informed choices in any given situation, highlighting the significance of teacher agency in discussions about teacher quality.

Embracing Teacher Agency

Embracing teacher agency in the evaluation of teacher quality is crucial for fostering a culture of innovation, growth, and student-centered education. Teacher agency refers to the ability of educators to make intentional choices and take purposeful actions in their teaching practice. It involves educators’ capacity to initiate and control the learning environment, make informed pedagogical decisions in any given situation, and collaborate with colleagues and students. Teacher agency is often seen as a key factor in promoting effective teaching and learning in higher education. By recognizing and valuing teacher agency, universities can tap into the expertise and unique perspectives of their educators.

Moreover, teacher agency encourages continuous professional development. When educators have the autonomy to explore and experiment with different instructional strategies, they are more inclined to seek out new research, attend workshops, collaborate with colleagues, and reflect on their own teaching practices. This proactive approach to professional growth ultimately benefits both educators and students, as it promotes a culture of lifelong learning and innovation within the educational institution.

Embracing teacher agency also cultivates a sense of trust and collaboration between faculty members and administration. Rather than imposing rigid evaluation criteria, universities can create opportunities for open dialogue, feedback, and collaboration, allowing educators to take an active role in shaping their own professional growth and the overall direction of the institution.

Conclusion

In conclusion, embracing teacher agency is a powerful facilitator for elevating teacher quality in universities. By empowering educators to exercise their expertise, make informed decisions, and engage in continuous professional development, universities can foster a dynamic and student-centered educational environment that nurtures innovation, growth, and excellence in teaching and learning. It is essential to engage in an active debate about the assessment of educational quality and challenge the predominant reliance on quantitative evaluations such as SET. Recognizing the complexity of assessing teacher quality, we propose a paradigm shift towards valuing teacher agency in universities. By fostering a culture that empowers educators, promotes collaboration, and encourages continuous learning, we can unlock the potential for lasting educational reforms. Embracing teacher agency is crucial for assessing educators’ quality effectively. By involving educators in the assessment process and valuing their expertise, autonomy, and professional judgment, we can create a more meaningful evaluation system. Practically, this can be achieved through collaborative goal setting, self-reflection and self-assessment, peer observations and feedback, diverse assessment methods, and continuous professional development. By recognizing educators as professionals and empowering them to take an active role in their own assessment, we create a comprehensive and empowering process that benefits both educators and students. Embracing teacher agency thus not only benefits individual educators, but also fosters an educational environment characterized by its dynamic and student-centered nature. It promotes innovation, encourages growth and strives for excellence in both teaching and learning. And that’s what we call: good teaching!

References

Biesta, G., Priestley, M., & Robinson, S. (2017). Talking about education: Exploring the significance of teachers’ talk for teacher agency. Journal of curriculum studies49(1), 38-54.

Cherng, H. Y. S., & Davis, L. A. (2019). Multicultural matters: An investigation of key assumptions of multicultural education reform in teacher education. Journal of Teacher Education70(3), 219-236.

Harris, A., & Jones, M. (2019). Teacher leadership and educational change. School Leadership & Management39(2), 123-126.

Imants, J., & Van der Wal, M. M. (2020). A model of teacher agency in professional development and school reform. Journal of Curriculum Studies52(1), 1-14.

Kusters, M., van der Rijst, R., de Vetten, A., & Admiraal, W. (2023). University lecturers as change agents: How do they perceive their professional agency?. Teaching and Teacher Education, 127, 104097.

Shevlin, M., Banyard, P., Davies, M., & Griffiths, M. (2000). The validity of student evaluation of teaching in higher education: love me, love my lectures?. Assessment & Evaluation in Higher Education25(4), 397-405.

Spooren, P., Brockx, B., & Mortelmans, D. (2013). On the validity of student evaluation of teaching: The state of the art. Review of Educational Research83(4), 598-642.

Tao, J., & Gao, X. (2017). Teacher agency and identity commitment in curricular reform. Teaching and teacher education63, 346-355.

Share This Post!Share on LinkedInTweet about this on TwitterShare on FacebookEmail this to someone

Passing the Buck — Blended Learning and Technology are both Accused of Low-Quality Education  0

This is from https://communitycomputerservices.com/does-a-broken-screen-mean-your-laptop-is-doomed/

Higher education institutions have adopted blended learning to actualize the benefits brought by technology. Specifically, the flexibility of learning afforded by blended courses make learning less dependent on specific space and time (Graham, 2006), and instructional method (Harding, 2012). In addition, blended learning is regarded as a more cost-effective approach than fully on-campus teaching (Harding, 2012). In addition to these potential benefits brought by blended learning, blended learning also provides a context in which students need to adapt themselves from the online environment to on-campus learning environments (Chiu, 2021) which at the same time, puts forward new requirements for teaching. Although many factors like teachers’ interaction with technology, academic workload, institutional environment, interactions with students, the instructor’s attitudes, and beliefs about teaching, and opportunities for professional development (Brown, 2016) are identified as factors influencing teachers’ adoption of blended learning, the institutional environment can always stand out in an authentic educational context. Many teachers are encouraged to use blended learning without thinking about whether it is aligned with their teaching goal and usually expect much from blended learning before they go to the classroom.

 

Potentials of blended learning

 

Blended learning is a balance of online learning and on-campus learning activities (Siemens, Gašević, & Dawson, 2015). Compared to on-campus and online learning, blended learning (through a Learning Management System) enables students to access learning materials whenever and wherever they want (Moskal, Dziuban, & Hartman, 2013) without the loss of obtaining in-person support and instruction in the classroom (Graham, Allen, & Ure, 2005). Additionally, blended learning can provide more effective instructional strategies to meet the course objectives than only online learning or fully on-campus learning (Graham et al., 2005; Morgan, 2002). This is probably because students can tailor their path of learning based on the personalized options afforded by blended learning (Medina, 2018). It can also improve students’ sense of belonging more than either on-campus or online learning (Rovai & Jordan, 2004). Zhu, Berri, and Zhang (2021) elaborated that, on the one hand, students may feel less disconnected because, with a blended learning situation, students can meet occasionally. On the other hand, they can have interactions and immediate feedback because they are interconnected on the online platform.

 

Challenges in Blended Learning

 

However, there are also many challenges in blended learning environments. Blended learning can be problematic if not appropriately designed and utilized. Even if appropriately designed and utilized it does not necessarily lead to the desired goal (Fisher, Perényi, & Birdthistle, 2021). Specifically, finding the optimal “blend” and coordinating all the elements is challenging for instructional designers and teachers of blended learning (Kaur, 2013). If the “blend” is not well-designed students will encounter difficulties in navigating between online learning and on-campus learning. Furthermore, there are technical challenges regarding ensuring student performance and achievement by utilizing and supporting appropriate technologies. In addition, isolation in online learning environments and distraction by non-academic online activities can hinder students’ engagement (Rasheed, Kamsin, & Abdullah, 2020). A crucial element is the amount of guidance and feedback in blended learning compared to on-campus learning due to the reduced in-person time with peers and teachers (Heinze & Procter, 2004). In all, blended learning requires more self-regulation skills and technology literacy than on-campus learning (Rasheed et al., 2020).

 

Technology as a Scapegoat

 

Ironically, although both advantages and disadvantages are afforded with the blended model, many stakeholders, like academics, course designers, and teachers, easily blamed technology for low engagement, increasing workload, and the failure of teaching. Specifically, some institutions and teachers were very glad when they heard they were allowed to return to campus. A teacher mentioned that she did not like teaching online with synchronous interaction because she felt that, to a large extent, she lost her control in building emotional bonds with students.

 

The distrust of technology and blended learning is very disappointing, especially to educational specialists who have invested huge amounts of energy in making blended learning more effective in learning. Joe O’Hara, a full professor of education at Dublin City University (DCU) and the president of the European Educational Research Association, commented “The ‘new normal? Two years of innovation and hard work consigned to the dustbin of (educational) history” under the news of “While some schools may close this week if local conditions are poor, they have been told days must be made up by cutting non-tuition activities.”

 

To be honest, we always have the problem of making education more effective. Teachers did not have scapegoats like technology to blame in the past rather than their own course design. It is always not the fault of any educational elements hindering the effectiveness of education. What really matters is how we use them in teaching and learning. To be specific, before deciding on using blended learning, think about how it can support the learning goals, fit specific learning tasks and facilitate the learning process.

 

Conclusions

 

There are no ways for us to go back to traditional teaching with the 2-year influence of online and blended learning during the pandemic. We highly recommend that teachers and course designers be more agentic in blended courses to make the best use of blended learning.

 

 

References

 

Brown, M. G. (2016). Blended instructional practice: A review of the empirical literature on instructors’ adoption and use of online tools in face-to-face teaching. The Internet and Higher Education, 31, 1-10. https://doi.org/10.1016/j.iheduc.2016.05.001

Chiu, T. K. (2021). Digital support for student engagement in blended learning based on self-determination theory. Computers in Human Behavior, 124, 106909.  https://doi.org/10.1016/j.chb.2021.106909

Fisher, R., Perényi, Á., & Birdthistle, N. (2021). The positive relationship between flipped and blended learning and student engagement, performance and satisfaction. Active Learning in Higher Education, 22(2), 97–113. https://doi.org/10.1177/1469787418801702

Graham, C. R. (2006). Blended learning systems. In Curtis J. Bonk, Charles R. Graham (Eds.), The handbook of blended learning: Global perspectives, local designs, 1, pp. 3-21. Wiley Publishers.

Graham, C. R., Allen, S., & Ure, D. (2005). Benefits and challenges of blended learning environments. In M. Khosrow-Pour, Encyclopedia of Information Science and Technology, First Edition (pp. 253-259). IGI Global.

Kaur, M. (2013). Blended learning-its challenges and future. Procedia-social and behavioral sciences, 93, 612-617. https://doi.org/10.1016/j.sbspro.2013.09.248

Harding, M. (2012). Efficacy of supplemental instruction to enhance student success. Teaching and learning in Nursing, 7(1), 27-31. https://doi.org/10.1016/j.teln.2011.07.002

Heinze, A., & Procter, C. T. (2004). Reflections on the use of blended learning. http://usir.salford.ac.uk/id/eprint/1658/1/4247745025H__CP_-_paper9_5.pdf

Medina, L. C. (2018). Blended learning: Deficits and prospects in higher education. Australasian Journal of Educational Technology, 34(1). https://doi.org/10.14742/ajet.3100

Morgan, K. R. (2002). Blended Learning: A Strategic Action Plan for a New Campus. Seminole, FL: University of Central Florida.

Moskal, P., Dziuban, C., & Hartman, J. (2013). Blended learning: A dangerous idea? The Internet and Higher Education, 18, 15-23. https://doi.org/10.1016/j.iheduc.2012.12.001

Rasheed, R. A., Kamsin, A., & Abdullah, N. A. (2020). Challenges in the online component of blended learning: A systematic review. Computers & Education, 144, 103701. https://doi.org/10.1016/j.compedu.2019.103701

Rovai, A. P., & Jordan, H. M. (2004). Blended learning and sense of community: A comparative analysis with traditional and fully online graduate courses. International Review of Research in Open and Distributed Learning, 5(2), 1-13. https://doi.org/10.19173/irrodl.v5i2.192

Joksimović, S., Kovanović, V., Skrypnyk, O., Gašević, D., Dawson, S., & Siemens, G. (2015). The history and state of online learning (pp. 95-131). In G. Siemens, D. Gašević, & S. Dawson, S. (Eds.), Preparing for the digital university: A review of the history and current state of distance, blended, and online learning, pp. 55-92. Retrieved from http://linkresearchlab.org/PreparingDigitalUniversity.pdf

Zhu, M., Berri, S., & Zhang, K. (2021). Effective instructional strategies and technology use in blended learning: A case study. Education and Information Technologies, 26(5), 6143-6161. https://doi.org/10.1007/s10639-021-10544-w

The featured image is from “Does a Broken Screen Mean Your Laptop Is Doomed https://communitycomputerservices.com/does-a-broken-screen-mean-your-laptop-is-doomed/

 

The research team

Linyuan Wang, PhD candidate (ICLON, Leiden University)

Dr. Arjen de Vetten (ICLON, Leiden University)

Prof. Wilfried Admiraal (Oslo Metropolitan University)

Prof. Roeland van der Rijst (ICLON, Leiden University)

 

Share This Post!Share on LinkedInTweet about this on TwitterShare on FacebookEmail this to someone

How are languages learned? A shared question for research at LUCL and ICLON  0

The title of this blog contains a Big Question: “How are languages learned?” I would say this is an intriguing question, considering how quickly children learn the vast amount of knowledge and skills needed for successful communication. They start with crying, giggling, and babbling some sounds, and by the age of three they are talking your heads off. This Big question is one of the fundamental questions of the broad research field of Linguistics. “How are languages learned” is also one of the big questions for practice-oriented research informing foreign language pedagogy, and the learnability question includes another relevant question for both fields: how language learning is the same or different for early versus late learners.

Fundamental questions in linguistics

If we consider the other three fundamental questions in the field of linguistics, we can even conclude that these questions are related to the first, learnability-question, and therefore also relevant for foreign language pedagogy research:

  • How do languages all over the world differ (are languages equally easy or hard to learn?);
  • how do languages change over time (how is language learning leading to language change?);
  • and how is language produced and understood in real time (how do cognitive processes lead to language learning and how are the cognitive processes the same or different for early and late language learners?).

Linguists take different angles when tackling these issues. The question about how languages differ is mainly a structural question, and linguists have described languages at different levels: from the level of sounds to the level of text or discourse organization. How and why languages change is mainly a social question, as languages change through generations, through contact between languages due to migration, and because of people’s need to belong to social groups. The question how languages are processed in real time is cognitive, where linguists investigate how and which cognitive processes are put into action, for instance for the enormous task of listening and speaking with about 5 words per second. The same structural, social, and cognitive viewpoints towards language are also relevant for research into foreign language pedagogy, to answer the more direct practice-oriented question “How to teach language?”.

A unique position for foreign language classroom research

This means that for such practice-oriented research, an enormous amount of relevant research as well as a vast tradition in research methods is available. If we would compare this to research into pedagogy of other school subjects, it becomes clear that this situation for foreign languages as school subjects is unique. Take Biology, for example. It is not the case that the research field of Biology busies itself mostly with the question “How is Biology learned?” Likewise, the main research question of History is not “How do people learn History?”, etcetera.

Practice-oriented research and fundamental research both inform foreign language pedagogy

My conclusion for this blog is that both fundamental and practice-oriented research are needed to better understand and promote language learning and teaching in the classroom. For instance, to investigate successful didactics for teaching verb forms in Spanish, researchers first need to know which verb forms are problematic and why. Or for (research into) teaching and assessing speaking skills, it is first necessary to understand the construct of speaking in a fundamental way. Because foreign language learning research is partly built upon the vast research tradition of Linguistics, I would argue that we can and must make use of this unique situation and interact with researchers who work in linguistics on the more fundamental parts of the big issues to inform practice-oriented research into foreign language teaching. At Leiden University, interaction between the Leiden University Centre for Linguistics (LUCL) at the faculty of humanities and the practice-oriented research at ICLON is already present. I hope we can make the collaboration in the future even stronger, for instance through the Language Learning Resource Centre as well as through working together as teachers on the two-year educational MA for (foreign) languages.

 

Share This Post!Share on LinkedInTweet about this on TwitterShare on FacebookEmail this to someone

Meerdere perspectieven in het klaslokaal. Het balanceren tussen afstand en nabijheid.  0

Welke onderwerpen vinden docenten geschiedenis gevoelig? Enkele jaren geleden is deze vraag aan 81 docenten geschiedenis uit Nederland voorgelegd. Wat speelt in hun lessen? Op welke manier is dat gevoelig, en voor wie?

Veelgenoemde thema’s waren; a) verschillen en conflicten tussen islamitische en niet-islamitische mensen, b) kolonialisme en c) de Tweede Wereldoorlog en Holocaust. Een kwart van de docenten gaf aan geen enkel thema als gevoelig te ervaren. Volgens docenten worden onderwerpen gevoelig doordat leerlingen sterke en tegengestelde meningen in de les naar voren brengen. Daarnaast noemen ze de emotionele betrokkenheid van leerlingen en de angst om leerlingen te kwetsen.  Soms ligt de aanleiding voor gevoeligheid bij de wens van docenten om een onderwerp op een objectieve manier vanuit meerdere perspectieven te bespreken. Dat wordt niet altijd goed ontvangen door leerlingen. Tot slot wordt de onverschilligheid van leerlingen genoemd, bijvoorbeeld rond het thema kolonialisme en oorlog.

Docenten verklaarden voornamelijk dat de sociale en religieuze identificatie van leerlingen een rol speelt bij de gevoeligheid van de onderwerpen. Dit bleek uit uitspraken als ‘kruistochten worden door leerlingen uit het Midden-Oosten gezien als voorafspiegeling van westerse inmenging in het MO’ of ‘leerlingen met een moslim achtergrond voelen zich hier ongemakkelijk bij’.

Wat speelt er rond het thema Islam?

Naar aanleiding van deze uitkomsten waren we van mening dat het thema ‘lesgeven over de Islam’ nader onderzoek verdiende. Er is nog weinig zicht op de complexiteit van ervaringen van docenten met Islam gerelateerde vraagstukken. Daarom zijn er interviews afgenomen met zes docenten geschiedenis met verschillende achtergronden en werkend in verschillende contexten (wel/niet islamitisch). Docenten is gevraagd om te reflecteren op de uitkomsten van de vragenlijst. En er zijn open vragen gesteld over hun omgang met dit onderwerp in de klas en hen is een casus voorgelegd.

Alle docenten herkenden de gevoeligheid rond dit thema maar gaven tegelijkertijd aan dat ze in hun eigen onderwijs geen belemmeringen ervaarden. Docenten benadrukten ook het gevaar van een eenzijdig en een ongedefinieerd gebruik van de termen Islam en Moslims waardoor de diverse percepties rond dit thema onderbelicht blijven.

Drie dimensies

Factoren die een onderwerp gevoelig maken zijn erg afhankelijk van de betrokken personen. Persoonlijke waarden, opvattingen en identiteit van leerlingen en docenten, ingebed in een maatschappelijke context, kunnen zorgen voor complexe situaties in een klaslokaal. Daarbij zijn er veel verschillen tussen individuen en is het afhankelijk van tijd en plaats en context. Om deze complexiteit en de ervaringen van de geïnterviewde docenten te ontrafelen gebruikten we drie dimensies.

De eerste, meest voor de hand liggende, overkoepelende dimensie betreft de interpersoonlijke relatie tussen docent en de leerlingen. Nabijheid op deze dimensie is een belangrijke voorwaarde voor het creëren van een veilig klimaat voor het bespreken van gevoelige onderwerpen. Anderzijds kan er ook afstand worden gecreëerd in de relatie tussen docent en leerlingen als er grenzen en normen bewaakt moeten worden.

De tweede dimensie betreft de mate waarin docenten en leerlingen een gedeelde identiteit ervaren. Maatschappelijke en religieuze identificaties die docenten niet delen met hun leerlingen kunnen ongemakkelijk zijn. Daarnaast moeten docenten afwegen hoeveel ze van hun eigen identiteit delen met leerlingen en in welke mate ze een neutrale positie kunnen behouden. (Nationale) historische verhalen hebben een identiteitsvormende functie. Verhalen kunnen gevoelens van verbondenheid creëren maar ook buitensluiten en afkeer oproepen. Verschillende perspectieven op een historisch narratief kunnen dus botsen met een behoefte aan een gedeeld verhaal, waarin ook een docent niet geheel onafhankelijk kan zijn.

De derde dimensie betreft de mate waarin leerlingen en docenten dezelfde bronnen van kennis gebruiken bij het construeren van bovengenoemde verhalen over het verleden. Vanuit deze dimensie kan de gevoeligheid van een onderwerp geduid worden door verschillen in brongebruik en de manier waarop deze bronnen gebruikt worden. Met betrekking tot de Islam is er bijvoorbeeld weinig of eenzijdige kennis in de lesboeken. Daarnaast kunnen leerlingen en docenten bronnen van kennis verschillend benaderen, wat voor spanningen in de klas kan zorgen. Verschillende omgang met bronnen kan spelen bij complottheorieën, fake news en de spanning tussen wetenschap en religie.

Waar staan de zes docenten?

Over het algemeen bleken de dimensies bruikbaar om de uiteenlopende percepties en ervaringen van docenten in kaart te brengen. We kregen zo een genuanceerder beeld van mogelijke gevoeligheden rond islam-gerelateerde kwesties in verschillende contexten.

Op interpersoonlijk vlak viel op dat docenten nabijheid creëren door persoonlijk contact en betrokkenheid maar ook bewust afstand bewaren zodat leerlingen de kans krijgen om hun eigen meningen te ontwikkelen. Docenten benoemden hierbij ook verschil in context (onder- en bovenbouw).

Uit de analyse bleek dat docenten zich bewust waren van hun verschillende identiteiten en hier flexibel mee konden omgaan en bepaalde aspecten deelden met hun leerlingen. Eén van de docenten kon bijvoorbeeld zijn eigen achtergrond (Moslim, regionale identiteit en migratie) inzetten in de klas. Het onderscheid tussen religieus/niet-religieus en Islamitisch/niet-Islamitisch bleek niet echt aan de orde te zijn bij het typeren van deze dimensie.

Met betrekking tot brongebruik gaven alle docenten aan op een kritische manier met bronnen om te gaan. Drie docenten ervaren dat ze dit delen met hun leerlingen (vooral bovenbouw), terwijl de anderen wat meer afstand ervaarden met betrekking tot het brongebruik van leerlingen. Deze docenten ervaren ongemak door het onkritisch gebruik van (religieuze) bronnen door leerlingen.

De zes docenten zijn op verschillende manieren te positioneren op de dimensies, sterk afhankelijk van het onderwerp en de context waarin ze lesgeven. We denken dat bewust omgaan met deze dimensies docenten kan helpen met het balanceren tussen afstand en nabijheid en hun eigen positie daarin. We denken ook dat de kennisdimensie explicieter aan de orde kan worden gesteld.

Vervolg

In de lerarenopleiding verkennen we verdere mogelijkheden om met deze dimensies docenten in opleiding bewuster te maken van de mogelijkheden om gevoelige onderwerpen bespreekbaar te maken. Vooral op het gebied van brongebruik zijn we aan het ontdekken hoe verschillende bronnen van kennis vanuit verschillende benaderingen kunnen worden bevraagd. Een voorbeeld is een onlangs begonnen project waarbij we docenten geschiedenis en biologie vanuit hun vak perspectieven controversiële thema’s zoals evolutie en gender laten bevragen. Daarover in een volgende blog meer.

Verder lezen:

Savenije, G. M., Wansink, B. G. J., & Logtenberg, A. (2022). Dutch history teachers’ perceptions of teaching the topic of Islam while balancing distance and proximity. Teaching and Teacher Education, 112, [103654]. https://doi.org/10.1016/j.tate.2022.103654

 

Share This Post!Share on LinkedInTweet about this on TwitterShare on FacebookEmail this to someone

Sharing classrooms, sharing worlds: The international CLIL community together again at last  0

“In classrooms on all populated continents of the world, learners are engaged in learning content in a language that they do not speak at home. Depending on where you are, you may know this as Content and Language Integrated Learning (CLIL) or under another name.

Through such approaches, learners develop as language users but also have the opportunity to broaden their horizons and to grow as global citizens. In these uncertain and even tumultuous times, having the knowledge, skills and attitudes to communicate across and within cultures and communities has never been more important.”

So read the text on the homepage of World CLIL 2022: a two-day conference that took place this month, dedicated to bringing together members of the extensive community of researchers, teachers and teacher educators who are engaged in CLIL and other multilingual approaches to teaching and learning throughout the world. The goal: Learning from and with each other in all our diversity, and in celebration of the diversity of our learners.

Based on the overwhelmingly positive response from participants and speakers, this goal seems to have been achieved.

 

Celebrating diversity

The conference theme referred to diversity among colleagues and also among learners. This was the common thread running through the four keynote addresses. Prof. Marisa Pérez-Cañado kicked off with an energising and engaging exploration of questions of “diversity, inclusion, and elitism” in CLIL settings, and of the extent to which old stereotypes still ring true. Prof. Joana Duarte followed this later with a reflection on how an asset-based approach to diversity, coupled with attention for global citizenship and intercultural communication, can help us to truly integrate the content, communication, cognition and cultures present in our classrooms.

On Day 2, Nihayra Leona focused on linguistic diversity, and how much there is to learn from the expertise and experiences of CLIL colleagues working in linguistically rich settings such as the Caribbean. Heading into the final afternoon, Prof. Fred Genesee rounded things off by exploring and dispelling the myth of the incompatibility between bilingualism and special educational needs.

Between these talks, we also saw a diversity of content, be explored in over 100 presentations, symposia, posters and workshops focusing on research, practice and combinations of both.

Together at last

In spite of apprehension earlier this year, when we feared Omicron might put an untimely end to years of planning and preparation, the hybrid conference ultimately boasted nearly 400 attendees, stemming from 44 different countries and six continents. Nearly two-thirds of the delegates attended the event in person, at Leiden University in the Hague. Among the onsite participants, there was a clear sense of excitement and readiness to once again engage in networking and professional learning by coming together in person.

That said, World CLIL would have fallen short of its goals if not for the 144 delegates who attended from afar. Online presenters and audience members attended sessions together with colleagues on location: a challenge to facilitate but well worth the investment. In addition, online delegates were offered the opportunity to meet informally in a ‘Meet&Greet’ networking activity. In the breaks, interviews conducted by ‘roving reporters’ in The Hague offered online participants a flavour of the onsite experience.

A further enrichment were the nineteen colleagues who had received a scholarship due to challenging local or personal circumstances, and were offered free online access. Due to the conference’s limited budget, this would not have been possible had the programme been fully offline.

And next… World CLIL 2024?

Now that we have got the ball rolling, it is time to hand over the baton. Our colleagues in Sheffield, UK, are already working hard on the preparations for their own CLIL 2023 event. After that, perhaps there is room for World CLIL 2024 in Edinburgh, Hong Kong, Willemstad or elsewhere…?

Through regular meetings in diverse locations, I for one hope that the international CLIL community will continue to grow in both its diversity and its unity, as we continue to learn by sharing our classrooms and sharing our worlds.

 

World CLIL was hosted by Leiden University, in collaboration with Nuffic, Amsterdam University of Applied Sciences, Utrecht University and the Dutch Network of Bilingual Schools.

The organising committee: Geerte Holwerda, Lineke Ouwendijk, Franke Schuurmans, Rick de Graaff, Tessa Mearns, Liz Dale, Gerrit Jan Koopman, Helen de Lange. Not pictured: Onno van Wilgenburg.

Special thanks to the rest of the organising committee, to the PhD researchers from ICLON and students from the World Teachers Programme and the HvA who helped with the organisation and moderation, and to all who shared their work at this event. Interested in seeing what was on the programme? It is still available via www.worldclil.com.

 

Photographs by Peter Sansom.

Share This Post!Share on LinkedInTweet about this on TwitterShare on FacebookEmail this to someone