Traditionally, the SET is one of the most widely employed instruments in higher education for evaluating the quality of teaching (e.g., Hendry & Dean, 2002; Hounsell, 2003). For a typical SET, after taking a course, students are asked to rate various aspects of the course (e.g., the clarity of the objectives, the usefulness of the materials, the methods of assessment) on a Likert scale. SET data is often the first and foremost source of information that individual teachers can use to evaluate both existing and new innovative teaching practices. SETs are often integrated in higher education professional development activities. For instance, at some faculties at Leiden university at Leiden University, SET results and their interpretation is an integral part of the University Teacher Qualification (a.k.a., BKO). Starting out teachers are expected to critically reflect in teaching portfolios on the results of SETs for the classes they have taught. Furthermore, the results of SETs can function as a source of information for teachers’ supervisors to guide discussions in yearly Performance and Development Interviews, sometimes leading to recommended or enforced future professional development activities for teachers.
However, for some time now, the SET has been subject to scrutiny for a variety of reasons. First, based on an up-to-date meta-analysis, the validity of SETs seems questionable. That is, there appears to be no apparent correlation between SET scores and student learning performance at the end of a course (Uttl, 2017). In fact, when learning performance is operationalized as the added value of a teacher to the later performance of students during subsequent courses (Kornell & Hausman, 2016), the relationship can even be inversed (i.e., teachers with lower SET scores appear to be of more added value). One explanation for this finding is that making a course more difficult and challenging can result in lower SET scores, presenting teachers with a perverse incentive to lower the bar for their students to obtain higher scores on a SET (Stroebe, 2020).
Second, the intensive and frequent use of SETs can lead to a form of “evaluation fatigue” among students (Hounsell, 2003), sometimes resulting in mindless and unreliable evaluations of teaching (e.g., Reynolds, 1977; Uijtdehaage & O’Neal, 2015). As a case in point, a classic article by Reynolds (1977) reported how a vast majority of students in a medical course chose to evaluate a lecture that had been cancelled, as well as a video that was no longer part of the course. In a rather ironic reflection on these results Reynolds concluded that:
“As students become sufficiently skilled in evaluating films and lectures without being there,… …, then there would be no need to wait until the end of the semester to fill out evaluations. They could be completed during the first week of class while the students are still fresh and alert.”
Third, the results of student evaluations of teaching can be severely biased (e.g., Neath, 1996; Heffernan, 2022). For instance, in a somewhat tongue-in-cheek review of the literature, Neath (1996) listed 20 tips for teachers to improve their evaluations without having to improve their actual teaching. The first tip on the list: Be male. Apparently, research suggests that, in general, male teachers receive higher ratings on SETs compared to female teachers. In a more recent review of the literature, Heffernan (2022) goes on to argue that SETs can be subject to racist, sexist and homophobic prejudices, and biased against discipline and subject area. Also, SETs that also allow for a qualitative response can sometimes illicit abusive comments most often directed towards women and teachers from marginalized groups. As such, SETs can be a cause of stress and anxiety for teachers rather than being an actual aid to their development.
Fourth, although studies often emphasize the importance of SETs for supporting and improving the quality of education, the underlying mechanism remains elusive (Harrison et al., 2022). It is unclear how SETs contribute to improving the quality of teaching. To the contrary, teachers can often find it challenging to decide what actual changes to make based on aggregated SET data that is largely quantitative in nature (Henry & Dean, 2010).
In short, the continued use of SETs for evaluating the quality of teaching in higher education is difficult to justify. The findings reported in the literature indicate that the validity and reliability of the SET are questionable, and the value for educational practice appears to be limited. One could argue that sticking with the SET is more a tradition than it is evidence-informed practice. Perhaps universities mostly persist in the routine in lack of an equally (cost-)efficient and scalable alternative. In this blog, we delineate the development and pilot of one possible alternative.
The FET. Late 2023, an Innovation Fund Proposal was awarded a small grant to develop an alternative approach for the evaluation of teaching. At the start of 2024, Mario de Jonge (researcher at ICLON), Boje Moers (project manager at LLInC), Anthea Aerts (educational consultant at LLInC), Erwin Veenstra, and Arian Kiandoost (developers/data analysts, LLInC) collaborated on the development and subsequent small-scale pilot of the FET (Formative Evaluation of Teaching).
The FET is designed to be more conducive for the improvement of teaching practices (formative, qualitative) and less focused on mere assessment of teaching (summative, quantitative). Like the traditional SET, the FET is fast, efficient and relatively inexpensive. However, the FET aims to give teachers clearer directions and qualitative input on how to improve their teaching.
In the first step of the FET survey (Figure 1), students are presented with a list of course aspects on which they can give feedback. Some of the aspects on the list are general (e.g., the methods of assessment), while some of them can be course-specific (e.g., learning objectives). Note that the course aspect pertaining to the teacher specifically asks students to direct feedback on their didactic approach. As noted, students’ evaluations of teaching can sometimes be prone to unconstructive abusive comments. By explicitly asking students to focus on the didactic approach, we hope to discourage these type of undesirable and unconstructive comments.
From the list of aspects, students are asked to select just one or two key aspects which they appreciated (i.e., tops), and one or two key aspects which they think could be improved upon (i.e., tips). With this design feature, we hope counter the threat of evaluation fatigue that is more likely to occur in more comprehensive surveys like the traditional SET that require students to evaluate each and every aspect of a course.
Figure 1. Screenshot of the first step in the FET survey where students are asked to select course aspects to give feedback on.
In the second step (Figure 2), after selecting one or two aspects as tips and tops, students are asked to write a short motivation for their respective selections. This set-up allows students to share their insights in a fast, efficient, and meaningful way.
Figure 2. Screenshot of FET step 2 where students are asked to elaborate on their selections.
After a given course has been evaluated, the FET output provides teachers with a ranking of aspects that were selected most frequently. Because selected aspects have also been enriched with qualitative textual input from students, teachers can engage in a focused review of those student contributions that are most relevant for improving their course (i.e., comments on aspects that were selected most frequently).
Going over the FET evaluation results should be a relatively straightforward task for those who teach small classes. However, for teachers with larger classes we anticipated that this could be a considerable burden. This is where AI comes into play. LLInC developer Erwin Veenstra and data analyst Arian Kiandoost worked on a way of complementing the raw data with an AI-generated summary of the main results. Specifically, we wanted to select a Large Language Model (LLM) that was capable of performing the task of summarizing the data in such a way that it is easy to process and interpret. We expected that, with the current level of sophistication of available LLMs, it should be possible to generate a high-quality descriptive summary of the qualitative data. It took a fair amount of experimentation, iteration, and team discussion about different possible LLMs, output formats, and the “right” prompt before we arrived at a model and approach capable of performing the task.
The LLM we ended up using was OpenAI’s GPT-4 API (OpenAI Platform, n.d.). Note that, in contrast to the non-API consumer service ChatGPT, the OpenAI API does not have the same privacy and security issues. That is, data sent to the OpenAI API is not used to train or improve the model. Still, because we ended up using a cloud-based LLM, the data were first anonymized before feeding it to the LLM. Also, we rearranged the survey data into a JavaScript Object Notation (JSON) format (JSON, n.d.) to make it easier for the LLM to group information per course aspect. The LLM was prompted in such a way that it recognized comments were grouped per course aspect, and that differences in magnitude should also be expressed in the summary (i.e., one Tip versus 10 Tops should not carry the same weight). Furthermore, we prompted the LLM to generate one synthesized integrated summarization for the tips and tops per course aspect. We found that this way of reporting helped to make explicit and nuance apparent contradictions in the data (e.g., half of the students stating one thing, the other half stating the opposite). After the summary was generated, any omissions in the output due to anonymization would be transformed back into the original values in the final report.
In the AI-generated summary, course aspects are presented in a descending order starting with the one that was selected most frequently. For each aspect, a short summary is generated to capture the overall gist of the student comments. Figure 3 shows a screenshot of an AI-generated summary and for one aspect, the working groups, of a course. Note that the summary only gives a descriptive synthesis of the students’ positive and negative, but the actual interpretation is left to the teacher. As is common knowledge, LLMs can sometimes be prone to “hallucinations”. We noticed that prompting the model to also provide an interpretation of the data, beyond what was in the text, increased the occurrence of hallucinations and also decreased the degree of reproducibility of the results. However, a simple more bare-bones LLM-generated descriptive summary provided what we felt was an accurate and reproducible representation of the data. To be sure, we prompted the LLM to supplement each summary with up to six “representative examples” (i.e., actual instances of student feedback) of tips and tops as a reference to the actual data. Furthermore, in the introduction text of to the AI-generated report, we encouraged teachers to cross-check with the actual raw data that was provided along with the summary, in case doubts would arise about the reliability.
Figure 3. Screenshot of an AI-generated summary for one of the course aspects in the FET (shared with permission from the participating teacher).
In the past couple of months, the FET has been piloted in different contexts at Leiden University, ranging from small-group settings such as an elective master class course (+-20 students) to a large-group setting such as a BA course (200+ students). The feedback from the participating teachers has been overwhelmingly positive. All teachers indicated wanting to use the FET again in the future and in their interactions with us, they were able to give multiple concrete examples of changes they intended to make in future iterations of their course. Based on the large BA course, the median time it took students to fill out the survey was around 2 minutes and 40 seconds, a duration we consider not to be too much of a burden for the students. Compared to the regular SET survey from a previous cohort, the FET survey produced much more qualitive student feedback in terms of the total number of student comments. Furthermore, although the average word count per comment that not differ much between the SET and the FET, students filling out the FET clearly put more effort into comments specifically directed at improving the course (i.e. Tips). Most important, after receiving and discussing the report, the participating teacher indicated having a high-degree of confidence in the reliability of the AI-generated summary based on cross-checking with the raw data. In short, the preliminary results of our small scale pilot suggest that the FET can be a valuable tool for efficient collection of high-quality student feedback that is formative and more conducive to the improvement of teaching practices.
Outreach activities (workshops and presentations about the FET project) have now spiked the interest in the FET project within the university. In the next phase, we hope to get further support and funding to scale up the project and see if we can replicate our findings in a broader range of contexts and faculties. Also, for future direction, we aim to use an LLM that can be run on a local server (e.g., Mistral AI, n.d., Meta-Llama, n.d.). To run the larger versions of these kind of models, we need a more powerful computer than the one we had access to during the current project. However, such a machine has recently become available at LLInC.
As the project enters the next phase, we aim to investigate how the FET survey can be successfully implemented to improve educational design and how it can support teachers professional development activities. Furthermore, in our future endeavors we plan to also take into account the student perspective. This was outside the scope of the current project, but it is vital to consider the student perspective if the project is going to move forward and scale up.
Lastly, In the FET we purposefully chose to collect only qualitative data. As already noted abusive comments can sometimes enter into qualitative evaluation data and this can cause stress and anxiety among teachers. However, the qualitative evaluation data from our small-scale pilot did not seem to contain any student comments that could be considered abusive. Perhaps this was due to the design of the FET and the phrasing of the aspects in the list from which students could choose. Or perhaps it was simply due to the fact that students were aware that they were participating in a pilot project. However, even if abusive comments would enter into the FET, we expect that the LLM should be capable of filtering out such unconstructive comments. This is one thing that we would also want to test in the future (e.g., by contaminating evaluation data with a preconstructed set of abusive comments, and training the model to filter the data).
In conclusion, we believe the FET allows teachers to collect valuable feedback on the efficacy of their teaching in a fast, efficient, and meaningful way. Furthermore, the FET holds the potential for enhancing and enriching existing teacher professionalization activities as it can facilitate critical reflection on one’s own teaching practice.
References
Harrison, R., Meyer, L., Rawstorne, P., Razee, H., Chitkara, U., Mears, S., & Balasooriya, C. (2022). Evaluating and enhancing quality in higher education teaching practice: A meta-review. Studies in Higher Education, 47, 80-96.
Heffernan, T. (2022). Sexism, racism, prejudice, and bias: A literature review and synthesis of research surrounding student evaluations of courses and teaching. Assessment & Evaluation in Higher Education, 47, 144-154.
Hendry, G. D., & Dean, S. J. (2002). Accountability, evaluation of teaching and expertise in higher education. International Journal for Academic Development, 7, 75-82.
Hounsell, D. (2003). The evaluation of teaching. In A handbook for teaching and learning in higher education (pp. 188-199). Routledge.
Reynolds, D. V. (1977). Students who haven’t seen a film on sexuality and communication prefer it to a lecture on the history of psychology they haven’t heard: Some implications for the university. Teaching of Psychology, 4, 82–83.
Stroebe, W. (2020). Student evaluations of teaching encourages poor teaching and contributes to grade inflation: A theoretical and empirical analysis. Basic and applied social psychology, 42, 276-294.
Uijtdehaage, S., & O’Neal, C. (2015). A curious case of the phantom professor: mindless teaching evaluations by medical students. Medical Education, 49, 928-932.
Uttl, B., White, C. A., & Gonzalez, D. W. (2017). Meta-analysis of faculty’s teaching effectiveness: Student evaluation of teaching ratings and student learning are not related. Studies in Educational Evaluation, 54, 22-42.
If you’re a researcher, you probably have conducted literature reviews or will do so in the future. Depending on your keywords, a search in online databases easily results in several hundred or even thousands of hits. One of the most time consuming steps is screening all those titles and abstracts to determine which articles may be of interest for your review. Isn’t there a way to speed up this process? Yes, there is!
Recently, we started a literature review about the use of Open Educational Resources (OER) in K12-education. We came across a great tool for article screening. In this blog, we will introduce this tool and share our experiences.
Training the system
Researchers at the University of Utrecht have developed an open source and free screening tool to help researchers go through the enormous digital pile of papers: ASReview LAB (see www.asreview.nl). In a 2-minute introduction video, you can learn how it works. Basically, the programs helps you to systematically review your documents faster than you could ever do on your own by, as they put it, “combining machine learning with your expertise, while giving you full control of the actual decisions”. We just had to try that!
First, we made sure the papers we’ve found all included a title and an abstract as that is what we would use to screen them on relevance. It was very easy to import our RIS file (from Zotero in our case, but can be from any reference management system) with all the hits from our search query. Then it was time to teach ASReview! We provided the system with a selection of relevant and irrelevant articles which it uses to identify potential matches, thus expediting the screening process. Following the guidelines provided in the ASReview documentation, we utilized the default settings of the AI model.
The researcher as the oracle
Once the system was trained, the screening phase could start. At each stage, we evaluated whether a document was relevant or not, providing notes to justify the decisions. In cases of uncertainty, where the abstract alone was not sufficient to make a judgment, we referred to the full text of the article. With each decision, ASReview adapts its learning model to ensure that as many relevant papers are shuffled to the top of the stack. That’s why it is important to make the ‘right’ decision. We worked in the ‘Oracle Mode’ (other modes are possible as well, but for reviews this is the best) which makes the researcher ‘the oracle’. ASReview describes the relevance of taking your time to make decisions: “If you are in doubt about your decision, take your time as you are the oracle. Based on your input, a new model will be trained, and you do not want to confuse the prediction mode.” (ASReview, 2023). So make sure that you carefully formulate your research questions and inclusion criteria before beginning to screen the articles. This helps to decide if an article might be of interest or not.
To avoid endless manual screening (which is kind of the point of using this tool), it was recommended to formulate a stop rule. To formulate our stop rule we made use of the recommendations provided by ASReview and Van de Schoot et al. (2021). According to our rule, screening would cease once at least 33% of the documents were reviewed AND ASReview presented 25 consecutive irrelevant items. This approach helped prevent exhaustive screening while maintaining rigor and reliability. A tip for maintaining focus is to spend a limited amount of time per day screening articles (for example a maximum of two hours a day). Throughout the screening process, ASReview’s dashboard provided a visual overview of progress and decisions made.
In total, 460 items were excluded by the system, while 324 were manually screened, with 173 rejected for various reasons. These reasons ranged from focusing on specific educational technologies to addressing broader educational issues beyond the scope of the study. To ensure the reliability of the screening process, a second researcher independently assessed a random sample of 10% of the documents.
Combining AI and human judgment
After completing the screening process, it is very easy to download a file with an overview of all the decisions made, including both relevant and irrelevant articles. The dashboard and the output files help you in reporting why certain articles were excluded from the review. Notably, the PRISMA model already accommodates for articles excluded through AI. So, in conclusion, ASReview offers a powerful solution for streamlining the literature review process, leveraging AI to expedite screening while maintaining the integrity of the review. It combines the efficiency of AI with human judgment, saving you time – something welcomed by all.
Van de Schoot et al. (2021). An open source machine learning framework for efficient and transparent systematic reviews. Nature Machine Learning, 3, 125-133. https://doi.org/10.1038/s42256-020-00287-7
Higher education institutions have adopted blended learning to actualize the benefits brought by technology. Specifically, the flexibility of learning afforded by blended courses make learning less dependent on specific space and time (Graham, 2006), and instructional method (Harding, 2012). In addition, blended learning is regarded as a more cost-effective approach than fully on-campus teaching (Harding, 2012). In addition to these potential benefits brought by blended learning, blended learning also provides a context in which students need to adapt themselves from the online environment to on-campus learning environments (Chiu, 2021) which at the same time, puts forward new requirements for teaching. Although many factors like teachers’ interaction with technology, academic workload, institutional environment, interactions with students, the instructor’s attitudes, and beliefs about teaching, and opportunities for professional development (Brown, 2016) are identified as factors influencing teachers’ adoption of blended learning, the institutional environment can always stand out in an authentic educational context. Many teachers are encouraged to use blended learning without thinking about whether it is aligned with their teaching goal and usually expect much from blended learning before they go to the classroom.
Potentials of blended learning
Blended learning is a balance of online learning and on-campus learning activities (Siemens, Gašević, & Dawson, 2015). Compared to on-campus and online learning, blended learning (through a Learning Management System) enables students to access learning materials whenever and wherever they want (Moskal, Dziuban, & Hartman, 2013) without the loss of obtaining in-person support and instruction in the classroom (Graham, Allen, & Ure, 2005). Additionally, blended learning can provide more effective instructional strategies to meet the course objectives than only online learning or fully on-campus learning (Graham et al., 2005; Morgan, 2002). This is probably because students can tailor their path of learning based on the personalized options afforded by blended learning (Medina, 2018). It can also improve students’ sense of belonging more than either on-campus or online learning (Rovai & Jordan, 2004). Zhu, Berri, and Zhang (2021) elaborated that, on the one hand, students may feel less disconnected because, with a blended learning situation, students can meet occasionally. On the other hand, they can have interactions and immediate feedback because they are interconnected on the online platform.
Challenges in Blended Learning
However, there are also many challenges in blended learning environments. Blended learning can be problematic if not appropriately designed and utilized. Even if appropriately designed and utilized it does not necessarily lead to the desired goal (Fisher, Perényi, & Birdthistle, 2021). Specifically, finding the optimal “blend” and coordinating all the elements is challenging for instructional designers and teachers of blended learning (Kaur, 2013). If the “blend” is not well-designed students will encounter difficulties in navigating between online learning and on-campus learning. Furthermore, there are technical challenges regarding ensuring student performance and achievement by utilizing and supporting appropriate technologies. In addition, isolation in online learning environments and distraction by non-academic online activities can hinder students’ engagement (Rasheed, Kamsin, & Abdullah, 2020). A crucial element is the amount of guidance and feedback in blended learning compared to on-campus learning due to the reduced in-person time with peers and teachers (Heinze & Procter, 2004). In all, blended learning requires more self-regulation skills and technology literacy than on-campus learning (Rasheed et al., 2020).
Technology as a Scapegoat
Ironically, although both advantages and disadvantages are afforded with the blended model, many stakeholders, like academics, course designers, and teachers, easily blamed technology for low engagement, increasing workload, and the failure of teaching. Specifically, some institutions and teachers were very glad when they heard they were allowed to return to campus. A teacher mentioned that she did not like teaching online with synchronous interaction because she felt that, to a large extent, she lost her control in building emotional bonds with students.
The distrust of technology and blended learning is very disappointing, especially to educational specialists who have invested huge amounts of energy in making blended learning more effective in learning. Joe O’Hara, a full professor of education at Dublin City University (DCU) and the president of the European Educational Research Association, commented “The ‘new normal? Two years of innovation and hard work consigned to the dustbin of (educational) history” under the news of “While some schools may close this week if local conditions are poor, they have been told days must be made up by cutting non-tuition activities.”
To be honest, we always have the problem of making education more effective. Teachers did not have scapegoats like technology to blame in the past rather than their own course design. It is always not the fault of any educational elements hindering the effectiveness of education. What really matters is how we use them in teaching and learning. To be specific, before deciding on using blended learning, think about how it can support the learning goals, fit specific learning tasks and facilitate the learning process.
Conclusions
There are no ways for us to go back to traditional teaching with the 2-year influence of online and blended learning during the pandemic. We highly recommend that teachers and course designers be more agentic in blended courses to make the best use of blended learning.
References
Brown, M. G. (2016). Blended instructional practice: A review of the empirical literature on instructors’ adoption and use of online tools in face-to-face teaching. The Internet and Higher Education, 31, 1-10. https://doi.org/10.1016/j.iheduc.2016.05.001
Chiu, T. K. (2021). Digital support for student engagement in blended learning based on self-determination theory. Computers in Human Behavior, 124, 106909. https://doi.org/10.1016/j.chb.2021.106909
Fisher, R., Perényi, Á., & Birdthistle, N. (2021). The positive relationship between flipped and blended learning and student engagement, performance and satisfaction. Active Learning in Higher Education, 22(2), 97–113. https://doi.org/10.1177/1469787418801702
Graham, C. R. (2006). Blended learning systems. In Curtis J. Bonk, Charles R. Graham (Eds.), The handbook of blended learning: Global perspectives, local designs, 1, pp. 3-21. Wiley Publishers.
Graham, C. R., Allen, S., & Ure, D. (2005). Benefits and challenges of blended learning environments. In M. Khosrow-Pour, Encyclopedia of Information Science and Technology, First Edition (pp. 253-259). IGI Global.
Harding, M. (2012). Efficacy of supplemental instruction to enhance student success. Teaching and learning in Nursing, 7(1), 27-31. https://doi.org/10.1016/j.teln.2011.07.002
Medina, L. C. (2018). Blended learning: Deficits and prospects in higher education. Australasian Journal of Educational Technology, 34(1). https://doi.org/10.14742/ajet.3100
Morgan, K. R. (2002). Blended Learning: A Strategic Action Plan for a New Campus. Seminole, FL: University of Central Florida.
Rasheed, R. A., Kamsin, A., & Abdullah, N. A. (2020). Challenges in the online component of blended learning: A systematic review. Computers & Education, 144, 103701. https://doi.org/10.1016/j.compedu.2019.103701
Rovai, A. P., & Jordan, H. M. (2004). Blended learning and sense of community: A comparative analysis with traditional and fully online graduate courses. International Review of Research in Open and Distributed Learning, 5(2), 1-13. https://doi.org/10.19173/irrodl.v5i2.192
Joksimović, S., Kovanović, V., Skrypnyk, O., Gašević, D., Dawson, S., & Siemens, G. (2015). The history and state of online learning (pp. 95-131). In G. Siemens, D. Gašević, & S. Dawson, S. (Eds.), Preparing for the digital university: A review of the history and current state of distance, blended, and online learning, pp. 55-92. Retrieved from http://linkresearchlab.org/PreparingDigitalUniversity.pdf
Zhu, M., Berri, S., & Zhang, K. (2021). Effective instructional strategies and technology use in blended learning: A case study. Education and Information Technologies, 26(5), 6143-6161. https://doi.org/10.1007/s10639-021-10544-w
Not all students enter university with the same economic, social and cultural capital. Therefore, access, inclusiveness and well-being for all are key in developments in higher education across the world. Higher Education Institutions (HEIs) are part of the broader social tissue and not just places where students acquire academic skills; they also help students become more resilient in the face of adversity and feel more connected with the people around them. Not least, HEIs are the first place where students experience society in all its facets, and those experiences can have a profound influence on students’ attitudes and behavior in life.
Importance of students’ sense of belonging
As higher education becomes increasingly competitive, students come under more pressure to succeed in their grades, which increases their levels of stress. Stress has been linked to mental health problems, which are highly prevalent among the student population and have been shown to impact learning and well-being (Stallman & King, 2016). A number of factors can affect student retention and well-being, including the student’s social experience within the higher education environment. Students’ sense of belonging to their institutions – personal feelings of connectedness to the institution occurring in academic and social spheres – has come to be recognized as one of the most significant factors in students’ success and retention in higher education. While individual characteristics such as personality and propensity to connect may have some impact, it is also acknowledged that institutional factors play an important role. Elements such as the culture of the university or curriculum design may affect the students’ experiences, including their sense of belonging and connection to other students, staff and the institution (Kahu & Nelson, 2018).
COVID-19 pandemic
The lockdowns in response to COVID-19 pandemic have interrupted conventional schooling and in many countries online teaching is now a new routine for many students in higher education, but it presents significant challenges. Many students experience challenges with respect to keeping a sense of belonging to their peers, staff and institution. Students in the most marginalized groups, who don’t have access to digital learning resources or lack the resilience and engagement to learn on their own, are at risk of falling behind. Universities from around the world have been uncertain about how long the COVID-19 crisis will last and how it might affect the mental health of students and faculty.
What to do?
Students have to cope with many challenges, both inside and outside HEIs, which can have immense consequences varying from poor access, low engagement and feelings of distress to delays and drop-out. Yet these challenges and consequences can be different depending on students’ social, cultural, economic and language backgrounds leading to a discrepancy between inclusive access and inclusive outcomes. But how should HEIs take up this massive challenge of access, inclusiveness and well-being for all? For sure not the umpteenth study on how students experience higher education in COVID-19 times. More attention for connecting students, socializing activities, and embodying social settings; less lecturing, testing and calls to account. Let’s stay connected to take up this massive challenge!
The last two decades have been characterized by extensive growth in the use of technology in education, such as the application of virtual learning environments, simulation software, games and gamification, virtual experiments, visualization of complex models as well as tools that enable students and teachers to communicate and collaborate through email, electronic forums, and instant-messaging systems. Due to the measures to prevent the spread of COVID-19 schools around the world radically changed to teaching and learning at a distance. What can we learn from 20 years of research on technology-enhanced teaching? Are we exploring new practices or just consolidating our insights into teaching and learning? I present ten implications for distance teaching from home, based on a large body of knowledge on technology-enhanced teaching from pre-COVID-19 times.
Explore new territories instead of consolidating land
Despite extensive growth in the use of technology in education, innovations in teaching with technology have entered the school sporadically: many teachers use the technology to do what they always have done and choose those activities that will help them accommodate their own perspectives on teaching and learning.
Transform distance teaching by exploring new pedagogies instead of transferring pedagogies from pre-COVID-19 times.
Focus on the learner, not on teaching
Students differ in their learning needs, preferences and motivation. Yet schools should provide a place for all students no matter their social, cultural, economic, language, and ability background. It is important to understand students’ needs for autonomy and support to align teaching with what students need.
Mentor student learning, instead of transferring knowledge online.
Share control of learning activities with students
In online learning, with too much learner control, many students feel lost and do not know how to regulate their own learning path; with too much teacher control, many students are not motivated to their do work. Shared control means that students have the autonomy to decide their pace, sequencing, time allotment, practicing and reviewing within a larger framework set by the teacher.
The more intrinsically motivated students are, the more learner control can be established.
Teach students how to learn
Self-regulation appears to be a crucial aspect in online learning, but many students have difficulties with regulating their own learning activities. The development of self-regulation has not commonly been addressed. And if so, teachers are focused on how to prepare learning activities (planning and making choices) and less on self-regulation during and after learning.
Teach and provide feedback on how students can monitor, review and redirect their own learning processes.
Insert reflection
Adaptive software has been regularly used to support students in practicing with, for example, language and math skills. Yet learning does not happen so much through the extensive practice, but because students compare and refine information, and surface, criticize, restructure and test intuitive understanding. Students give ‘meaning’ to what they learn. In other words:
Encourage students’ reflection on what they have done and achieved.
Strengthen active learning
Students should actively construct knowledge by integrating new information and experiences into what they have previously come to understand, revising and reinterpreting old knowledge in order to reconcile it with the new.
Acknowledge distance teaching as means of knowledge construction and discovery, rather than as means of passive acceptance of knowledge transfer.
Situate learning in real-life world of students
Learning occurs only when students process new information in a meaningful way that makes sense within their own frames of reference. Interested students challenge their existing knowledge and are more likely to develop conceptual frameworks that integrate prior knowledge and new information into understanding. In distance education from home, the link with the real-life world is even more apparent.
Connect to students and situate learning in their real-life world
Encourage social interaction
Constructing meaning comes from interacting with others – teachers, peers, friend, parents, family and casual acquaintances- to explain, defend, discuss, and assess ideas and challenge, question, and comprehend the ideas of others. Social interaction is a critical component of situated learning –students become involved in a learning community that embodies certain beliefs and behaviors to be acquired.
Encourage social interaction to construct meaning and bind students to a learning community
Constructively align teaching, assessment, materials and tools
Teaching, learning activities, assessments, materials and tools should be constructively aligned to reach the learning goals that are set. The more these are aligned, the more students target their learning activities to reach the learning goals.
Teaching, materials and tools should be aligned to support targeted learning activities of students.
Share with colleagues
Many evaluations of technology-enhanced teaching in school show that the more interventions are school-wide approaches, the more shared by teachers and the more effective these are. Students are similarly approached by all teachers and in all school subjects, which strengthens learning effects. Additionally, more effort will be put into the educational design of teaching approaches and practices.
Share your practices with colleagues and the more effective yours will be!
Het notaoverleg van de vaste commissie voor Onderwijs, Cultuur en Wetenschap over de Strategische Agenda Hoger Onderwijs heeft veel stof doen opwaaien in de sociale media. Zo is er een voorstel van VVD-kamerlid Dennis Wiersma voor digitalisering van colleges. Zo veel mogelijk colleges zouden voor iedereen gratis online beschikbaar moeten komen. Aankomende studenten zouden dan kunnen beoordelen of een bepaalde studie voor hen geschikt is, digitaal onderwijs zou makkelijk kunnen worden gecombineerd met een baan of zorgtaken en mensen hebben zo altijd toegang tot kennis en kunnen bijblijven in hun vak. Vervolgens heeft vooral de opmerking dat het hoger onderwijs dan ook wel wat goedkoper kan, veel reacties opgeroepen. De Minister lijkt overigens niet erg enthousiast te zijn over dit plan, mede omdat zij inschat dat het juist meer geld kost.
Problemen bij online onderwijs
Maar waar de discussie eigenlijk over moet gaan is of digitalisering van hoger onderwijs daadwerkelijk de toegankelijkheid tot kennis verbetert. Er moet een onderscheid worden gemaakt tussen onderwijs(aanbod) en leren. Uit de inmiddels omvangrijke kennisbasis over open online hoger onderwijs weten we dat studenten erg verschillen in de reden waarom zij het onderwijs volgen en deze redenen veranderen ook nog eens gedurende een cursus. Ook verschillen zij in hun voorkennis over het onderwerp, hebben zij verschillende voorkeuren om zich kennis en vaardigheden eigen te maken, en denken dat zij verschillend over wat een student en een docent zou moeten doen in onderwijs. En al deze verschillen zijn niet bekend als het open online onderwijs wordt gemaakt en uitgevoerd.
Zoveel mogelijk aanbod
De oplossing voor al deze verschillen tussen studenten in open online onderwijs is zoveel mogelijk aanbod klaar zetten waaruit iedere student dan een keuze kan maken. Maar onderzoek heeft al aangetoond dat:
veel studenten in open online hoger onderwijs deze benodigde zelfregulatievaardigheden om goed hun weg in het aanbod te vinden onvoldoende bezitten;
het geboden onderwijs vooral gericht is op kennisoverdracht door de docent, iets waarvan we al veel langer weten dit niet effectief is, en
studenten weinig actief bezig zijn met de inhoud van het onderwijs, iets waarvan we al langer weten dat het juist wel werkt.
Dus…
Dit leidt ertoe dat in open online hoger onderwijs goede studenten het meeste leren en de minder goede studenten afhaken of uitvallen. De digitalisering van onderwijs vergroot weliswaar de toegankelijkheid van alles wat er is, maar verkleint de toegankelijkheid van het daadwerkelijk verkrijgen van meer kennis en vaardigheden. Een verschil tussen onderwijs(aanbod) en leren.
Literatuur
Hendriks, R. A., de Jong, P. G. M., Admiraal, W. F., Reinders, M. E. J. (2019). Teaching modes and social-epistemological dimensions in Medical Massive Open Online Courses: Lessons for integration in campus education. Medical Teacher, 41(8), 917-926.
Hendriks, R. A, Jong, P. G. M., Admiraal, W. F., & Reinders, M. E. J. (2020). Instructional design quality in medical Massive Open Online Courses for integration into campus education. Medical Teacher, 42(2), 156-163.
Jansen, R. (2019). Dealing with autonomy. Self-regulated learning in open online education. Dissertatie. Universiteit Utrecht.
Pilli, O., Admiraal, W., & Salli, A. (2018). MOOCs: Innovation or stagnation? Turkish Online Journal of Distance Education, 19(3), 169-181.
Docenten geven les met tablets en smartphones zoals zij altijd al deden. Afgelopen weekend bezocht ik de jaarlijkse IADIS mobile learning conference in Lissabon (http://mlearning-conf.org/). Een kleine onderzoeksconferentie met als focus mobiele technologie die het leren en onderwijzen ondersteunt. Veel presentaties zoals ruim 10 jaar geleden: mooie ict-projecten, opgezet door onderzoekers en ontwerpers, die vooral buiten het reguliere curriculum plaatsvinden. Veel betere techniek dan 10 jaar geleden, dat wel. Draadloos internet, tablets en smartphones zijn niet meer weg te denken uit de maatschappij, de school en het klasklokaal. Maar allemaal niet als onderdeel van de reguliere lespraktijk van docenten.
Doorbraak
Dat hoopte we met het onderzoek in het kader vaan Doorbraak ICT en onderwijs te doorbreken (https://leerling2020.nl/landelijk-onderzoek). In dit project hebben docenten uit het primair en voortgezet onderwijs experimentjes uitgevoerd in hun eigen lespraktijk om met ict gepersonaliseerd leren van leerlingen te faciliteren. Resultaten van dit onderzoek heb ik op de IADIS gepresenteerd. Maar wat wil het geval: overall gezien zien we van de interventies weinig of geen effecten op de prestaties, de motivatie en zelfregulering van leerlingen in het voortgezet onderwijs. Kort door de bocht:
docenten passen hun experimentjes aan het rooster, curriculum en structuur waarin zij (behoren te) functioneren en doen dus wat ze altijd al deden, maar nu met mobiele technologie en
het mobiele karakter van de ingezette smartphones, tablets en laptops wordt niet benut. Geen omgevingsonderwijs; leerlingen blijven in de klas en op school, op hun vaste plek. Het boek en de reader zijn vervangen door een tablet en de digitale leeromgeving.
Docentprofessionalisering?
Om dit te veranderen wordt vaak geroepen dat we meer moeten investeren in de professionele ontwikkeling van docenten. Eerlijk gezegd is dat ook een belangrijke suggestie die wij in het onderzoeksrapport hebben opgenomen. Maar het is de vraag of dit gaat helpen. En valt de docent wel wat te verwijten? Docenten passen hun projecten aan aan de reguliere methode en systematiek omdat zij hierop worden aangesproken. Er moet voldoende contacttijd zijn en alle geplande leerstof moet worden behandeld. Bovendien hebben docenten beperkt tijd hebben om andere dingen te doen dan lesgeven; niet-lestijd gaat op aan voor- en nawerk, administratieve klussen en overleggen met je collega’s.
Het roer moet om
Willen we een doorbraak bereiken in onderwijs moet het systeem om: meer ruimte (tijd, veiligheid en kunde) om onderwijs te vernieuwen, met ict of op andere manieren. Het roer moet om. Als wij kunnen aantonen in meer dan 40 interventies met meer dan 6000 leerlingen uit ruim 30 scholen voor voortgezet onderwijs dat het overall weinig uitmaakte of en hoe docenten gepersonaliseerd leren met ict in hun onderwijs inzetten, is het tijd voor actie! En dat is niet het afschuiven op de kwaliteit van docenten. Goed gebruik van de ict die nu beschikbaar is en moderne ideeën over hoe je leerprocessen van alle leerlingen kunt ondersteunen vereisen een grotere ingreep in het systeem:
Weg met onderwijs in kleine schoolvakken, maar onderwijs in grotere vakdomeinen en multidisciplinaire thema’s
Weg met individueel lesgeven, maar team teaching om ook ruimte te geven voor experimenten en leren van elkaar
Weg met het roosteren van al het onderwijs in contacturen, maar ruimte voor projectonderwijs, in en buiten de school, in de maatschappij en bedrijven
Geef docenten en leerlingen meer ruimte om onderwijs in te richten zoals zij dat willen.
Please, feel free to add comments and questions to the posts by clicking on the title of the post. At the bottom you can put your comments.
Please share the post in your network by clicking on the icon (-s) below the posts.
made use of the recommendations provided by ASReview and Van de Schoot et al. (2021). According to our rule, screening would cease once at least 33% of the documents were reviewed AND ASReview presented 25 consecutive irrelevant items. This approach helped prevent exhaustive screening while maintaining rigor and reliability. A tip for maintaining focus is to spend a limited amount of time per day screening articles (for example a maximum of two hours a day). Throughout the screening process, ASReview’s dashboard provided a visual overview of progress and decisions made.
Recent Comments