Author: Aidan Clerkin

Mode effects in the transition to digital testing: Evidence from TIMSS in Ireland, Azerbaijan, Bahrain, and Flanders

Published in: Large-scale Assessments in Education

Abstract:

Background

About half of the countries participating in TIMSS 2019 transitioned from paper-based to digital assessment in that cycle, with most remaining countries completing the transition in TIMSS 2023. All countries transitioning in 2019 completed a paper-based ‘bridge’ study to examine mode effects (differences in achievement related to the mode of administration) and ensure the comparability of scores from previous cycles. However, only four countries opted to administer a national comparison (bridge) study in 2023: Ireland, Azerbaijan, Bahrain, and Flanders (Belgium). This Brief Report documents findings from these four national comparison studies. The aim is to verify the extent to which the transition to digital testing may have affected student performance in these countries, noting that ten countries in TIMSS 2019 were previously found to have displayed country-level mode effects in at least one subject or grade.

Findings

We find no evidence of significant differences in overall performance by mode of administration, in either mathematics or science, in any of the four included countries. We also find no evidence of significant mode effects among boys or girls in Ireland, Flanders, or Bahrain. In Azerbaijan, boys achieved a significantly higher science score on the digital version of TIMSS compared to the paper version. Some differences in achievement by mode are observed at the level of the mathematics and science content and cognitive domains. These are limited and present no distinctive pattern in three of the countries studied although, again, more consistent differences are noted in Azerbaijan. Finally, some preliminary evidence is presented of differences in student performance on the digital assessment related to students’ access to a computer at home in Flanders, Bahrain, and Azerbaijan.

Conclusions

The findings provided here confirm that achievement estimates in TIMSS 2023 in these four countries were substantively unaffected at a high level by the change in the mode of administration. This should strengthen confidence in the use of these data for policymaking, educational practice, and further research. In particular, trend data from 2019 to 2023 in these countries can be more confidently interpreted as reflecting substantive trends or changes in achievement, giving researchers greater confidence that trends have not been significantly confounded by the procedural change of assessment mode. Evidence of some specific differences in each country could be considered further by national experts in curriculum and assessment. In particular, several differences have been observed that merit further consideration by researchers and policymakers in Azerbaijan.

 

Citation: Clerkin, A., Verhelst, D., Mahdi, A., Mammadov, S & McHugh, G. (2026). Mode effects in the transition to digital testing: Evidence from TIMSS in Ireland, Azerbaijan, Bahrain, and Flanders. Large-scale Assessments in Education, 14, 4. https://doi.org/10.1186/s40536-026-00313-x

Reflections on the National Literacy and Numeracy Strategy 2011-2020: Evidence from TIMSS and PIRLS (2011-2023)

Published in: Irish Journal of Education

Abstract:

The National Strategy to Improve Literacy and Numeracy Among Children and Young People 2011-2020 (hereinafter referred to as the 2011 Strategy) was one of the flagship policy initiatives of the Department of Education and Youth for much of the last 15 years. Amongst other aspects, it set out significant goals relating to initial teacher education, instructional time, parental involvement, assessment practices and use of standardised tests, use of large-scale assessment data for system-level monitoring, and major curriculum reform. The publication of a successor Literacy, Numeracy and Digital Literacy Strategy in 2024 – covering the next decade up to 2033 – creates a follow on from the 2011 Strategy. It therefore seems timely now to look back over the 2011-2023 period to examine the extent to which progress towards a set of aims specified in the 2011 Strategy had been accomplished by its endpoint.

 

For this analysis, we draw on data from multiple cycles of two large-scale assessments: PIRLS 2011, TIMSS 2011, PIRLS 2021, and TIMSS 2023. These studies, which are conducted at Fourth Class in primary school, fulfil three key criteria for the current purpose: (i) they provide population-level data that allow generalisations to be made to the relevant population of pupils across the country as a whole, rather than just a sample; (ii) they provide comparable datapoints on key topics at either side of the lifetime of the 2011 Strategy; and (iii) they provide data relevant to the teaching and learning of both literacy and numeracy. Findings are presented under the six “pillars” alluded to in the 2011 Strategy and a subsequent 2017 Interim Review. Next, the findings are contextualised from an international perspective, highlighting common drivers for policy creation, approaches used when policymaking, and challenges that arise when trying to evaluate the effectiveness of policy. The findings and contextualisation inform a discussion on the changes and progress in the lifetime of the 2011 Strategy while considering the updated goals and objectives set out in the current Literacy, Numeracy and Digital Literacy Strategy 2024-2033.

 

McHugh, G., Clerkin, A., & Hastedt, D. (2026). Reflections on the National Literacy and Numeracy Strategy 2011-2020: Evidence from TIMSS and PIRLS (2011-2023). Irish Journal of Education, 50(4), 1-49. https://doi.org/10.70092/IJE.50.04

 

Six strategies for bridging the transition to post-16 chemistry: Ireland’s Transition Year model.

Published in: Education in Chemistry

Abstract:

Article on science education in Ireland’s Transition Year alongside contributions from science teachers in Northern Ireland, England, Scotland and Wales.

Clerkin, A. (2026). Six strategies for bridging the transition to post-16 chemistry: Ireland’s Transition Year model. Education in Chemistry. https://edu.rsc.org/ideas/how-to-manage-the-transition-from-pre-to-post-16-chemistry/4022624.article

Learning from employers’ perspectives on Transition Year work experience

Published in: Irish Educational Studies

Abstract:

Work experience is regarded as a vital component of Transition Year (TY) by students, parents, and teachers. Despite this centrality, the manner in which placements are organised has evolved in an ad hoc manner, and limited attention has been paid to the perspectives of hosting organisations. This study explores the experiences and views of 11 providers of TY placements. Situating TY work experience within its historical and policy context and drawing on research on work-based learning and career development, the paper examines providers’ motivations, expectations, and challenges. While organisations expressed strong commitment to supporting young people’s early engagement with the world of work, the findings reveal significant ambiguities concerning the educational purpose of placements and the nature of school-employer relationships. The study argues that stronger structures, clearer shared understandings, and more systematic engagement between schools and providers are needed for the full potential of TY work experience to be realised.

Clerkin, A., & Jeffers, G. (2026). Learning from employers’ perspectives on transition year work experience. Irish Educational Studies, 1–31.

https://doi.org/10.1080/03323315.2026.2675305

Assessing inconsistent responding in digital and paper formats: A replication study using the TIMSS 2023 Confidence in Mathematics scale in Ireland

Published in: Frontiers in Psychology

Abstract:

This study replicates and extends previous research on inconsistent responding in mixed-worded scales. Using a posttest-only experimental design, we examine the prevalence of inconsistent responding on the TIMSS 2023 Confidence in Mathematics Scale by comparing randomly equivalent digital and paper questionnaire administrations in Ireland. Two detection methods are employed: Mean Absolute Difference and Factor Mixture Analysis. Results indicate slightly higher rates of inconsistent responding in digital than in paper administration formats across most comparisons. Also, fourth-grade students exhibited higher rates of inconsistent responding compared to eighth-grade students. The study further investigates the role of student characteristics in predicting inconsistent responding. Among student background and digital-specific variables considered in binary logistic regression models, mathematics achievement was the only systematic negative predictor of inconsistent responding, explaining most of the variance, especially in younger students. This study highlights the importance of considering administration mode in large-scale assessments and provides implications for survey methodology, especially as digital testing becomes more widespread.

Konstantinidou, E., Pitsia, V., Clerkin, A., & Michaelides, M. P. (2026). Assessing inconsistent responding in digital and paper formats: a replication study using the TIMSS 2023 Confidence in Mathematics scale in Ireland. Frontiers in Psychology, 17, 1817706. https://doi.org/10.3389/fpsyg.2026.1817706

Mathematics in context: students’ attitudes towards mathematics and their experiences of the mathematics classroom in PISA 2022

This report describes students’ experiences of mathematics in Ireland, their feelings towards the subject, and also trends in mathematics performance, both overall and across different content areas and cognitive processes. Drawing on data from the 2022 cycle of Programme for International Student Assessment (PISA), collected from 15-year-old students and their school principals, various aspects related to the teaching and learning of mathematics are examined and compared with international averages for the OECD and the EU. The composition of the mathematics classroom, and the role of digital technologies, are examined along with the disciplinary climate in the mathematics classroom. Student perceptions of various teaching behaviours and their own learning strategies are explored. The report goes on to look at students’ familiarity with and exposure to various aspects of mathematics, their attitudes towards mathematics and the extent of confidence and anxiety associated with mathematics, concluding with a discussion of the key themes that emerge from the data.

The e-Appendix to this report is available: PISA 2022 Mathematics in context E-appendix.

Infographic

The home learning environment of students in Ireland: New findings from TIMSS 2023

This report focuses on the home learning environment of Fourth Class and Second Year students in Ireland using data from TIMSS (Trends in International Mathematics and Science Study) 2023. It describes findings about students’ socioeconomic status, the availability of selected home resources for learning and home possessions, digital resources at home and use of digital resources for learning, parental education and occupation, home language, early childhood learning and skills, and the attitudes and expectations towards education of parents and students. These data are examined at a nationally representative level and for key subgroups of students: by gender, socioeconomic status, and school DEIS (Delivering Equality of Opportunity in Schools) status. Implications for policymakers in the Irish national context are discussed.

Infographic

 

International Gender Differences in Civic Knowledge and Attitudes: Implications and Ways Forward

Published in: IEA Compass: Briefs in Education

Abstract:

This Compass Brief examines gender differences in students’ civic knowledge and attitudes toward globally important social topics. It uses data from IEA’s International Civic and Citizenship Education Study (ICCS) 2022, which assessed the civic knowledge of eighth-grade students across 22 countries. The findings from ICCS 2022 indicate a notable gender gap in civic knowledge in most participating countries, with boys scoring lower on average than girls in all such cases. Students exhibited strong positive attitudes toward gender equality, although boys were significantly less likely than girls on average to express support for it. The disparity in support for gender equality was larger than for any other indicator. In most countries, and on average internationally, boys were significantly less likely than girls to express positive attitudes about environmental protection and concern about global environmental threats. In the event of national emergencies in democratic systems, boys in half of the participating countries were significantly more likely than girls to accept restrictions, such as prohibiting large gatherings, imposing travel restrictions, or identifying people whose behavior might put others at risk. These gender differences in attitudes present challenges for building more democratic, equitable, and sustainable societies worldwide.

Clerkin, A., Welmond, M., & Gregory, L. (2025). International Gender Differences in Civic Knowledge and Attitudes: Implications and Ways Forward. IEA Compass: Briefs in Education #28. IEA. https://www.iea.nl/sites/default/files/2025-10/CB28%20ICCS%20Gender%20Differences.pdf

Continuity and change in Ireland’s schools and classrooms: TIMSS 2011–2023

This report provides a comprehensive analysis of developments in schools, classrooms, and the teaching and learning of mathematics and science at primary and post-primary levels in Ireland between 2011 and 2023. Drawing on data from the Trends in International Mathematics and Science Study (TIMSS), collected from students, teachers, and school principals, the report examines trends in school composition, resources, and learning environments. It also explores the organisation and delivery of mathematics and science instruction, assessment practices, instructional challenges, and the integration of digital devices in teaching and learning. These findings are contextualised within national education policies and initiatives, highlighting how Ireland’s education system has adapted to changing pedagogical, technological, and societal expectations. By tracing patterns of continuity and change, the report provides valuable insights into the evolving educational landscape and provides a robust evidence base to inform future policy, practice, and evaluation in mathematics and science education.

e-Appendices associated with this publication: Chapter 2, Chapter 3, Chapter 4, Chapter 5, Chapter 6

 

Infographic

Exploring randomised controlled trials in education: key features and selected examples from Ireland.

Published in: Irish Educational Studies 2025

Abstract:

Randomised controlled trials (RCTs) are the gold standard for evaluating educational interventions. However, Ireland is approximately 20 years behind the US and 10 years behind the UK in implementing randomised controlled trials in education. This paper examines the role of RCTs in advancing educational research in Ireland, building on a recent commentary by Gilleece and Clerkin (2024. “Towards More Robust Evaluation of Policies and Programmes in Education: Identifying Challenges in Evaluating DEIS and Reading Recovery.” Irish Educational Studies). It explores the fundamental characteristics of RCTs, including randomisation, the use of control groups, blinding (or masking), sampling, power and analytical considerations. Additionally, the paper discusses other key aspects such as process evaluations, economic evaluations, ethical issues, and the importance of transparency in conducting RCTs. Selected examples of RCTs conducted in Ireland are presented to make the discussion directly relevant to the Irish context. RCTs are an essential tool for evidence-informed educational policy and practice and, where appropriate and feasible, can be used to address key national issues such as the Literacy, Numeracy, and Digital Literacy Strategy (2024-2033) implementation plan and the use of AI in education. We hope that this paper will serve as a practical guide for policymakers, researchers, and educators and will stimulate further discussion on RCTs in Irish education.

Slattery, E. J., Clerkin, A., Gilleece, L., Butler, D., Marshall, K., & Hedges, L. V. (2025). Exploring randomised controlled trials in education: key features and selected examples from Ireland. Irish Educational Studies, 1-26. https://doi.org/10.1080/03323315.2025.2523301

 

Trends in students’ school experiences and attitudes towards mathematics and science: TIMSS 2015–2023

This report describes the school experiences and attitudes of Fourth Class and Second Year students in Ireland. Drawing on data from the 2023 cycle of the Trends in International Mathematics and Science Study (TIMSS), it delves into the experiences of students in relation to their attendance at school, sense of belonging, and experiences of bullying. Also discussed in the report are students’ attitudes towards mathematics and science; in particular, the extent to which students like learning, feel confident, and value mathematics and science. A key feature of this report is that it examines trends in students’ experiences and attitudes across the three most recent cycles of TIMSS (2015, 2019 and 2023). The findings point towards some noteworthy changes in Irish students’ experiences and attitudes over time. Finally, some conclusions and implications for policymakers in Ireland are outlined.

Infographic

Promoting participation and engagement: Ireland’s experiences across three decades of international large-scale assessments

Published in: Frontiers in Education

Abstract:

In large-scale assessments, the collection of high-quality representative data depends in part on (i) securing high participation rates and (ii) ensuring that participants demonstrate a sufficient level of engagement. This article explores the challenges of promoting participation and engagement with reference to Ireland’s experiences throughout multiple cycles of international and national large-scale assessments. Some factors likely to have influenced participation and/or engagement in the Irish context include: (i) the rising profile of large-scale assessments due to their prominence within a national strategy to improve literacy and numeracy; (ii) the transition of studies from paper-based to digital administration; (iii) the publication of new data protection legislation (both European and national); and (iv) pressures resulting from the COVID-19 pandemic. Initiatives implemented by Ireland’s national study centre with the aim of promoting participation in and engagement with large-scale assessments can be broadly classified as relating to consultation, promotion, and support. Lessons from Ireland’s experiences are discussed in relation to future avenues of investigation that may increase our understanding of facilitators and barriers to engagement. Ireland’s experiences offer valuable lessons that could inform practices in other countries.

Delaney, E., Clerkin, A., Karakolidis, A., Gilleece, L., & Millar, D. (2025). Promoting participation and engagement: Ireland’s experiences across three decades of international large-scale assessments [Perspective]. Frontiers in Education, Volume 10 – 2025. https://doi.org/10.3389/feduc.2025.1565557

Environmental knowledge and attitudes in Ireland’s schools: Findings from TIMSS 2023

This report presents findings for Ireland for two aspects of students’ environmental awareness that were captured for the first time in the 2023 cycle of TIMSS (Trends in International Mathematics and Science Study). These are (i) students’ environmental knowledge – a subscale of the science assessment, drawing on selected items that are related to environmental and climate issues – and (ii) students’ attitudes towards environmental preservation. The report presents data for Fourth Class (primary) and Second Year (post-primary) students, and also describes environmentally relevant practices and attitudes of their parents, teachers, and principals, as recorded via questionnaires. The findings, which are novel in an Irish context, are discussed with reference to the broader national policy context of environmental awareness in schools.

Infographic

Reducing educational disadvantage across post-primary schools: Evidence from the first decade of the DEIS plan in Ireland

Published in: Studies in Educational Evaluation 2025

Abstract:

Equity in education is a policy priority internationally with policies and programmes designed to reduce gaps between more and less ‘advantaged’ learners. Evidence is mixed about the effectiveness of these, although limited use has been made of experimental or quasi-experimental designs in evaluation. Ireland’s main policy response to disadvantage since 2005 is the Delivering Equality of Opportunity in Schools (DEIS) programme. This study uses a regression discontinuity design (RDD) to examine the effect of DEIS on outcomes for post-primary schools between 2007 and 2016. Using national-level data across this period, we find no impact on school-level outcomes including examination performance and retention rates. Furthermore, we find no effect of DEIS on the percentages of low or high achieving students in key subjects. These null effects are found despite widely-held beliefs that DEIS has had a positive effect. Findings highlight the importance for evidence-informed policy of methods that allow for causal interpretation.

Gilleece, L., Clerkin, A., & Flannery, D. (2025). Reducing educational disadvantage across post-primary schools: Evidence from the first decade of the DEIS plan in Ireland. Studies in Educational Evaluation, 86, 101466. https://doi.org/https://doi.org/10.1016/j.stueduc.2025.101466

How Large-Scale Assessments Have Informed Education Policy in Ireland

Published in: IEA Compass: Briefs in Education

Abstract:

In recent decades, large-scale assessments in Ireland have revealed consistent trends, phases of change, and some surprises. This Compass Brief explores how data from large-scale assessments informed Ireland’s National Literacy and Numeracy Strategy 2011–2020, particularly regarding priorities, target-setting, and monitoring. It also examines how related initiatives including curriculum redevelopment and standardized testing reform have reflected large-scale assessment findings. Looking ahead, we outline how large-scale assessments can inform Ireland’s next National Strategy to 2033, which encompasses digital literacy along with literacy and numeracy. More broadly, using Ireland as an example, we identify important issues for policymakers in other countries to consider when using ILSA (international large-scale assessment) data within their own national contexts.

Clerkin, A., & Delaney, E. (2025). How Large-scale Assessments Have Informed Education Policy in Ireland. IEA Compass: Briefs in Education #26. IEA. https://www.iea.nl/sites/default/files/2025-01/CB26-LSA-Data-Informing-Policy-Ireland.pdf

Computer-based testing in Ireland, 2005-2024: Challenges, lessons learned, and future possibilities.

Published in: E-testing and computer-based assessment: CIDREE Yearbook 2024

Abstract:

This chapter traces the use of computer-based testing in schools in Ireland in two contexts: international large-scale assessments (ILSAs) (since 2005) and national standardised testing (since 2016). Learning gleaned from Ireland’s participation in computer-based ILSAs informed the development of a bespoke online platform for administering and scoring standardised tests (the ERC Drumcondra Online Testing System [ERC DOTS]). Conversely, knowledge about schools’ use of ERC DOTS has informed Irish approaches in subsequent ILSA cycles. Across ILSAs and standardised testing, recurrent challenges of computer-based testing are identified, including wide variation in: (i) education technology infrastructure in schools, and (ii) students’ prior experiences using this for schoolwork. Although common to primary and post-primary settings, infrastructural challenges were most apparent at primary level while differences in students’ experience were most pronounced below Grade 3. Adaptations and solutions trialled are discussed. Looking ahead, further possibilities of computer-based testing, including enhanced accessibility, additional item types, and adaptive testing, are considered.

Delaney, E., O’Flaherty, A., Perkins, R., Clerkin, A., & Cunningham, R. (2024). Computer-based testing in Ireland, 2005-2024: Challenges, lessons learned, and future possibilities. In B. Ranđelović et al. (eds), E-testing and computer-based assessment: CIDREE Yearbook 2024. CIDREE. https://www.cidree.org/wp-content/uploads/2024/11/cidree_yearbook-2024.pdf

TIMSS 2023: Insights into mathematics and science achievement in Ireland

This report presents the achievement results for Ireland in the Trends in International Mathematics and Science Study (TIMSS) 2023. TIMSS assesses the mathematical and scientific knowledge and skills of students in Fourth Grade and Eighth Grade (Fourth Class and Second Year in Ireland). TIMSS is a study of the International Association for the Evaluation of Educational Achievement (IEA) and has been conducted every four years since 1995. TIMSS 2023 is the eighth cycle of the study and included 656,360 students from 65 countries.

This national report focuses on trends in overall achievement in Ireland, as well as the distribution of achievement (performance at various percentiles), student performance at the TIMSS International Benchmarks, relative strengths and weaknesses on the TIMSS content and cognitive domains, and teachers’ reports of curriculum coverage. Performance in Ireland is explored in the context of selected countries; compared with previous cycles of TIMSS where data are available (since 2011 for Fourth Class and since 2015 for Second Year); and, where applicable, examined by gender and by socioeconomic status at both individual level and at school level (DEIS status).

TIMSS 2023 also marks Ireland’s transition from paper-based to digital testing, as part of which Ireland completed a national mode effect study to examine the extent to which any differences in student achievement exist between digital and paper testing. Initial findings from this mode effect study are also presented.

Finally, some conclusions and implications for policymakers in Ireland are outlined.

e-Appendix associated with this publication – download

Infographics

Digital Learning Framework (DLF) national longitudinal evaluation: Wave 2 Final Report.

This is the final report on the Digital Learning Framework national longitudinal evaluation, following data collection for Wave 2 from Autumn 2021-May 2022. The data presented in this report provide an opportunity to consider how schools and students responded to the challenges of the COVID-19 pandemic with reference to the use of digital technologies for teaching, learning and assessment. The evaluation also considers the ways in which schools have had successes and met challenges in their embedding and implementation of the Department of Education’s Digital learning Framework. The report provides an overview of the current policy and research landscape with respect to the use of digital technologies in schools and over the course of the pandemic. It also presents new data at both primary and post-primary levels from schools (digital learning team leaders, school principals, and teachers), students, and professional development advisors specialising in the use of technology in education. The key findings from Wave 2 of the evaluation are highlighted, together with examination of changes from previous waves. Finally, the main successes and challenges encountered by schools in their implementation of the Digital Learning Framework over the period 2017-2022 are identified.

Final report of the longitudinal Digital Learning Framework Evaluation

This is the final report on the Digital Learning Framework national longitudinal evaluation, following data collection for Wave 2 from Autumn 2021-May 2022. The data presented in this report provide an opportunity to consider how schools and students responded to the challenges of the COVID-19 pandemic with reference to the use of digital technologies for teaching, learning and assessment. The evaluation also considers the ways in which schools have had successes and met challenges in their embedding and implementation of the Department of Education’s Digital learning Framework. The report provides an overview of the current policy and research landscape with respect to the use of digital technologies in schools and over the course of the pandemic. It also presents new data at both primary and post-primary levels from schools (digital learning team leaders, school principals, and teachers), students, and professional development advisors specialising in the use of technology in education. The key findings from Wave 2 of the evaluation are highlighted, together with examination of changes from previous waves. Finally, the main successes and challenges encountered by schools in their implementation of the Digital Learning Framework over the period 2017-2022 are identified

An in-depth analysis of the relative strengths and weaknesses of students in Ireland in mathematics and science in TIMSS 2019

This report is aimed at informing the work of teachers, school leaders, teacher educators, and curriculum developers in Ireland. It describes in detail the performance of Fourth Class (primary) and Second Year (post-primary) students on a range of mathematics and science topics and test items. In interpreting the data presented here from the TIMSS 2019 study (the Trends in International Mathematics and Science Study), students’ performance is discussed with reference to relevant aspects of the Irish curricula for mathematics and science, and to international comparisons of student performance.

The findings presented here build on the initial national report on TIMSS 2019 (Perkins & Clerkin, 2020), which described student performance in mathematics and science in terms of average achievement, distributions of achievement among higher- and lower-achieving students, gender differences, benchmarks of performance (Low, Intermediate, High, or Advanced), and high-level
descriptions of performance on the TIMSS cognitive and content domains. This report expands on the latter aspect by providing a detailed examination of Fourth Class and Second Year students’ performance on the TIMSS content domains and, in particular, on their constituent subdomains and topic areas.

Associated e-appendices

 

Flyers

Strengths and Weaknesses in Maths and Science at Fourth Class

Strengths and Weaknesses in Maths and Science at Second Year

The ERC publishes a periodic newsletter, eolas ón ERC, to keep stakeholders updated on research publications, ongoing projects in schools and standardised testing.