Evaluation of the Digital Learning Framework (DLF)

The Educational Research Centre has been asked by the Department of Education ‎and skills to carry out an evaluation of the Digital Learning Framework (DLF). The new ‎framework, for primary and post-primary schools, was published in September 2017. ‎The framework is a key part of the Digital Strategy for Schools 2015-2020. ‎The evaluation uses a longitudinal design, and runs from 2018-2022. It was preceded by a field trial phase which ran from 2017-2018.

Project team: Jude Cosgrove, Emmet Feerick and Eva Moran

DLF longitudinal evaluation (2018-2022)

The ERC’s evaluation of the implementation of the Digital Learning Framework involves collecting information from schools at baseline, and subsequently following the progress of representative samples of primary, post-primary and special schools with surveys of school staff and focus groups with school staff and students, over multiple waves from 2019-2021.

Baseline phase: During DLF seminars provided by the PDST (and NCSE in the case of special schools) (October 2018-April 2019), participants completed surveys designed by the ERC. In all, 1,534 primary schools, 327 post-primary schools and 64 special schools took part. Survey responses have been weighted to provide nationally representative population estimates.

Report on baseline phase

Baseline survey (PDF)

Press release

Longitudinal phase: Nationally representative samples of primary (150), post-primary (100) and special (30) schools are followed longitudinally from 2019 to 2021 through surveys of school staff, and focus groups of staff and students in a small subset of these schools. The surveys will focus on progress and change. They also seek to move beyond the notion of ‘how often’ digital devices are used in teaching, learning and assessment contexts towards a better understanding of how digital technologies are being used to enhance or support teaching, learning and assessment. Another aim of the longitudinal phase is the development and validation of a multi-faceted School Index of Digital Technologies in Teaching, Learning and Assessment, which may have applications beyond the immediate context of reporting on the DLF.

Longitudinal phase reporting plans: Reporting on Wave 1 of the longitudinal data is scheduled for Q4 2020. Wave 2 reporting is planned for Q2 2022. A final synthesis report, to include recommendations, is planned for publication in Q3 2022.

Complementary reporting plans: Other large scale assessments provide information that complements the DLF evaluation. Specifically, the Programme for International Student Assessment (PISA), which examines reading, mathematics and science achievement, also collects detailed information on 15-year-old students’ access to, usage, and views on digital technologies. A report drawing on the PISA 2018 data is planned for publication in Q1 2021. Also, the forthcoming National Assessment of Mathematics and English Reading (NAMER) 2021, will provide data on digital technologies from primary school children, and these will be analysed and reported on during 2022.

PDF copies of the longitudinal survey (Phase 1) (P1)

P1 Primary and Special schools – English

Digital Learning Team Leader (or Principal) Questionnaire

Teacher Questionnaire

P1 Primary and Special schools – Gaeilge

Digital Learning Team Leader (or Principal) Questionnaire

Teacher Questionnaire

P1 Post-primary – English

Digital Learning Team Leader (or Principal) Questionnaire

Teacher Questionnaire

P1 Post-primary – Gaeilge

Digital Learning Team Leader (or Principal) Questionnaire

Teacher Questionnaire

More information on the Digital Learning Framework is available from the ‎Department’s website and the PDST website. There is also a website for schools, developed by the PDST, at DLplanning.ie.

Contact:‎ Emmet Feerick, Eva Moran or Jude Cosgrove
DLF@erc.ie

DLF trial evaluation (2017-2018)

Prior to the longitudinal evaluation, a trial of the framework in 20 post-primary and 30 primary schools ran from ‎October 2017-May 2018.‎

The Educational Research Centre was responsible for developing and hosting ‎questionnaires for school principals, teachers and PDST staff; analysing the results; ‎and drafting a report on the evaluation. ‎

In addition, focus groups in six of the participating schools were carried out. ‎

The evaluation report was submitted to the Department of Education and Skills in ‎July 2018.‎

Information on the evaluation can be found in this presentation.

Contact:‎
DLF@erc.ie

Final report on DLF trial evaluation

The final report on the Digital Learning Framework trial evaluation was published on October 22, 2018. The report describes the progress, successes and challenges experienced by staff in the 48 primary and post-primary schools that took part. It follows on from the baseline report, published in May 2018. It identifies seven implications for the national implementation of the Digital Learning Framework, which is being led by the PDST technology in education team. Jude Cosgrove and Alice Duggan presented the key findings of this report to participating school staff at the Digital Learning Framework Trial Shared Learning Day in Croke Park on October 22.

Read the executive summary here

Read the full report here

Baseline report on DLF trial evaluation

The Baseline report on the DLF trial evaluation can be downloaded here.

The report presents a starting context for interpreting the overall findings and progress made by participating schools over the course of the trial. The overall findings will be published as a final report during the third quarter of 2018. Sincere thanks to school staff, teachers, and PDST advisors for their participation.​

PDF copies of questionnaires for participating schools and PDST advisors

Note that all responses should be submitted using the online versions of the questionnaires. Links to the online questionnaires are sent via email to participants. The PDF versions below are for review purposes only.

Phase 2 (April-May 2018)

Post-primary teachers

Post-primary principals/Digital Learning Team leaders

Primary teachers

Primary principals/Digital Learning Team leaders

PDST advisors

Phase 1 (November-December 2017)

Primary schools:

Principal questionnaire

PDST first visit questionnaire for principals

Teacher questionnaire

Post-primary schools:

Principal questionnaire

PDST first visit questionnaire for principals

Teacher questionnaire

PDST advisors:

PDST first visit questionnaire for PDST advisors