In the standardisation, classes were randomly allocated to sit the reading and maths tests on either paper or computer. The content of the tests was the same; only the mode of delivery was different i.e. paper or computer. After the standardisation, we looked at average performance on both modes, across all class levels and test forms. On average, pupils sometimes performed slightly better on the paper version of the test, and sometimes slightly better on the computer-based version. In general, these differences were small. The exception to this was Level 2 (usually Second class), where the computer-based tests were more difficult, on average, than the paper tests. For this reason, we decided not to release the Level 2 tests on computer for now.
For Levels 3 to 6, the paper and computer-based tests were normed separately. This means, for example, that if a pupil takes Form 3A of the New Drumcondra Primary Reading Test (DPRT) on computer, their standard score is based on a comparison with the pupils in the standardisation sample who also took Form 3A on computer. This makes it a like-with-like comparison.
Yes, standard scores, STen scores and percentile ranks can be compared across test modes. Raw scores (number of correct answers) should not be compared directly. The reasons for this can be explained by using an example.
Example: Lena and Matthew
Lena and Matthew are friends who go to different schools. At the end of the last school year, Lena’s class sat the New DPRT on paper and Matthew’s class took it on computer. They both sat the same form of the test – Level 4, Form A.
Both Lena and Matthew achieved a standard score of 110. This corresponds to a STen score of 7 and percentile rank of 75. Lena’s standard score of 110 is based on comparing her performance on the test to those pupils in the standardisation sample who also took Form 4A of the New DPRT on paper. Matthew’s score of 110 is based on a comparison with the pupils in the standardisation who took Form 4A on computer. The percentile rank of 75 means that both Lena and Matthew performed as well as, or better than, 75% of pupils who took the same form of the test as them on the same mode.
To achieve a standard score of 110, Lena answered 61 question correctly out of 80 on the paper test. However, to achieve the same standard score, Matthew answered 60 questions correctly on the computer-based test. This shows why raw scores should not be compared directly. The same raw score can lead to slightly different standard scores and vice versa. This is due to small differences in difficulty (as explained above) and also to differences in how scores were distributed among the samples of pupils in the standardisation.
The new Drumcondra primary tests of English reading and mathematics were normed in spring/summer 2018 only, and, as a result, there are no autumn norms. This follows on from Circular 0056/2011, which implemented changes to standardised testing requirements, as part of the Literacy and Numeracy Strategy. This circular states that standardised tests should be completed by all pupils in 2nd, 4th and 6th class during May/June (with the usual allowances for exemptions).
Circular 0056/2011 can be accessed here: https://www.education.ie/en/Circulars-and-Forms/Active-Circulars/cl0056_2011.pdf
Pupils in 2nd, 4th and 6th class must sit standardised tests in May/June, with aggregated results subsequently reported to the Department of Education and Skills (DES). However, schools may wish to administer the tests to other class levels near the start of the school year. In this case, the class level in question should sit the test one Level below e.g. pupils at the start of 3rd class should take the Level 2 test. For maths in particular, this will ensure that pupils have covered all of the content of the assessment.
Previous standardisations that were carried out at both times of the year (spring and autumn) indicate that the differences between the two time points are reasonably small. Pupils taking the tests in autumn tend to perform slightly better than those taking the same Level in the spring. This is to be expected given the slight differences in age and curriculum coverage. Results on the overall scales between time points are usually within a few standard score points, and within one STen score of each other. For autumn administration of the new (2018) tests, it should be noted on the results of each class that the tests were taken in autumn rather than spring, and that pupils took a test one Level below their grade. Results should be interpreted with this in mind.
In addition, if pupils take the same Level of the test within the same calendar year (i.e. spring followed by autumn), it is advisable for pupils to receive a different form of the test on each occasion to reduce practice effects.
In line with the 1999 Primary School Mathematics Curriculum (PSMC) pupils can use calculators for some questions in the New DPMT for Levels 4, 5 and 6 (usually 4th to 6th class).
For these Levels, pupils complete Part A of the test without a calculator. This part of the test includes any questions that directly assess pupils’ ability to carry out operations and procedures relevant to their class level. There are also a number of questions in this part of the test that are ‘calculator neutral’ e.g. reasoning questions without computation. Pupils can use calculators to complete Part B of the test. In this part, the focus is not on the ability to carry out computations by hand, but rather on integrating, reasoning and problem-solving skills.
While pupils are allowed to use calculators for Part B, it is not a requirement. That is, if pupils at certain class levels in your school are not accustomed to using calculators, they can still complete the test without them. A small number of schools took this approach in the standardisation. However, please note that the test must still be completed within the time limits outlined in the administration manual.
Yes, the New DPMT is available in Irish for all class levels. Following standardisation, the tests were professionally translated and were made available for sale to Irish-medium schools at the same time as the English language versions. The administration manual is also available in Irish.
A review of the translations is currently underway to ensure the language used is as child-friendly as possible. Resulting changes, if any, will not be in place for testing in 2020 but will be implemented for testing in 2021.
Levels 3 to 6
At Levels 3 to 6, pupils work through the full test independently. However, as for Part B of Level 2, teachers may read any words for pupils in any part of the test. Again, pupils are notified of this as part of the administration script. The restriction on defining mathematical terms also applies here. Examples of relevant terms that feature at these levels include ‘parallel’, ‘area’, ‘average’ and ’prime number’.
The test administration manuals indicate that you may exempt pupils who have a physical or intellectual/learning disability that would prevent them from engaging with the test in a meaningful way, or who have insufficient experience (generally, less than one year) of instruction through English or Irish.
In general, however, we advise that pupils be included whenever possible. You may decide that it is possible to include a pupil in testing only if special accommodations are made. This can be done provided that:
- You note the accommodation next to the pupil’s result;
- You interpret results cautiously, as the test has been taken outside of standardisation conditions;
- If the pupil is in 2nd, 4th or 6th class, you mark them as exempted on returns to Esinet (i.e., you do not include their result in this report).
Accommodations could include, but are not limited to:
- a child taking the test at a lower level than their class level;
- a reader for the mathematics test;
- taking the test in a quiet room;
- answering a paper test in the Level 3 – 6 range using the test booklet rather than an answer sheet;
- extra time for comfort breaks;
- taking the test with the support of a Special Needs Assistant.
Additional guidance about exemptions and special accommodations is available from the DES here.
This document states that the decision to exempt a particular pupil lies with the school principal, taking into account the pupil’s needs, abilities and best interests.