Results
Research Summary: The Reading Tutor Program (RTP) delivered in MultiLit Literacy Centres
Parents of children who attend tutoring at one of MultiLit’s two Literacy Centre locations will already understand how closely each child’s reading progress is monitored. Students are assessed thoroughly upon entry into the program and then again after each term of instruction. This individual information is fed back to parents, alongside more frequent feedback of results from progress monitoring measures. Recently, a larger-scale investigation of The Reading Tutor Program (RTP) delivered in the two MultiLit Literacy Centres was conducted by the MultiLit Research Unit, in order to evaluate the program’s efficacy.
The study included students new to the Literacy Centre, who were in Years 3 through 6 and who attended between the years of 2015 and 2019. In total, 164 students (61 females, 103 males) from both Bella Vista and Macquarie Park locations who were included in the analyses. At program commencement, the average age of the students was 9 years, 6 months. Assessments were conducted at the start of the program and again at the end of their second instructional term in the program, after approximately 6 months.
These assessment measures included:
- Word reading accuracy (Burt Word Reading Test; Gilmore et al., 1981);
- Spelling accuracy (South Australian Spelling Test; Westwood, 2005);
- Phonological decoding (i.e., nonword reading accuracy; Martin and Pratt Nonword Reading Test; Martin & Pratt, 2001);
– a Passage reading accuracy (Neale Analysis of Reading Ability – 3rd, Neale 1999);
– a Passage reading comprehension (Neale Analysis of Reading Ability – 3rd), and;
– b Passage reading fluency (Wheldall Assessment of Reading Passages; Wheldall & Madelaine, 2013).
The majority of students (80%) attended for two sessions per week, and most (61%) attended in person, rather than via online lessons (28%) or a combination of the two formats (11%). Each session was approximately 50 minutes in duration.
Results from students with other diagnoses or concurrent speech pathology intervention were not excluded from analyses. However, to isolate skill gains associated solely with weekly RTP sessions, results from those who attended a school holiday program in their first two instructional terms were excluded.
Student Achievements
Did reading skills improve over the two terms of instruction?
Based on analyses of raw scores, statistically significant improvements (p < .001) were seen on all measures (see Table 1). The gains were associated with large effect sizes (Cohen’s d < 0.96). Reading (and spelling) skills did therefore improve substantially over the two terms of instruction.
Table 1. Raw score means (and standard deviations) for pre- and post-test time points on all literacy skills measured.
Note: When interpreting Cohen’s d effect sizes, a small (S) effect is 0.2; a medium (M) effect is 0.5; and a large (L) effect is 0.8. Where data were non-normally distributed, a Wilcoxon Signed-Rank Test was performed to confirm the statistical significance of parametric t-test results.
Did reading skills improve beyond what might be expected of same-aged peers, on average?
The average gain for all students over six months would be expected to be six months. But, on average, this group made crude ‘age equivalent’ gains that far exceeded the average six months between testing time points:
– 13 months in word reading accuracy;
– 18 months in phonological decoding (i.e., nonword reading accuracy);
– 13 months in spelling;
– 13 months in passage reading accuracy, and;
– 11 months in passage reading comprehension.
Passage reading fluency results also indicated that, at the end of the instructional period, students read 28 (38%) more words correctly per minute.
The largest increase in ‘age equivalent’ scores was seen for nonword reading accuracy. On this measure, the standard score increase from pre-test (86.80) to post-test (94.89) was also significant (p < .001) and associated with a large effect size (Cohen’s d = 1.02). Hence, the improvement was far beyond what would be expected for age-related change.
Reading nonwords (e.g., ‘shribe’) accurately is not an important goal in itself; however, performance on this measure represents the ability to decode unfamiliar words using knowledge of letter-sound relationships. It is therefore an important foundational skill contributing to broader reading comprehension.
The Martin and Pratt Nonword Reading Test (hereafter ‘Martin & Pratt’) has Australian normative data available, thereby allowing us to further compare our results with those of same-aged children. As can be seen in Figure 1, the majority of students who entered the RTP program (at ‘pre-test’) were in the bottom quartile (i.e., performing poorer than 75% of their same-aged peers) for nonword reading. By the end of two instructional terms (at ‘post-test’), many of these students moved into the middle or ‘average’ range, and there was even a small proportion of students performing in the upper quartile.
Figure 1. Quartile distributions on phonological decoding measure.
Note: Pre-test median percentile = 18 (mean = 22.67, SD = 15.35); post-test median percentile = 37 (mean = 38.80, SD = 22.03).
Based on what can be inferred from crude reading ages and percentile distribution shifts, RTP students receiving tutoring in the Literacy Centres made more progress than their similarly aged peers. Although many were not at the level of their peers by post-test, it must be kept in mind that these students were referred for tutoring intervention because of their low starting points and levels of progress in the classroom. In that context, the accelerated gains are very pleasing to see.
Was there a difference between online and face-to-face delivery formats?
Of the total 164 students, 146 were included in a follow-up analysis to determine whether there were any differences in the results that could be attributed to the program delivery mode. The remaining 18 students received instruction via a combination of formats, and so were excluded. The results indicated that there were no differences in the gains achieved by students who were tutored via online lessons, versus those tutored via face-to-face lessons.
Conclusion
On the whole, the results from this most recent set of analyses demonstrate the effectiveness of the RTP program, as delivered in the MultiLit Literacy Centre. Additionally, similar improvements were found for students attending lessons in our centres and those attending lessons via online tutoring. This means Literacy Centre parents can be confident that RTP is associated with good reading progress, regardless of the mode by which it is delivered.
a n = 124 for these measures.
b n = 158 for this measure.
We at MultiLit are proud of the great gains made and we congratulate our students on their fine performance.
— Kevin Wheldall and Sarah Arakelian
Contact us for more information
and answer any questions you might have.