Did reading skills improve over the two terms of instruction?
Based on analyses of raw scores, statistically significant improvements (p < .001) were seen on all measures (see Table 1). The gains were associated with large effect sizes (Cohen’s d < 0.96). Reading (and spelling) skills did therefore improve substantially over the two terms of instruction.
Table 1. Raw score means (and standard deviations) for pre- and post-test time points on all literacy skills measured.
Note: When interpreting Cohen’s d effect sizes, a small (S) effect is 0.2; a medium (M) effect is 0.5; and a large (L) effect is 0.8. Where data were non-normally distributed, a Wilcoxon Signed-Rank Test was performed to confirm the statistical significance of parametric t-test results.
Did reading skills improve beyond what might be expected of same-aged peers, on average?
The average gain for all students over six months would be expected to be six months. But, on average, this group made crude ‘age equivalent’ gains that far exceeded the average six months between testing time points:
- 13 months in word reading accuracy;
- 18 months in phonological decoding (i.e., nonword reading accuracy);
- 13 months in spelling;
- 13 months in passage reading accuracy, and;
- 11 months in passage reading comprehension.
Passage reading fluency results also indicated that, at the end of the instructional period, students read 28 (38%) more words correctly per minute.
The largest increase in ‘age equivalent’ scores was seen for nonword reading accuracy. On this measure, the standard score increase from pre-test (86.80) to post-test (94.89) was also significant (p < .001) and associated with a large effect size (Cohen’s d = 1.02). Hence, the improvement was far beyond what would be expected for age-related change.
Reading nonwords (e.g., ‘shribe’) accurately is not an important goal in itself; however, performance on this measure represents the ability to decode unfamiliar words using knowledge of letter-sound relationships. It is therefore an important foundational skill contributing to broader reading comprehension.
The Martin and Pratt Nonword Reading Test (hereafter ‘Martin & Pratt’) has Australian normative data available, thereby allowing us to further compare our results with those of same-aged children. As can be seen in Figure 1, the majority of students who entered the RTP program (at ‘pre-test’) were in the bottom quartile (i.e., performing poorer than 75% of their same-aged peers) for nonword reading. By the end of two instructional terms (at ‘post-test’), many of these students moved into the middle or ‘average’ range, and there was even a small proportion of students performing in the upper quartile.
Figure 1. Quartile distributions on phonological decoding measure.
Note: Pre-test median percentile = 18 (mean = 22.67, SD = 15.35); post-test median percentile = 37 (mean = 38.80, SD = 22.03).
Based on what can be inferred from crude reading ages and percentile distribution shifts, RTP students receiving tutoring in the Literacy Centres made more progress than their similarly aged peers. Although many were not at the level of their peers by post-test, it must be kept in mind that these students were referred for tutoring intervention because of their low starting points and levels of progress in the classroom. In that context, the accelerated gains are very pleasing to see.
Was there a difference between online and face-to-face delivery formats?
Of the total 164 students, 146 were included in a follow-up analysis to determine whether there were any differences in the results that could be attributed to the program delivery mode. The remaining 18 students received instruction via a combination of formats, and so were excluded. The results indicated that there were no differences in the gains achieved by students who were tutored via online lessons, versus those tutored via face-to-face lessons.
On the whole, the results from this most recent set of analyses demonstrate the effectiveness of the RTP program, as delivered in the MultiLit Literacy Centre. Additionally, similar improvements were found for students attending lessons in our centres and those attending lessons via online tutoring. This means Literacy Centre parents can be confident that RTP is associated with good reading progress, regardless of the mode by which it is delivered.
a n = 124 for these measures.
b n = 158 for this measure.
Contact us for more information
We’ll walk you through the best options for your child and answer any questions you might have.