Language Arts Canadian Lead Primary School

Assessment: Reliability and Validity

One year of input should equal one year of progress, for all students, no matter where they start.”

John Hattie 2016 (Full article here)

The first question I ask any school I work with is, “What does your assessment battery look like?” Without valid and reliable testing, teachers and school systems are not going to know who is really making progress and who isn’t. Without that information, the following sub-optimum situations occur:

  • Children who are struggling with reading and writing go unnoticed too long.
  • Children who performed poorly in unreliable or non-valid tests get stuck at “levels” too low for their actual ability.
  • These children then, are not given access to an age-appropriate curriculum while they catch up and so never catch up.
  • Children are mis-labelled as “lazy” or “lacking in confidence” while underlying foundational skills are not addressed.

The two key definitions in any assessment are:

  • validity and
  • reliability.

Validity: “Validity refers to the degree to which evidence and theory support the interpretations of test scores entailed by proposed uses of tests.”

American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999) Standards for educational and psychological testing. Washington, DC: American Educational Research Association.

In other words, does your test measure what it’s supposed to be measuring? A good non-example of a valid test is the use of Running Records.

If, as stated in The Simple View of Reading[1], there are two separate processes involved in reading, i.e. word-level decoding multiplied by linguistic comprehension, a test that mixes the two while decoding is still developing, risks invalid results.

Reliability: Reliability refers to how dependably or consistently a test measures a characteristic.

In other words, if the test were taken again, or if the test were conducted by a different person, would the scores still be the same? Once again, Running Records fall very short on this aspect of testing. 

There are several free language arts assessments available online, as well as many superb paid ones.

A superlative grassroots movement of teachers wanting the science of reading and learning to increase in schools has come up with an assessment scope and sequence, including links to all their recommended products on their facebook page: Reading Science in Schools. My advice is to join that page and avail yourself of that and other treasures!

In Language Arts, being able to identify what a child struggles with leads to targeted, effective intervention. Knowing a child’s working memory and attentional capacity alongside their transcription and text generation ability, helps predict response to intervention.

[1] Coming next module!

Optional further study:

The video below shows a webinar done to outline the assessments used here at Lifelong Literacy.

Assessment and Screening Tools (57:20)

To help with revising school assessment batteries, we have also linked to a pdf by the Reading Science in Schools Facebook group outlining a proposed assessment scope and sequence from Pre-Foundation to Y6.

You can view this document here.

2 thoughts on “Assessment: Reliability and Validity”

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.