That Test

So much has been written about that test, and the messages are confusing.

Indigenous students’ literacy and numeracy skills have improved at twice the rate of their peers’ over the past decade, new NAPLAN data shows.

Sydney Morning Herald 08/04/2019

… since 2008 there have been statistically significant gains and year levels, particularly at the primary school level

‘Education Matters’ online magazine 10/4/2019

Yet:

National teacher survey gives thumbs down to NAPLAN

Nearly nine out of 10 public school teachers say that NAPLAN is ineffective as a method of assessing students, … nearly seven out of 10 public school principals said NAPLAN is not effective as a measure of school performance, while nearly eight out of 10 public school principals say NAPLAN is not effective for comparing school performance with other schools.

According to the survey, nearly six out of 10 public school teachers say they are spending too much time preparing for, and administering, standardised tests, leading to a reduced focus on other areas of the curriculum.

AEU Report here

If this testing program is apparently indicating such progress, why is it so unpopular with the people who work with actual students?

ACARA

We need to look at what the Australian Curriculum and Assessment Authority (ACARA) says about its own nationally imposed testing program, and the way it reflects the curriculum content that teachers deliver every day.

From the NAPLAN support site:

NAPLAN tests skills in literacy and numeracy that are developed over time through the school curriculum. Excessive test preparation using previous tests is not necessary or useful.

NAPLAN tests identify whether all students have the literacy and numeracy skills that provide the critical foundation for their learning, and for their productive and rewarding participation in the community.

Students are assessed using common national tests in reading, writing, language conventions (spelling, grammar and punctuation) and numeracy.

NAPLAN tests broadly reflect important aspects of literacy and numeracy in the Australian Curriculum adopted by each state or territory.

(A.Gray emphasis)

https://www.nap.edu.au/naplan/the-tests

Let’s consider some of the detail of this statement, in the light of the Australian Curriculum which frames and directs our teaching. We can also consider some of the reasons for calls for major changes to be made to NAPLAN.

Literacy & Numeracy and NAPLAN

Literacy and numeracy are two of the general capabilities of the Australian Curriculum. The concept of capability in this context:

encompasses knowledge, skills, behaviours and dispositions. Students develop capability when they apply knowledge and skills confidently, effectively and appropriately in complex and changing circumstances, in their learning at school and in their lives outside school.

Australian Curriculum: Capabilities

The general capabilities are not subjects. Rather, they inform all subjects, and are addressed through curriculum content. They are about learning and developing, about thinking, and about making choices in the complexities of learning and life.

Literacy skills

The text of the NAPLAN support site states that NAPLAN tests identify …. Perhaps ‘helps to identify’ would be more appropriate. The test contains a finite range of aspects of English language skills isolated by the test developers. It is these that are measured.

However, a single test within a constrained time frame has severe limitations. They provide one kind of evidence only. McKnight & Morgan (2019) see parallels with the medical field, which relies on randomised controlled trials and ignores evidence such as:

practical “on-the-job” expertise; professional knowledge; insights provided by other research such as case studies; intuition; wisdom gained from listening to patient histories and discussions with patients that allow for shared decision-making or negotiation.

In education, what is ignored by the standardisation of NAPLAN includes the evidence of the classroom, teaching expertise, wisdom and knowledge, increasingly recognised collaborative practices such as Instructional Rounds, and the daily teaching and learning negotiations that occur in our schools.

To continue, rather than the literacy and numeracy skills that provide the critical foundation …, it would be more accurate to acknowledge that the test may identify that some students may have only some aspects of literacy and numeracy. Teachers in Australia know these literacy and numeracy skills because they are articulated in all subjects in the Australian Curriculum, and inform our daily practice. They are itemised in the National Literacy and Numeracy Progressions where support provides:

authentic contexts from within the learning area where literacy and numeracy skills are evident in meeting the standard for that particular learning area.

Literacy learning area advice

But this is not what this test measures.

The support goes on … for their productive and rewarding participation in the community. How does this look as we peer into our crystal balls to see the future?

The problem is, while schools and students are focussed on how our NAPLAN scores look as opposed to the school down the road, we miss the opportunity to make deep learning central to what our schools do. We miss the opportunity to focus on non-NAPLAN or ATAR learning. Learning that incidentally we keep hearing is going to be critical and is critical now in our ever-evolving society replete with complex, contradictory and chaotic problems.

Anderson, AARE 04/03/2019

The Australian Curriculum provides support for consolidation of this learning for the future by confirming:

The general capabilities play a significant role in the Australian Curriculum in equipping young Australians to live and work successfully in the twenty-first century.

The general capabilities in the Australian Curriculum are used by teachers to add depth and richness to student learning … in learning area content descriptions and elaborations.

Literacy learning area advice

There is little evidence of even acknowledgement of depth or richness in the NAPLAN test. Certainly, the ‘behaviours and dispositions’ of literacy are not measured in the literacy sections of the test.

Assessment of the general capabilities

The importance of contextualization of the general capabilities in subject content is repeated in the comment on assessment:

Teachers are expected to teach and assess general capabilities to the extent that they are incorporated within learning area content.

Then the door is opened wide into the shadowy world of NAPLAN and its confusion:

State and territory education authorities will determine if and how student learning of the general capabilities is to be further assessed or reported.

Literacy learning area advice

The advice to teachers is to be aware of the imperative for literacy and numeracy, and the five other general capabilities, to be contextualized for teaching, learning, and assessment. They are advised to use their professional knowledge to do so.

Yet the educational standing and progress of our students, our teachers, our schools, and Australia as a nation is measured by a national test that does not appear to value anything other than a mark generated by an indiscriminate test, through that opened door.

What’s wrong with the test

Firstly, the most basic assumption of any test of learning is that it measures what has been taught. This is one of the essential principles of effective assessment in Australian schools, and is articulated in the NSW Educational Standards Authority’s Advice on Assessment.

Another issue is the type of teaching and the time given to test ‘preparation’:

Professor Andy Hargreaves, the president of the International Congress of School Effectiveness and Improvement, cites research showing teachers avoid innovation in the months before standardised tests. “All the deep learning projects were in the years after testing,” he says.

Sydney Morning Herald 12/04/2019

As well, concern has been expressed about the long-term results on overall student achievement:

Director of the Gonski Institute for Education, Professor Adrian Piccoli, said there was little evidence the now 10-year-old NAPLAN had improved education outcomes for students.

Jones (21/3/2019)

Then there is the political agenda we have been subjected to over the past decade, and, as previously mentioned, the growing international recognition of the need for more attention to be given to teacher judgement in the assessment of our students’ progress:

Much of the debate about standardised national testing occurs at a political level, heavily influenced by the national education agencies and the need for data. … in Singapore, the Netherlands, Scotland, Alberta (Canada), and China education policies are shifting towards less-frequent and lower-stakes standardised assessments to give more room for teachers’ professional judgment in assessment and reporting.

NAPLAN Writing: Sample text

This practice test item looks as though it fits into the type of text described as ‘Imaginative’.

https://www.nap.edu.au/_resources/NAP_website_example_writing_prompt_narrative_The_Box.pdf

See Primary Learning Blog Writing with purpose in tests for some detail on supporting students to address the register of text in assessment such as this.

With such a question I expect my students to start with where the curriculum starts –

Imaginative texts – their primary purpose is to entertain through their imaginative use of literary elements. They are recognised for their form, style and artistic or aesthetic value.

Australian curriculum

Where is that in the instructions? No mention of entertainment, of originality, of what we teach using the Australian curriculum and the NSW Syllabus – representation of ideas, feelings and mental images, making new connections, or being characterised by originality, freshness and insight. It seems that there is an unspoken ‘do NOT be original, or represent feelings, or make new connections’.

Surely we can do better than this.

Detailed marking criteria used for the assessment of writing are available on the ACARA NAPLAN website here. They are illuminating, in both the lack of attention given to the purpose of imaginative text, and the subjectivity required by the marker.

What’s wrong

That’s pretty much what is wrong with the literacy test, as with any test that purports to present knowledge as a given, assuming that what the young will need to know in the future is already known. (Strauss 2017). It does not reflect the rich diversity and flexibility encouraged by the curriculum, or the capabilities of our students, or the professional skills of teachers.

Maurie Mulheron, President of the NSW Teachers Federation:

“After a decade of this failed testing regime, we need to critically analyse the impact of standardised testing and we need to implement assessment processes which are intricately linked with teaching and learning in our schools, not just used for system data collection purposes,”

AEU Report here

It … is clear that [a new National Assessment System that includes diagnostic, formative and summative student assessments and the monitoring of school performance] should be based on commonly agreed policy and practice that is evidence-based and is informed by the profession and international research.

Piccoli & Sahlberg 2019 pp 1-2

If we have students who are making progress on what is measured by that test, we need to be able to examine more the whole process of assessment more closely.