Did the Census work?
The Census 2018 review is out, and it’s pretty clear things didn’t go so well. In fact, the head of Stats NZ Liz MacPherson resigned because of it. The review showed there were problems with the approach and it has meant the response rate was significantly low.
Polly Atatoa Carr is an Associate Professor at the University of Waikato, working in Population Studies. Lillian Hanly spoke with Atatoa Carr to find out more about the response rates and the future of censuses in New Zealand.
You can find the full interview here and the written transcript below. What follows is a write up from Angus Coker Grant.
Censuses are the massive undertaking of taking record of a country’s citizens, in order to plan for future needs, such as health, education, and infrastructure. Last year’s census faced difficulty, meaning the information for future foundations cannot be properly verified until the next census, in 2023.
Population Studies Associate Professor at the University of Waikato, Polly Atatoa Carr notes the undercount of Māori communities. “A particular lack of data collected for the count for Māori population, around 68% with some completed information”.
Pacific populations also suffered a similar issue - about 65% had completed forms.
2018 Census was different to previous censuses, in that “it took a digital-first approach online”, Atatoa Carr explains. But this modern method meant other areas suffered, Atatoa Carr adds, “less of an emphasis on having backups available.”
The Census is more crucial than just a headcount of the nation, however. “We really need to get that [the census] right if we are going to achieve well-being and equity in Aotearoa.”
Fortunately, what was collected is not a total write-off. There is enough to extrapolate the remainder, Atatoa Carr says. “Those gaps are being filled by using previous census data or by using other data like administrative data from our integrated data infrastructure.”
Using five-year-old statistics for something intended to be present-day seems counterproductive, however. Then there’s the questionable method of using other sources of data like hospitals. Atatoa Carr is well aware of these dilemmas. “Whether we’re filling those gaps with appropriate and accurate data is another question and whether we have the appropriate ethical arrangements around the use of that data is also another question.”
In regards to future censuses in 2023 and onwards, Atatoa Carr closes with how we can use this census as a learning experience.“we will need to get that right from the very beginning, from the design phase, all the way through, including testing and including Māori governance and involvement of Pacific communities and others that are most affected.
By Angus Coker Grant
Lillian Hanly: Why was Census 2018 such a failure?
Polly Atatoa Carr: What we see from Census 2018 is that we’ve had a significant undercount for our population and that that undercount isn’t equal. So we know that we’ve got a particular lack of data collected for the count for Māori population, around 68% with some completed information and also a lack of information from our Pacific population, so around 65% from the forms completed for the 2018 Census. So we’ve got some challenges with our counts and some unequal challenges around those counts for the population that we really need the most accurate and robust data for.
Just some context, the last census that was undertaken was 2013 and to my understanding you don’t always get everybody but [for] 2018, the problem is that it was significantly lower than normal.
Yeah, that’s right. And the 2018 Census took a different approach, so it took a digital-first approach online, with people filling out their census forms online and there was less of an emphasis on having backups available so paper-based forms available and people going door-to-door in order to support completion of the census forms.
So it’s quite clear what was lacking?
Yeah the review that was released today has some really strong recommendations. That there was some areas lacking in the design phase and in partnering with our Māori and Pacific communities in the design phase. Also some issues with the lack of the testing of the approach and the lack of having some, y’know, backups or some buffers available when things don’t go so right.
There’s a specific, well, a significantly lower response rate in that Māori and Pasifika numbers. Why have we seen this?
Well I think what we’ve realised is that the approach to the 2018 Census doesn’t work so well for those communities. There’s also geographic regions where there are issues, there’s also characteristics within the counts that we are missing. So what we’re finding is that the approach for the 2018 Census didn’t work for everybody.
The Census is really important for just knowing who we’ve got in our country, for one: drawing electoral lines, but also in allocating funding, is that true?
That’s absolutely right. So the census data and the counts from the census data are used for the planning and the funding of our services that really make a difference to our population. Those services are things like health and education and our social sector, as you described. And we really need to get that right if we are going to achieve well-being and equity in Aotearoa.
So what happens now? Because obviously a count from 2013 isn’t going to provide us a very accurate or realistic picture of our population, but if we have a non-accurate picture of our population from a more recent count, what do we do?
Yeah, you’re right. So there are two, I think, important issues - at least two important issues to discuss here. First is the count issue, and so what has been done by Stats New Zealand and by others advising Stats New Zealand is figuring out a way where the gaps in the counts can be filled. So those gaps are being filled by using previous census data or by using other data like administrative data from our integrated data infrastructure. We have administrative data from our hospital records and other records are collected. So we can fill some of those gaps in our counts. Whether we’re filling those gaps with appropriate and accurate data is another question and whether we have the appropriate ethical arrangements around the use of that data is also another question. So that’s kind of the big picture question around the counts and the gaps and how we fill those. But then there’s also really important missing information around the characteristics of our population. Over and above what we’ve seen missing in the count. So we may well be able to fill some of the overall counts in electoral boundaries for example, but we can’t replace the information collected in the census about things like families and households and smoking status and other things that are collected from our population as a whole that we can’t backfill from any other information sources. And that’s going to be a problem moving forward.
Is there anything else you wanted to add?
I think it’s really important that we carry on these conversations. What’s happened with the release of this review is people may be thinking a little bit more about what the census means and what information is collected on them, by them, and how that information is used and how it’s governed.
These are problems that are going to be ongoing, right? Specifically to the 2018 census, this is something that we might solve, then the next year something else might come up, that sort of thing. Is it likely that they will attempt to fix these things but we will eventually have to do another census?
I absolutely agree that this will be ongoing. We can’t go back now. We’ve got gaps in the census and we’re having to fill them with data and we need to absolutely understand the challenges, the inaccuracies, the potential discrimination involved in gathering that data. And we will need to do another census. I think the review did recommend a census in 2023, and we will need to get that right from the very beginning, from the design phase, all the way through, including testing and including Māori governance and involvement of Pacific communities and others that are most affected.
Photo credit: TVNZ