Report on the State of Government Recordkeeping 2019/20
Foreword from the Chief Archivist
E ngā minita, me ngā Kaiwhiriwhiri o te Whare Pāremata – tēnā koutou, tēnā koutou, tēnā koutou katoa.
Well, where to start? This past year has definitely been, as they say, one for the history books. History books, though, have often reflected one side of history. And history has often become a myth where the dominant narrative crowds out other voices.
That’s why it has been so heartening, and heart-breaking, to hear the stories told by survivors of abuse in the Redress Hearing for the Abuse in Care Inquiry. Many of the witnesses spoke of the lack of agency they felt trying to engage with the Crown, and of how trying to access their information held by organisations felt like being re-victimised. Many of them mentioned the relief they experienced on hearing that there were records in the repository at Archives to back up their experiences.
The Abuse in Care Inquiry has highlighted just how critical the role of our national archive is. On the one hand we ensure that archives are kept safe and accessible, so that people can find them where appropriate; while on the other, we work with the public sector to ensure they’re creating and managing full and accurate records in the first place.
Often information management (IM) can seem unimportant or bureaucratic, especially when time and resources are stretched. But the Royal Commission demonstrates how government accountability is more than just a sound-bite, which is why the whole team at Archives has thrown themselves so passionately into supporting Royal Commission work. Seeing first-hand how items we hold here at Archives have contributed to returning mana and dignity to care leavers shows us the deeply human side of IM and its ability to change lives.
It has also shown us that a national archive is only as good as the records it holds. If public offices are not creating and maintaining full and accurate records and information, then in twenty-five years’ time, in fifty years’ time, people won’t be able to find and use it. It takes a village to manage the record of government.
Part of our role in that village is regulating government IM and auditing it. This year Archives has notified the public offices we will be auditing in the 2020/21 financial year and published a forward schedule for the following four years. We have also developed a maturity framework for organisations to measure themselves against, which will be released in early 2021. This will enable organisations to undertake a self-assessment to see how they’re doing and where they might need to focus their attention and resourcing.
Audit is just one of the tools we have to ensure a full and accurate record of government is created and maintained. Another ongoing piece of work for Archives has been our annual survey of public sector information management. Some of the findings from the survey’s key indicators are covered within this report. The full survey findings report will be issued separately.
This is our second year conducting this survey, and we are starting to develop a clearer picture of the information management environment across the public sector. The survey’s findings will feed into improving IM systems across government as part of our ‘building systems together’ strategic focus.
I also hope that this piece of work will improve the profile of IM professionals across the sector. I know that this work is often conducted by a single person, or a small team in the workplace, and has traditionally been undervalued and under-resourced work. Certainly, we saw this as the government had to react quickly to the COVID-19 pandemic lockdown, working from home, with organisations often needing to make decisions outside of their usual processes. With many agencies working together to coordinate their response, it was critical that information was managed appropriately, and much of that burden fell to a small group of people.
I would like to thank all those IM professionals who have kept calm and carried on throughout the COVID-19 lockdowns and beyond. Recording the government’s response to a once-in-a-lifetime pandemic presented some tricky problems, but the sector demonstrated that it was up to the challenge.
I personally took comfort from the archives we held from the 1918 influenza epidemic. What struck me most was that times change, and technology changes, but humans remain remarkably similar. Then, as now, there was concern and care for one another in the public record, as well as the mechanics of how to pull through the crisis at hand. I hope this is something that we all take from this year—the human side of what we do.
This year’s annual report on the state of government recordkeeping will be my last as Aotearoa’s Chief Archivist Kaipupuri Matua. My time in this role has been deeply important to me, with a healthy mixture of challenging and uplifting work. Archives New Zealand Te Rua Mahara o te Kāwanatanga has been a home and family for me and I am grateful to have had the opportunity to lead this organisation through the first few years of its transformation to 2057.
The Archives whānau are profoundly passionate about the mahi and kaupapa of the organisation and this shines through in the achievements we have made together. I wish the Archives whānau, and the new Chief Archivist, Stephen Clarke, nothing but the best for the future. We are well on our way to becoming the vibrant, trusted national archive Aotearoa needs.
Ngā mihi whānui ki a koutou katoa
In 2014/15, the Chief Archivist expressed concern about the low level of maturity of recordkeeping in more than half of the public offices audited under the Public Records Act 2005 (PRA). Since then, a programme of work to optimise Archives New Zealand’s regulatory role has been undertaken, with a mandatory Information and records management standard (the Standard) issued, along with a Regulatory statement, the revision of core guidance material and the embedding of the Executive Sponsor role.
These have provided a strong foundation from which to develop our monitoring framework. The annual survey of public sector information management practice is now into its second year. Some of the findings from this are covered later in this report. In 2020, we are launching the second round of audits under the PRA and a newly devised maturity self-assessment. These will assist regulated parties in understanding where they are at, and where they need to be in order to reach acceptable levels of maturity in their information management practice.
Five years is long enough for changes to a system and we are beginning to see these come through in the responses to our annual survey, and in our work assessing compliance with the PRA. The redevelopment of our regulatory programme has highlighted that a considerable driver of low maturity levels across the public sector is inadequate resourcing of information management areas. This is reflected in the results of the annual survey, which show low volumes of IM staffing, a lack of resources for IM activities, unsuitable system set-up and low prioritisation of IM within organisations.
Key to improving the state of government recordkeeping is the inclusion of IM considerations at the executive level. We have seen a slight rise in regulated parties reporting that they have included IM components to their governance arrangements. This is a positive result and we strongly encourage all organisations to ensure that IM requirements are considered when developing organisational strategies and policies, systems and processes. This is a foundation for elevating the importance of IM in organisations and integrating it into business operations.
Through our work assessing compliance with the PRA, it is clear that the management of digital records, and legacy digital records in particular, remains an issue for public offices and local authorities. Survey results also indicate a decrease in the number of organisations incorporating IM requirements into new business systems. Again, this appears to be grounded in less than ideal resourcing, as we are noting various degrees of compliance with principle 1.4 of the Standard. This requires organisations to have IM staff or access to appropriate skills. The onus for this sits with the leadership of organisations (principle 1.2 of the Standard).
As digital technologies are critical to public service operations and in some cases rapidly adopted during COVID-19 lockdowns, it is critical that the transition to digital processes does not lose sight of the requirements of the PRA and the Standard, and the relevant privacy and security considerations. In March this year, we released guidance on the adoption of Microsoft 365 software, noting that out of the box, these commonly used tools are not compliant with the PRA. We discuss this in more detail further on in this report.
Information is at the core of government business and is a key strategic asset. IM is the discipline that allows information assets to be governed, protected and prioritised. Survey results for 2020 indicate a slight decrease in organisations who have identified their high-risk and/or high-value information. This is concerning. Knowing and understanding the value and risk related to information assets is good IM practice. It also helps protect an organisation from risks to its reputation, from potential financial or material loss, and breach of statutory obligations.
Lastly, we have noted a decrease in the volume of authorised disposals over the past year. Destruction of digital information remains very low compared to physical information. Several factors contribute to this including the top three challenges of inadequate system set-up, resourcing and low prioritisation. The ongoing suspension of transfer to our Wellington repository and the Moratorium on the disposal of records relevant to the Royal Commission of Inquiry into Abuse in Care will have slowed disposal. However, it is concerning to see a lack of progress on digital destruction given that risk management is indicated by most organisations as an extremely important driver for good IM practice and the increasing number of organisations transitioning from paper-based to digital business processes. We will continue to work with regulated parties to improve responses in this area and ensure that authorised disposal takes place where appropriate.
Highlights for the year
Information management in the time of COVID
The information and records created in response to COVID-19 are invaluable in ensuring that the government response is effective, and that government can be held accountable now and over the long term for its decision making. Many of these records will be of long-term archival value.
In response to COVID-19, we advised the public service that as they operated under the restrictions imposed by the various alert levels, the information that was being created, often in rapidly improvised systems and workflows, would still need to be properly managed in accordance with the Standard, and current disposal authorities. We reminded organisations that it is imperative that all reasonable measures are taken to ensure that this information is identified, preserved and classified to support appropriate disposal in the future. The demands of the COVID-19 response across government reinforces the need to be able to share information between organisations safely and legally, maintaining the integrity of the record.
In June 2020, we released our guidance on the use of Microsoft 365 services. This was timely as the shift to work from home has seen an increase in the uptake and popularity of many of the software services that are commonly part of the 365 suite. These include Outlook mail services, Skype for Business, SharePoint Online, the collaboration tool Teams, and the standard Office applications such as Word and Excel, among others.
Minimum compliance requirements of the Standard apply to the Microsoft 365 environment as they do with any system that creates and manages public or local authority records. In its standard roll-out form, Microsoft 365 is not compliant with the Standard or the PRA. The implication of this for information and records management will depend on how organisations configure their software, the type of licence held and whether Microsoft 365 is integrated into an electronic document records management system (EDRMS) or enterprise content management system (ECMS).
Our guidance sets out how organisations can move toward compliance with the Standard and the PRA by applying controls, recognising risks and taking the principles of the Standard and the PRA into consideration. This requires organisations to understand the administrative applications and tools used to manage information and records in Microsoft 365. It is necessary to understand where and how information is stored across the suite including whether off-shore storage is appropriate for the type of information being held. Gaps in capability may need to be addressed by using third-party add-ons for IM functionality.
We recommend that IM and Information Technology staff work closely together to monitor and manage their instance of Microsoft 365 to ensure that updates and enhancements do not undermine compliance measures that have been implemented.
Open Government Partnership – Commitment 10: Monitoring the effectiveness of public body information management practices
The Open Government Partnership is an international agreement by governments to create greater transparency, increase civic participation and use new technologies to make governments more open, effective and accountable. The Public Service Commission leads this work in New Zealand.
Under the Third National Action Plan, we are leading Commitment 10: Monitoring the effectiveness of public body information management practices. The objective of Commitment 10 is to make the creation and management of government information more visible and therefore transparent, by developing and implementing a monitoring framework that supports public reporting on the effectiveness of IM in central and local government organisations.
Throughout 2019/20, we continued to develop and roll out the monitoring framework which reflects the Standard. The framework consists of the annual survey of public sector IM launched in 2019, a new Information Management Maturity Assessment and published Public Records Act audit reports. The monitoring framework is part of a larger work programme to implement our long-term strategy, Archives 2057, and feeds directly into our strategic focus areas of upholding transparency and building systems together. These focus areas support open government principles and shape the processes and systems of public sector information and records management.
Public Records Act Audits
Monitoring public sector information and records management is a shared responsibility between Archives and each public sector organisation. Section 33 of the PRA confers the audit function on the Chief Archivist.
The PRA audit is a point-in-time view of core IM practices that recognises an organisation’s operating environment and IM challenges. It is also an opportunity to show an organisation’s IM strengths and where there might be opportunities for improvement. Only public offices are audited under the PRA; we are not mandated to audit the IM practices of local authorities. Some public offices are outside the current scope of the audit programme, such as school boards of trustees and Ministers of the Crown.
Audits help ensure that public offices are delivering high-quality information to decision-makers, other government organisations, customers and stakeholders. They enable continuous improvement within a rapidly changing environment.
During 2019/20, we developed our audit programme for public offices. The first-year cohort of audits began in December 2020 and will be completed by the end of June 2021.
PRA audits are based on the Standard and the PRA, as represented through the IM Maturity Assessment. The audit programme will consider eight key areas of IM practice outlined in the IM Maturity Assessment: governance, self-monitoring, capability, creation, management, storage, access and disposal. Our follow-up approach will track remedial and improvement actions undertaken by public offices as a result of audit findings. After each audit, there is an expectation that the public office will use the recommendations identified by the independent auditors and Archives to create an action plan for their organisation to improve IM maturity. We will work with the public office audited to monitor their progress through the activities identified within their action plan.
This will also assist us to target advice, guidance, and other activity to support public offices and local authorities based on insights gained from the surveys and audits. You can read more about our audit programme, and view the schedule for public office audits here.
Regulating Government Information Management
How we identify issues
Information and records are vital for accountability and transparency. They form the basis for significant decision making. When issues arise with information management that frustrate accountability, the Chief Archivist’s regulatory role allows intervention.
The PRA does not provide Archives and the Chief Archivist with explicit investigative powers, but potential non-compliance is assessed against the Standard and the principles of the PRA. If we find through the assessment process that a breach of the PRA or the Standard has occurred, we take action and work with the parties concerned, including the responsible organisation, on remedies.
Our tools include the ability to direct a public office to report to us, to inspect records and to give directions about how estray records must be managed. The PRA also contains offences, notably damaging or improperly disposing of records. We have not actively considered prosecution this year.
We find out about potential non-compliance issues comes from various sources:
our monitoring of regulated parties
our daily interactions with regulated parties
complaints from affected or concerned members of the public
information reported in the media or received as part of journalists’ investigations
complaints from concerned third-party organisations
referrals from the Office of the Ombudsman.
533 contacts to the RKAdvice service
22 incidents that required assessment against the PRA
Of the 22, 8 require ongoing remediation or support following resolution of the initial issue
13 assessments closed with satisfactory assurances or required remediation issues raised
1 continuing at the time of this report
Working with the Ombudsman
The PRA intersects with several other Acts. The main ones are the Official Information Act 1982 (OIA), the Local Government Official Information and Meetings Act 1987 (LGOIMA) and the Privacy Act 2020.
Throughout 2019/20, we continued our close working relationship with the Office of the Ombudsman, Tari o te Kaitiaki Mana Tangata.
Some complaints made to the Ombudsman include instances where organisations cannot supply information because:
it cannot be located, despite extensive searching
it would require significant collation and research to be made available, and/or
it is determined not to be held when it could be reasonably expected that the information should exist.
Under section 28(6) of the OIA and section 27(6) of the LGOIMA, the Ombudsman may notify the Chief Archivist when an information request has been refused by an organisation for these reasons.
Case Study – Ombudsman investigation into Local Councils’ Official Information Act responses
Request for assessment
Staff at the Office of the Ombudsman consulted Archives New Zealand during an investigation they were conducting into four local councils.
During a standard investigation into responses by local councils to LGOIMA (Local Government Official Information and Meeting Act) requests, it was discovered that one council did not keep unredacted copies of records released in response to requests for official information. There is an obligation on local authorities to retain the full source material and to ensure that it is accessible, according to principle 3.3 of the Standard.
By only retaining the redacted response to requests for official information, this council was obscuring the decision-making process around the release or withholding of official information.
The Chief Archivist and Ombudsman agreed that the creation of a redacted document is the creation of a new record. A redacted record in no way replaces the source document as an authoritative version, nor does it exempt a local authority from its obligation to preserve and maintain the original document.
The Chief Archivist affirmed that all local bodies must consider the PRA and LGOIMA together to take responsibility for their own compliance.
The council concerned has advised the Office of the Ombudsman and Archives that they have since changed their process to ensure source and redacted responses are kept within the same OIA folder.
Understanding Public Records Act obligations
While Archives New Zealand receives requests for assessments from the Office of the Ombudsman and from the public, we also respond when the media or other sources publicise IM concerns. One of our key tools is the provision of advice and guidance. We work closely with agencies to understand their needs, concerns and areas for improvement in order to build better compliance with the PRA and improve our advice and guidance.
Case Study - Ministry of Social Development
Request for assessment
The Ministry of Social Development (MSD) notified Archives New Zealand of a potential unauthorised disposal of MSD records.
After becoming aware that some of its surplus office furniture may have contained MSD records when it was sold, MSD proactively alerted Archives New Zealand and the Office of the Privacy Commissioner.
Contact was made with the purchasers of the furniture, requesting that any records found with the furniture were returned and/or destroyed as appropriate. All responders indicated they would comply with the request.
MSD halted further sale of any furniture, and the remaining surplus furniture was stored in a facility under MSD control. Unfortunately, before a search of that furniture could take place, the facility where the furniture was stored was burnt to the ground.
MSD has been unable to determine what documents may have been destroyed in the fire, and whether any documents remaining in the furniture that was sold were appropriately disposed of.
Archives New Zealand worked with the Ministry’s Executive Sponsor to review the information management policies and procedures at MSD.
During this process MSD confirmed that its guide for information management during office relocations had only recently been updated. MSD worked with the business on the updated guidance and tested it on subsequent moves, to ensure it reduced the risk of future information loss.
We will meet regularly with the MSD Executive Sponsor throughout the next year to continue discussions and support for improved information management.
Case study – Te Puni Kōkiri
Request for assessment
We were made aware that archives loaned to Te Puni Kōkiri (TPK) through our government loans service had been returned with pages missing.
As part of the government loans service, public offices that transfer records to Archives New Zealand may request them back if they need to refer to them. However, after TPK borrowed records relating to Māori housing and welfare, they returned a file with a substantial number of pages missing. Under section 61 of the PRA, it is an offence to damage, dispose of or destroy a public record without the authorisation of the Chief Archivist. The staff member who had removed the pages was no longer working at TPK and no one knew where the missing pages were. The loss of these pages is significant. TPK is one of the Crown response agencies to the agencies to the Abuse in Care Inquiry; it is possible that the missing pages would be relevant to the Inquiry and may have been required as evidence.
The Chief Archivist sets conditions for the government loans service. We can suspend off-site access for agencies that fail to comply or apply a penalty of up to $10,000.
A full suspension of access was ruled out in this case because of TPK’s role as a Crown response agency to the Abuse in Care Inquiry. Instead, we put a partial suspension in place, which limited TPK to viewing items in our reading rooms, or through digitised copies securely supplied. TPK were penalised $2000 for the missing pages. They were also warned that any further breaches would result in a full suspension of access to archival material.
TPK have since commenced a review of their government loans process to ensure all records loaned back from Archives are accounted for. This review process is ongoing.
When incidents like this occur, we also examine our own processes to ensure that expectations are clear. The Chief Archivist took the decision that digital delivery of government loans would mitigate the risks created by agencies borrowing physical files. This has now become standard procedure.
Records, rights, and royal commissions
Archives New Zealand has multiple roles in supporting inquiries and plays a pivotal role in supporting the Royal Commission of Inquiry into Historic Abuse in State Care and in the Care of Faith-Based Institutions (Abuse in Care Inquiry). We provide advice and instruction on government records management, the regulation of disposal of public records, and the preservation of and provision of access to public archives that document New Zealand’s history, heritage and identity.
Records created in the course of government activities can seem trivial or transitory at the time of creation. To a care-leaver or abuse survivor however, such records can contain information about their childhood and life which may not form part of their own memory or knowledge. The records created by government organisations, and their agents, can help care-leavers understand who they are and where they come from, telling them something about their parents, birthplace, childhood or other significant events.
Our work in supporting the Abuse in Care Inquiry has highlighted several areas of IM and PRA compliance that are concerning to us.
Recordkeeping must support inquiries
During 2019/20, district health boards (DHBs), among other organisations, began to receive Notices to Produce from the Abuse in Care Inquiry. These notices are issued under section 20 of the Inquiries Act 2013 and require the receiving agency (or individual) to produce documents and provide information necessary for the Abuse in Care Inquiry to identify, examine and report on the matters within scope of the inquiry. As the Abuse in Care Inquiry is expected to deliver its final report in 2023, we expect these requests for public records and archives to continue for some time.
DHBs will have records that are relevant to the Abuse in Care Inquiry. As the terms of reference for the Inquiry extend from 1950 through to the present day, relevant records could be active, inactive or archival records stored at the DHB, contracted storage, with Archives New Zealand or at an alternative approved repository.
As section 20 Notices to Produce continue to be issued, we expect that adequate recordkeeping controls are in place over the records relevant to any request. Section 17 of the PRA requires public offices to create and maintain in an accessible form, full and accurate records of their affairs. We strongly recommend that organisations that receive section 20 Notices confirm the mandate for research and ensure that they only provide access to records within scope of the Notice, ensuring that the IM staff know the contents of the records they are providing access to before giving researchers access. Keeping a register of all records that are accessed and copied, and by whom, will be important in ensuring the chain of custody for the records.
If organisations are meeting their recordkeeping requirements, then there is less likelihood of failing to produce documents for any inquiry. Costs will be incurred in identifying, locating, retrieving and copying relevant documents. These costs are not a convincing reason for failing to produce them, and, if high, are a likely indicator of fundamental information and records management deficiencies.
Ongoing disposal moratorium
The Chief Archivist has issued a moratorium on the disposal of records held by all public offices which may be relevant to the subject matter of the Abuse in Care Inquiry. This means that no public records that may be relevant to the Abuse in Care Inquiry can be destroyed, discharged, or transferred while the Royal Commission is in progress.
A number of DHBs have asked us whether they can destroy patient records in line with their disposal authorities where the records are deemed to not be of relevance to the Inquiry. A driver for this appears to be storage and cost pressures. The Abuse in Care Inquiry is responsible for determining what records are relevant, not Archives New Zealand or any of the Crown response agencies. Our advice to agencies that are not sure about whether records are relevant or not is: if in doubt, retain the records. This mirrors the advice received from the Abuse in Care Inquiry to DHBs that while they can continue with appraisal and sentencing of records, no records can be destroyed until after the moratorium is lifted at the end of the Commission’s mandate.
Outsourcing of functions
Engaging external parties to perform business functions has implications for information and records management. It is reasonably common for DHBs particularly to outsource some functions. Under the PRA, every public office must create and maintain full and accurate records of its affairs, in accordance with normal, prudent business practice. Outsourcing a business function or activity does not lessen an organisation’s responsibility to ensure that it is carried out properly and that all requirements for information and records management are met.
When outsourcing business, public offices are responsible for ensuring that the contracted party is actively creating and maintaining information and records of their affairs. We recommend that these provisions are clearly articulated in contracts with service providers, and that these are monitored to ensure obligations are being met. Organisations can learn more about their responsibilities here.
Similarly, while the moratorium on disposal of records relevant to the Abuse in Care Inquiry does not directly apply to non-public records, that is records created by non-government agencies, any records created by non-government organisations that relate to work carried out on behalf of the government agencies are covered by the moratorium.
Public sector information management survey
Key findings from the Survey of public sector information management 2020
In 2019/20 Archives New Zealand conducted its second annual survey of information management (IM) practices in public offices and local authorities.
Objectives of the survey
Establish and track how well public sector organisations are performing against the requirements of the PRA, the information and records management standard, and good IM practice
Allow tracking of improvements on organisations' performance over time
Identify the risks, challenges, opportunities and emerging trends affecting IM in organisations, so we can feed this intelligence into responsive regulation
Provide public visibility of organisations' performance
Information is at the core of government business and is a key strategic asset. IM is the discipline that allows information assets to be governed, protected and prioritised.
In 2018/19 we reinstated an annual survey. As part of the survey design, we selected five key indicators to measure the overall state of public sector IM and provide a high-level perspective on whether IM within the public sector was improving, deteriorating, or remaining stable. With the completion of this year’s survey, we can begin to see whether progress is being made against the indicators and identify where we need to take more proactive steps to bring about improvements. This year, we made some refinements to the survey questions that affected our ability to directly compare some results with the first survey. The picture of change over time is still emerging and will improve over successive surveys.
The key indicators are not the sole measure of the state of public sector IM, but they have been selected because we consider them to be fundamental building blocks for effective IM. The full survey results provide more comprehensive data on the performance of public sector organisations. These results will be reported in a separate findings report, and we will publish the raw data on data.govt.nz. We will also use them to inform our interactions with individual organisations over the coming year.
Who was surveyed?
The survey was sent to 270 public sector organisations, including:
191 public offices, which were required to respond by direction to report (section 31 of the PRA)
79 local authorities, which were invited to respond.
The survey recorded an 80% response rate, slightly down on last year’s response rate of 89.7%. We received one unsolicited response, two late responses and five incomplete responses, all of which were excluded from the analysis. A total of 47 organisations did not respond, comprising 22 public offices and 25 local authorities. The responses from the Government Communications Security Bureau and New Zealand Security Intelligence Service are excluded from the analysis.
Indicator 1: An increasing number of organisations have implemented governance groups for information management
What we asked and why it is important
We asked survey participants if they have a formal governance group in place which is either dedicated to IM or has IM oversight as part of its mandate (Q.9).
The Standard requires that: Information and records management must be the responsibility of senior management. Senior management must provide direction and support to meet business requirements as well as relevant laws and regulations (1.2).
The role of an active governance group is to ensure, at a strategic level, that IM requirements are considered when developing organisational strategies and policies and implementing systems and processes. It is a foundation for elevating the importance of IM in organisations and integrating it into business operations.
What we found and how it compares to last year
This year, just over half of respondents (52%) have a formal governance group in place. Last year, 30% of respondents had a formal governance group in place, while 24% had one in development. Although we did not provide respondents with an ‘in development’ answer option this year, the data suggests that there has been an increase in the proportion of organisations implementing governance groups for IM (Figure 1). This change may reflect work completed by respondents who previously had a governance group ‘in development.’ That said, it is concerning to see that a significant number of respondents (48%) are lacking a formal governance group and missing out on the potential benefits such a group can bring to the organisation’s IM.
30% of respondents have a governance group for IM
52%of respondents have a governance group for IM
Figure 1: Indicative change over time for Indicator 1
Looking at the response split by organisation size it is encouraging to see that for very large organisations (3000+ full-time equivalents) most organisations have a formal governance group in place (Figure 2). These organisations are more likely to operate in a complex information environment and hold high-value and/or high-risk information. However, as a whole there does not seem to be an obvious relationship between organisation size and the presence or absence of formal governance groups.
In 2018/19 we recommended that executive sponsors take a lead in their organisations to establish and/or sustain governance groups for IM. Reflecting on the insights from this year, we will seek to influence change in those organisations that do not have a formal governance group. This approach may include:
making expectations more explicit in our guidance, communications and interactions with executive sponsors
engaging with selected organisations that hold high-value and/or high-risk information, on the basis that an absence of formal IM governance heightens the risk of IM failure and consequent public harm
engaging with individual organisations that lack formal IM governance, as a component of follow-up on their audit findings.
Indicator 2: An overall increasing number of IM staff employed by public sector organisations
What we asked and why it is important
We asked how many dedicated, full-time equivalent (FTE) IM staff organisations employed (Q.5). The question asked respondents to exclude staff in geospatial information systems, business intelligence, data management, medical records or staff whose main role is not IM.
The Standard requires that: Organisations must have information and records management staff, or access to appropriate skills (1.4).
IM impacts all areas of business, and IM specialists should be involved and included in a wide variety of business activities. These include system and process design, information and records sharing, risk management, and managing information, data and records for accountability and value.
As new technologies proliferate at speed, the opportunities and challenges for meeting IM requirements also multiply. IM specialists remain essential for the proper functioning of digital government, through their IM leadership and advocacy, and by harnessing the abilities of technology to make IM easier for their organisations.
What we found and how it compares to last year
This year, 79% of respondents have some dedicated, specialised IM resources. Last year, the figure was also 79%. This suggests that from a sector-wide perspective the level of resourcing has been essentially static over the past 12 months (Figure 3).
79% of respondents have 'some' IM staff
79% of respondents have 'some' IM staff
Figure 3: Change over time for Indicator 2
For the 2019/20 survey we refined the response options for the question so that organisations could tell us the exact number of IM staff they employed, rather than selecting a range. This means that after next years’ survey we will be able establish whether there has been an overall increase in IM staff numbers. Table 1 shows this year’s data, configured against the range we used last year, alongside last year’s data.
|Number of staff||2018/2019 Survey responses as number||2018/19 Survey responses as %||2019/20 Survey responses as number||2019/20 Survey responses as %|
|No IM FTE||47||20.8%||44||20.6%|
|1 IM FTE or less||55||24.3%||74||34.1%|
|More than 1 up to 3 IM FTE||58||25.7%||44||20.6%|
|More than 3 up to 6 IM FTE||30||13.3%||25||11.7%|
|More than 6 up to 10 IM FTE||16||7.1%||20||9.3%|
|More than 10 IM FTE||20||8.8%||8||3.7%|
|Total number of responses||266||214|
Table 1: Breakdown of change in IM staffing levels between surveys
In 2018/19 we recommended that executive sponsors take steps to consider whether their staffing levels or lack of dedicated IM staff created risks for the organisation and take actions to manage those risk. While this recommendation still stands, the findings suggest that more work is required on our part to:
engage with selected organisations that have no dedicated IM staff, as a component of follow-up on their audit findings
establish recommended staff metrics, which can assist executive sponsors with assessing appropriate staffing levels.
Indicator 3: An overall increase in organisations that have identified their high-value and/or high-risk information
What we asked and why it is important
We asked survey participants if they have identified their high-value and/or high-risk information (Q.32).
The Standard requires that: High-value and/or high-risk information areas of business, and the information and records needed to support them, must be identified and regularly reviewed (2.2).
For an organisation, high-value information is information that is critical to performing its core, legislated functions. High-risk information is information that, if mismanaged, could expose the organisation to major operational failure, financial or material loss, breach of statutory obligations, or loss of public or Ministerial confidence.
For New Zealanders, high-value information is information that supports their individual or collective rights, entitlements, identity and aspirations. High-risk information is information that, if mismanaged, could result in public harm. Actions such as improper release of information or barriers to access can have real-world impacts on the lives of New Zealanders. Those impacts can include physical, emotional and psychological harm. We have seen this through the work of the Abuse in Care Inquiry.
Identifying high-value/high-risk information is a foundation for other IM activities. It is a critical first step towards mitigating associated risks and extracting maximum value from information assets.
What we found and how it compares to last year
This year, 36% percent of respondents have identified their high-value and/or high-risk information, while 43% say that work is ‘in progress’. Last year, 64% of respondents had identified their high-value and/or high-risk information and we did not provide an ‘in progress’ answer option. Although refinements were made to the question this year with the addition of the ‘in progress’ answer option, on the face of it the data suggests that there has been a decrease in the proportion of organisations that have identified their high-value and/or high-risk information (Figure 4). This downward trend is concerning but, factoring in the modifications made to the question this year, we will look to next year’s survey for confirmation of that trend.
64% of respondents have identified their high-value/high-risk information
36% of respondents have identified their high value/high risk information
Figure 4: Indicative change over time for Indicator 3
We also asked organisations about whether they had an information asset register (IAR) or similar tool in place (Q.23), as a helpful means of documenting high-risk and/or high-value information assets. An IAR-type tool puts an organisation in a strong position to manage accessibility and usability, mitigate risks that might affect the assets and manage their relevance, currency, retention and disposal.
Only 38% of respondents said that they do not have an IAR, while 48% responded ‘yes’ or ‘in development’, and 14% responded ‘work started but deferred’ (Figure 5). This suggests that just under half of respondents are moving beyond the discovery stage and ensuring that their high-value and/or high-risk information assets are documented.
Although for many organisations we have indications of promising progress with the discovery and documentation of high-value and/or high-risk information, it is an activity we consider important for all public sector organisations, regardless of their size, function or complexity.
In 2018/19 we recommended that public sector organisations take actions to help them manage their high-value and/or high-risk information, such as using an IAR and including IM requirements in system specifications for business activities that create this type of information. The findings show that there is still work to be done with those organisations that have not made a start on discovery and documentation, so in the coming year we will focus on:
engaging with organisations that have not identified their high-value and/or high-risk information as a component of follow-up on audit findings.
Indicator 4: An overall increase in the number of organisations building IM requirements into new business systems
What we asked and why it is important
We asked survey participants whether they have built a process for managing information through its lifecycle into new business information systems (i.e. systems implemented in the last 12 months) (Q.35).
The Standard requires that: Information and records management must be design components of all systems and service environments where high risk/high value business is undertaken (2.3).
Building IM requirements into a business system from the very beginning is a key enabler for proper management of the information created and stored in that system. This means that the system is optimised to support the creation and maintenance of complete, accurate and accessible information, as well as its eventual, authorised disposal.
We recognise that it can be extremely challenging to retroactively add or plug-in IM requirements to existing systems, particularly when they have already been in operation for an extended period and are bespoke, no longer supported or at end of life. For new systems, we expect these requirements to be built in from the start.
Business information systems are not limited to electronic documents and records management systems or enterprise content management systems. Information that has to be managed in accordance with our requirements is created and stored across a wide variety of business systems, including:
finance and human resources
line-of-business systems that support the organisation’s unique functions
systems that support collaboration between government organisations and/or external parties
email and email archiving systems
What we found and how it compares to last year
This year, of the 147 organisations that implemented a new business information system in the last 12 months, half (50%) have built in a process for managing information through its lifecycle, while 17% of respondents replied, ‘don’t know.’ Last year, we asked a similar question without the 12-month timeframe and found that 23% of respondents had built IM requirements into new systems, while 62% had partially built in requirements, for a combined figure of 85%.
Refinements were made to both the question and answer options in this year’s survey, with the addition of the 12-month timeframe and the removal of the ‘partially’ answer option. If we combine the ‘partially’ and ‘fully’ responses from last year’s survey as a ‘yes’ response, then the data suggests that there has been a decrease in the proportion of organisations that have built IM requirements into new business systems (Figure 6). This downward trend is concerning but, factoring in the modifications made to the question this year, we will look to next year’s survey for confirmation of that trend. In any case, we consider that the proportion of organisations building IM requirements into new systems remains too low.
85% of respondents have fully or partially built IM requirements into new business systems
50% of respondents have built IM requirements into new business systems
Figure 6: Indicative change over time for Indicator 4
Figure 7 plots this year’s responses against the presence or absence of a formal IM governance group. This shows that when a formal governance group is present there appears to be a greater likelihood that the organisation has also built a process for managing information through its lifecycle into new business information systems. One of the purposes of a formal governance group is to ensure, at a strategic level, that IM requirements are considered when developing systems.
This year we repeated a question about the challenges affecting organisations’ ability to build in IM requirements when developing new systems. The top three challenges remain the same as last year: lack of awareness amongst internal staff, the number of systems in use and lack of consultation with IM staff.
In 2018/19 we made several recommendations about how executive sponsors could address the challenges their organisations face with building IM requirements into new business information systems. This year’s findings suggest that we need to take further steps to help organisations overcome those challenges. Those steps may include:
using our guidance, communications and interactions with executive sponsors to emphasise the importance of involving IM staff in new business system projects and of raising awareness amongst staff responsible for the system build
engaging with the ICT community and industry on involving IM in new business system projects.
Indicator 5: An overall increase in the number of organisations actively doing authorised destruction of information
What we asked and why it is important
We asked survey participants if they have carried out any authorised destruction of information in the past 12 months (Q.55 on physical information and Q.56 on digital information).
The Standard requires that: Information and records must be systematically disposed of when authorised and legally appropriate to do so (3.7).
Our general disposal authorities (GDAs) (GDA 6 and GDA 7) have been developed for the public sector to enable the lawful destruction of common corporate records without requiring organisation-specific authorisation from the Chief Archivist. GDAs are designed to make it easy to destroy information that has no long-term value.
This indicator focuses on destruction as one of the approved methods of disposal because it is an activity that all public sector organisations can be doing. Even if they do not have an organisation-specific disposal authority in place, organisations can still apply and action the GDAs.
Although destroying information may seem daunting or risky, it is an important component of effective IM. Typically, a large proportion of the information an organisation creates does not have long-term value for the organisation or New Zealanders, and a time will come when it is no longer required and can be safely destroyed.
The benefits of active, authorised destruction include:
mitigating the risks associated with retaining information for longer than required, such as privacy or security breaches and unauthorised access
minimising the quantity of digital information an organisation has to manage, thereby increasing the efficiency of business systems (e.g. fewer irrelevant search results to wade through) and making the organisation’s high value information easier to discover and manage
decreased storage costs, for both physical and digital. The cost of storing digital information over the long-term should not be underestimated. The price per gigabyte combined with the cost of storing back-ups, versioning and vendor costs, such as retrieval charges, may be high. On 28 March 2019, a moratorium was put in place on the disposal of any records relevant to the Royal Commission of Inquiry into Historical Abuse in State Care and in Faith-Based Institutions. This is likely to have had an impact on authorised destruction by some public offices during the timeframes of the survey. However, the impact on destruction practices was not measured as an explicit component of the survey.
What we found and how it compares to last year
This year, we found that 58% of respondents had done some form of destruction (i.e. either physical or digital), compared to last year’s figure of 63%. This is a small but notable decrease in the number of organisations doing authorised destruction (Figure 8).
63% of respondents have done authorised destruction
58% of respondents have done authorised destruction
Figure 8: Change over time for Indicator 5
We repeated a question about the challenges affecting organisations’ ability to undertake regular, authorised destruction (Q.57). Figure 9 shows that the top three challenges remain the same: system set-up, lack of resources and lack of prioritisation by staff responsible for electronic deletion.
Authorised destruction of digital information remains low (26%) compared to physical information (52%) possibly as a reflection of the top three challenges. It is concerning to see the lack of progress on digital destruction given that:
a high proportion of respondents (77%) told us that risk management is an extremely important driver for good IM practices and processes in their organisation (Q.7) and yet regular destruction is a component of managing risk
a high proportion of respondents (92%) told us that they are transitioning from paper-based to digital business processes (Q.21) which will increase the volume of digital information that is ultimately going to need to flow through a digital destruction process
as discussed for Indicator 4, only half of the respondents (50%) that have implemented new business systems in the past 12 months have built in IM requirements (Q.35) which are fundamental for enabling destruction.
In 2018/19 we recommended that organisations establish the groundwork for applying the general disposal authorities (GDAs) issued by the Chief Archivist and, where applicable, start applying their organisation-specific disposal authorities. The findings suggest that we need to take further steps to help IM staff members overcome the challenges identified. This approach may include:
making the importance of regular, authorised destruction more explicit in our guidance, communications and interactions with executive sponsors. This could include addressing those challenges that a formal governance group has influence over, such as resourcing, prioritisation, and design of new business systems
engaging with the ICT community on system functionality that supports disposal.
In additions to these actions, we will also engage with organisations that are subject to the disposal moratorium and reported undertaking authorised destruction in the past 12 months. We will seek to confirm that the information destroyed is outside the scope of the moratorium.
Understanding the Public Records Act 2005 and the role of Archives New Zealand Te Rua Mahara o te Kāwanatanga
The Public Records Act 2005
Records of public sector activities have always been kept. However, until the introduction of the Public Records Act 2005 (PRA), there was no general legislative requirements for what information and records needed to be created and managed.
The PRA sets out the regulatory framework for information management across the public sector. The primary purpose of the PRA is to enable the accountability and transparency of government decision making by ensuring organisations create and maintain full and accurate records of their activities.
The PRA also establishes the statutory role and duties of the Chief Archivist.
exercising a leadership role of information management across public offices
setting standards for public sector information management
authorising the disposal of records when they are no longer required for business purposes
providing advice and support for organisations so they can comply with the requirements of the PRA.
Public offices and local authorities are covered by the PRA, although with different compliance requirements. A wide range of organisations are public offices, including government departments, district health boards, Crown entities, state owned enterprises, school boards of trustees and government ministers.
Regional councils and territorial authorities are local authorities under the PRA. Council-controlled organisations, council-controlled trading organisations and local government organisations are also considered local authorities under the PRA.
Archives New Zealand is a regulator
The PRA establishes the Chief Archivist as an independent information regulator within government. In delivering this role, the Chief Archivist and Archives New Zealand have responsibility for supporting, monitoring and directing the public sector to facilitate compliance with information management requirements. These requirements are set out in the Standard.
We regulate approximately 3,000 public offices and local authorities, which includes 2,500 school boards of trustees. These organisations vary widely in their size, complexity, access to funding, staffing levels, and the number of functions they carry out.
These factors all affect the level of information management maturity in organisations, as well as the level of risk associated with not being able to find or access information that has been created.
Determining the status of public offices
Archives New Zealand monitors change in the structure of the public sector and relevant legislation to ensure that both Archives and regulated parties correctly understand the extent of our regulatory responsibilities.
There are two instances where an organisation can be defined as a public office by legislation:
if an organisation is defined as a public office in section 4 of the PRA
if an organisation is otherwise defined as such in other legislation.
The types of organisations are that defined as public offices under the PRA include:
all departments as defined in section 5 of the Public Service Act 2020 including departmental agencies and interdepartmental executive boards
interdepartmental ventures as defined in section 5 of the Public Service Act 2020
all Offices of Parliament as defined in section 2(1) of the Public Finance Act 1989
all state enterprises as defined in section 2 of the State-Owned Enterprises Act 1986
all Crown entities as listed in Schedule 1 and Schedule 2 of the Crown Entities Act 2004 including:
autonomous Crown entities
independent Crown entities
Crown entity companies
All Crown entities as defined in section 7(1) of the Crown Entities Act 2004 including for example:
boards of trustees for state and state integrated schools
tertiary education institutions including colleges of education, polytechnics, specialist colleges, universities, and wānanga
Crown entity subsidiaries (dependent on control factors).
We will be clarifying the way entities now defined by the Public Service Act 2020 need to comply with the PRA.
Glossary of key terms
Means a record of class of records, in any form, in whole or in part, created or received by a public office in the conduct of its affairs, this includes estray records.
The Information and records management standard (16/S1) issued by the Chief Archivist.
The range of activities, defined in the Public Records Act 2005, that can be applied to public and local authority records that are no longer of active business value. This covers transfer of control; sale; alteration; destruction; or discharging of records.
Regional council or territorial authority, including:
a council-controlled organisation
a council-controlled trading organisation
a local government organisation.
Public Sector Organisation/Regulated Party
Umbrella term used by Te Rua Mahara Archives New Zealand to describe all organisation subject to the Public Records Act 2005, including public office and local authorities.
A phone and email advisory service for all government and local authorities seeking advice or assistance with IM matters.
Last updated on 19 February 2021