Context:

Data integration facilitates opportunities to build partnerships and collaboration with the goal of meeting policy and research needs. In the last year, ABS has improved stakeholder partnerships and collaboration on statistical data integration activity, as demonstrated by the range of data integration projects undertaken in partnership with other agencies.

Through DIPA, ABS collaborates with partners across government to make better use of existing public data to support cross-portfolio research and improved analytic capability within the APS.

ABS works closely with four analytical units to deliver DIPA research projects. These are the:

• Physical Environment Analytical Network (PEAN) - lead agencies are the Department of Environment and Energy (DoEE) and the Department of Agriculture and Water Resources (DAWR)
• Economic Data and Analysis Network (EDAN) - the lead agency is the Department of Industry, Innovation and Science (DIIS)
• Social, Health and Welfare Analytical Unit (SHWAU) - lead agencies are the Department of Health (Health) and the Department of Social Services (DSS)
• Central Analytics Hub (CAH) - led by Prime Minister and Cabinet (PM&C).

There is also high demand for ABS data integration capability beyond the DIPA program. The ABS works closely with state and Commonwealth government partners, the RBA and academic researchers to maintain a balance of work and ensure data integration activities support key policy priorities outside of DIPA.

ABS has maintained its relevance in the national statistical community by collaborating with stakeholders to advance the value of statistical data integration. Together with a variety of partners the ABS has innovated to harness the value of public data to create statistical products to meet the demands of Australian decision makers. The ABS continues to invest in the provision of innovative solutions that maximise the value of existing data sets thereby increasing customer satisfaction and meeting internal transformation objectives.

Result: Achieved

Effective partnerships and collaboration throughout 2018-19 enabled delivery of a wide range of data integration projects.

Through DIPA collaborations, all 33 Policy Delivery Plans (PDPs) have been delivered, with expected and significant progress on seven infrastructure projects. 2018-19 PDPs include research topics relating to mental health, older Australians, productivity, natural disasters and water policies. These PDPs make extensive use of the Multi-Agency Data Integration Project (MADIP) and BLADE integrated data assets, which have also been enhanced through DIPA collaboration (see Target 3.2.2 for further details).

As well as undertaking the data linkage to enable these projects, the ABS works collaboratively with analytical units to extract and assemble customised integrated data in line with research objectives, and is supporting researchers to build analytical capability, through secondments, training, and provision of methodological support and technical advice.

Through these collaboration initiatives, access to integrated data assets has increased considerably over the past year, with 266 researchers currently analysing MADIP and/or BLADE integrated data assets (up from 103 at June 2018).

Collaboration with partner agencies, including state and Commonwealth government departments, has also enabled delivery of a number of data integration projects outside of DIPA, with 16 projects initiated and a further 35 existing projects extended in 2018-19. Key projects include:

• Partnership with National Centre for Vocational Education Research to link data from the Census and the National Vocational Education and Training (VET) in Schools Collection, to track over time the post-school employment and further education destinations of VET students in secondary schools
• Collaboration with South Australian Government to link state business administrative datasets with the BLADE to explore industry and employment dynamics and inform economic development policy
• Partnership with Australian Institute of Health and Welfare (AIHW) to deliver the Coordination of Health Care Study, which links data from selected health-related administrative datasets with patient-reported experiences of health care to enable analysis and understanding of the impact of coordination and continuity of care on health outcomes in Australia (See Case Study 1: Coordination of Health Care Study)
• Working with the Department of Education on new data insights to provide more accurate funding arrangements for non-government schools using different government information combined through the MADIP (See Case Study 4: New data insights provide more accurate information for school funding model)
• The RBA has utilised BLADE to examine: how Australian businesses manage their balance sheets and how their financial structure affects various outcomes like investment, growth and survival; the effects of changes in minimum and award wages on the labour market; and assess the causes and effects of changes in business productivity in Australia.

Result: Achieved

Feedback provided to ABS SES indicates that joint data integration initiatives provide benefit to Commonwealth and jurisdictional agencies. Such projects use pre-existing datasets and data capabilities to enable improved health, education and economic policy development.

ABS Strategic Partnership Managers and outposted officers are seen to be knowledgeable, helpful and responsive - helping stakeholders navigate the complex new world of data integration and microdata projects.

The ABS’ Statistical Services Group also undertook a number of projects with a range of Commonwealth agencies over the past year. An example of this is the Department of Foreign of Affairs and Trade and Australian Trade and Investment Commission funded project that provides an insight into the economic activity of foreign owned businesses in Australia by integrating detailed business characteristics from BLADE and other administrative data sets with official trade statistics.

Interviews conducted with key senior stakeholders by an independent consultant in late 2018 reveal an ever increasing demand for data integration projects from the ABS especially in the areas of health, education, human services, tax and employment.

Although stakeholders see the ABS as a willing collaborator on projects, and data access has improved as a result of deliberate and concerted work to streamline access processes and evolve our approach to managing data access to different classes of users, there is still unmet demand for data and data access, which stakeholders attribute largely to funding reductions to the ABS.

Context:

Public data is a valuable resource that has the potential to enable new insights and statistical solutions to inform important social and economic policy analysis, research and evaluation. However, the information needed to address complex policy issues and research questions is often spread across datasets held by a number of agencies. Data integration enables analysis that would not be possible using only separate, unlinked data sources.

The ABS is the primary Accredited Integrating Authority for the DIPA. Through DIPA, the ABS is growing new integrated data assets and delivering them to approved researchers to inform cross-portfolio social, economic and environment policy priorities. The ABS is also developing innovative data access solutions to enable more flexible and informative analysis of integrated data, while keeping private information secure and protected.

The capability developed through DIPA is enabling the ABS to deliver on a number of other data integration initiatives, to address unmet statistical need and to improve the efficiency of costly collections by better leveraging existing data to meet current and emerging information requirements.

Result: Achieved

The ABS is collaborating with partners across government to enhance and deliver new integrated data assets, as well as expand access to integrated data assets for government and non-government researchers.

ABS collaborates with DIPA partners to acquire and safely integrate a wide range of public data. Throughout this process, ABS engages closely with data custodians to ensure approval and governance requirements are met.

Collaborative initiatives to use existing public data is also enabling a number of projects outside of DIPA. For example, the 2016 iterations of Australian Census and Migrants Integrated Dataset (ACMID) and Australian Census and Temporary Entrants Integrated Dataset link data held by Department of Home Affairs with Census data to create new statistical solutions that deliver novel insights into the population. For example, by combining 2016 Census data and temporary visa information from the Department of Home Affairs, a comprehensive picture is available of where groups of temporary residents live, the countries they come from, what work they do, what they earn and if they are studying.

The ACMID also allows Census information to be cross-classified by migrants' entry conditions (e.g. visa status, location onshore or offshore and whether a main or secondary applicant), providing valuable insights into settlement patterns of overseas born persons by visa stream as well as information on citizenship, housing, income, labour force characteristics, changing occupations, educational pathways and family characteristics.

Better use of public data is also facilitating improvements to ABS collections. As a result of research led by the Census Futures team, the 2021 Census will use administrative data to improve the quality and efficiency of Census outputs. Use of public data will:

• improve the Census count by using counts of people from administrative data to better inform on the number of people living in houses where no form is received
• further improve these counts by providing a 'signs of life' indicator for houses to assist determining whether they were occupied on Census night, and
• assist Census collectors to efficiently follow up Australians where forms have not yet been received.

Result: Achieved

In 2018-19, the ABS has improved the data available in its key integrated data assets, as follows the:

• MADIP expanded from 8 to 13 ongoing linked datasets. The MADIP was also used to undertake 5 one-off linkage projects.
• BLADE expanded from 9 to 10 ongoing linked datasets, with one additional year of data added. The BLADE was also used to undertake 10 one-off linkage projects.

Other new integrated datasets released in 2018-19 include:

• A TableBuilder product for the Linked Employer-Employee Database (LEED) - allowing approved users to create their own customised and confidentialised tables with LEED data.
• 2016 Census data being included in the Australian Census Longitudinal Dataset (ACLD) - the ACLD now provides access to three years' of Census data (2006, 2011, 2016).

These enhancements provide new and customised integrated datasets for analysis by government researchers.

Result: Achieved

In 2018-19, the ABS progressed seven significant and innovative projects to support broader whole of government access to, and use of, high quality integrated data. These projects are undertaken in partnership with a range of Commonwealth Agencies as part of DIPA.

1. Interoperability - the ABS and AIHW developed and tested the use of interoperable methods for combining integrated data assets. Trials demonstrated that data from each agency could be brought together to support policy research using a federated approach that preserves privacy, achieves high quality results and avoids duplication. Interoperability was used to generate research data sets for analysis of the use of government Services by older Australians. See https://www.abs.gov.au/websitedbs/D3310114.nsf/home/Statistical+Data+Integration+-+MADIP+Research+Projects
2. Location Index - the ABS, Department of the Environment and Energy, Department of Agriculture and Water Resources, Geoscience Australia, the Commonwealth Science and Industrial Research Organisation, and the Department of Industry, Innovation and Science together developed the Location Integration Capability known as Loc-I. Loc-I is designed to streamline and quality assure the integration of datasets based on location. See http://locationindex.org/linked.html
3. Protari - in collaboration with Data61, the ABS conducted user trials of Protari. Protari is an Application Programming Interface that enables the generation of insights from tabular data in a way that preserves privacy, is secure and fosters innovation. As a part of the trial, approved Protari users generated tables from the MADIP Basic Longitudinal Extract using the 2011 and 2016 Censuses. See https://www.abs.gov.au/protari
4. On-site DataLab - the number of analysts using the secure on-site DataLab for deep research rose to over 1,000 as a result of DIPA as well as increased use for other projects. The ABS improved the DataLab with additional statistical software and increased computing power.
5. Cloud DataLab - the ABS commenced a project to build a new, secure cloud DataLab for analysts to access integrated data products. The new DataLab will provide a more streamlined research experience in a secure, scalable and flexible environment. The platform will be used by researchers from across government, as well as by academics and other authorised researchers.
6. Person linkage spine - the person linkage spine, that provides the central index around which person-centred linkage is managed, continued to mature. An annual refresh process, including the addition of migration information, was instituted.
7. Integrated data platforms - the ABS continued to develop scalable data linkage, and content and assembly platforms; for use by specialist staff. These platforms ensure integrated data is stored and managed securely, and enables the more efficient creation of integrated person-centred datasets.

These projects supported increased access to, and use of, integrated data for policy and research. Further developments will continue to streamline data integration while maintaining privacy and security, creating high quality statistical products, and providing access in a scalable, useable and secure way.