Introduction
The ‘Northern Ireland Road Network and Condition Statistics’ report is a new report first published in 2023 by Analysis, Statistics and Research Branch (ASRB) within the Department for Infrastructure (DfI, the Department).
DfI has responsibility for the road network in Northern Ireland. Under Article 8 of the Roads (Northern Ireland) Order 1993, the Department has a statutory duty to maintain the province’s road network.
This report is based on data provided by the Department to ASRB and the current plan is to publish on an annual basis in Summer / Autumn; this will be reviewed in line with user requirements and available resources.
Quality assurance of administrative data (QAAD) report
In 2015 the UK Statistics Authority published a regulatory standard for the quality assurance of administrative data. The standard was developed in response to concerns about the quality of administrative data and in recognition of the increasing role that such data is playing in the production of official statistics. The standard encourages risk based judgements and supports a proportionate approach.
The standard is supported with an Administrative Data Quality Assurance Toolkit which provides useful guidance to statistical producers about the practices they can adopt to assure the quality of the data they utilise.
In accordance with this toolkit, ASRB have carried out a quality assurance of administrative data (QAAD) assessment on selected data from the RMCS and Account NI systems – the administrative data sources used to produce a selection of the statistics included in ‘Northern Ireland Road Network and Condition Statistics’ report.
This QAAD will focus on the following:
- Road Lengths (RMCS)
- Surface Defects Instructed and Repaired (RMCS)
- Public expenditure on NI roads (Account NI)
(Note: - the ‘Northern Ireland Road Network and Condition Statistics’ report also includes survey data (outside of the scope of this QAAD – please see below)).
The diagram below shows the sources of data that are used in the production of the “Northern Ireland Road Network and Condition Statistics” report: -
Further information on the quality of the other data sources used to produce the ‘Northern Ireland Road Network and Condition Statistics’ report is available in the accompanying Background Quality Report and Technical Report.
The Administrative Data Quality Assurance Toolkit provides helpful guidance to statistical producers about the practices they can adopt to assure the quality of the data they utilise. The purpose of this paper is to document the administrative data sources that are utilised in the production of the Northern Ireland Road Network and Condition Statistics report and to outline the results of the quality management actions that have been undertaken for assuring the suitability of the data sources for this purpose.
Use of Data
ASRB use the data extracted from RMCS to produce statistics included in the ‘Northern Ireland Road Network and Condition Statistics’ report. This report was published for the first time in 2023 and will be published on an annual basis thereafter.
In line with the ASRB User Engagement Strategy, the content of the publication is informed by an understanding of customers’ information needs. The data presented in the publication aims to meet the needs of both internal and external users.
A user engagement exercise carried out on Northern Ireland Transport Statistics identified an interest in the statistics formerly produced within Chapter 1 of the Transport Statistics bulletin. This survey, along with stakeholder engagement, a Northern Ireland Audit Office Report (Structural Maintenance of the Road Network), and a report by Jim Barton on the review of structural maintenance funding in 2018, as well as the volume of queries received on road conditions highlighted that there is a need to publish robust statistics on the road network in Northern Ireland.
All users on the Transport Statistics distribution list were notified of the development of a new Road Network and Condition Statistics report and asked to opt-in to the new distribution list if interested in this topic. The first report was also circulated to these users who have indicated an interest in the statistics and further engagement will take place now that the report has been published.
The report is highly relevant to DfI Transport and Road Asset Management internal customers and the statistics on road lengths and surface defects repaired will be used to respond to Assembly Questions and ad hoc queries on these subjects. At an overall level there is likely to be limited media interest, with interest limited to a niche user base.
As the producers of the statistics, ASRB meet regularly with key suppliers and internal users of the statistics. Following the publication of the first report further user engagement will be undertaken. For general ad hoc/bespoke customers, ASRB engage when providing a query response by including a link to a short survey welcoming their feedback on the service provided. Results to this survey are collated and analysed on a quarterly basis.
Section 1 - QAAD assessment of Surface Defects Repaired, Surface Defects Instructed and Road Lengths data from the Road Maintenance Client System (RMCS)
Data supplier
DfI Transport and Road Asset Management
Operational context and administrative data collection
The RMCS is managed by DfI Transport and Road Asset Management. DfI Transport and Road Asset Management is a business unit within the Department for Infrastructure playing a significant role in facilitating the safe and convenient movement of people and goods throughout the province and the safety of road users, through the delivery of road maintenance services and the management and development of the transport network. It also informs the Department's policy development process to ensure that measures to encourage safe and sustainable travel are practical and can be delivered.
The DfI Transport and Road Asset Management headquarters is in Belfast and there are four divisional offices in Belfast, Coleraine, Craigavon and Omagh. Each division is divided into a number of sections that generally correspond to district council areas. Separate Consultant and Contractor units have their headquarters in Downpatrick and Ballymena respectively.
Data Type
Summary data in tabular format.
Data Content
Annual data for surface defects instructed, surface defects repaired and road lengths from 2019/20 to the latest financial year available.
Road lengths data are provided by the following road classifications along with urban (speed limit of 40 mph or less) and rural (speed limit of more than 40 mph) breakdowns.
- Motorway - A normally dual carriageway road designed for higher speed traffic with designated places for joining and leaving.
- Trunk Road Network (TRN) - Motorways and all purpose trunk roads owned by DfI who have responsibility for maintenance and operation. These are strategic roads with a high proportion of long distance traffic although some trunk roads may also have lengths, with the same number, designated as an ‘A’ principal road, where traffic is predominantly local in nature.
- A roads - major roads intended to provide large-scale transport links within or between areas.
- B roads - roads intended to connect different areas, and to feed traffic between A roads and smaller roads on the network.
- C roads - smaller roads intended to connect together unclassified roads with A and B roads.
- Unclassified roads - local roads intended for local traffic.
The figures only cover adopted roads i.e. public roads which are maintained by DfI Transport and Road Asset Management. Slip road route lengths add up to approximately 19km but are not included in motorway route lengths. Data also exclude car parks and footpaths.
In 2006 and 2007, the Department awarded two Public Private Partnership (PPP) contracts for the Design, Build Finance and Operate (DBFO) of small sections of motorway and Trunk ‘A’ roads. As part of the agreement, the Department also transferred the responsibility for maintenance of about 90% of all motorways, along with relevant sections of Trunk ‘A’ roads to the private sector.
The surface defect figures (both instructed and repaired) include details of the number of carriageway, hard shoulder and layby surface defects (which includes potholes, cracking, depressions etc.). Data on footways and maintained laneways are not included. Defects such as white lining, deflective signage etc are also not included.
Each defect is prioritised dependant on the severity and risk to the public according to the following classification: -
- R 0 – Repair or make safe as soon as practical
- R 1 – Repair or make safe before the end of the next calendar day
- R 2 – Repair or make safe within 5 working days
- R 3 – Only Defects on low traffic, rural roads and medium and low traffic (high risk) footways to be repaired within 4 weeks
- R 4 – Repair during the next available programme or by target response time as specified, or review condition at next inspection
There are some R3 defects that are not included in the R3 definition above and therefore not recorded until such times as the next inspection if they meet intervention levels as detailed above. These are not included in the Road Network and Condition Statistics report.
The surface defect figures in the Road Network and Condition Statistics report, cover defects that are classified as either R 1, R 2 or R 3. Users should also note that surface defects are only included in this report following an inspection by DfI.
The surface defect figures in the report cover defects that are classified as either R 1, R 2 or R 3. It should also be noted that surface defects are only included in this report following an inspection and therefore this is not inclusive of all roads that haven’t been inspected.
Data on surface defects instructed and repaired are provided by Local Government District (LGD). However, it should be noted that locations of LGDs do not coincide exactly with boundaries used by DfI Transport and Road Asset Management for road maintenance purposes.
Data Supply and communication with Data Suppliers
Once all the survey data have been collated and checked by the Road Asset Maintenance branch, they are provided to ASRB. These are supplied in the summer but it is hoped that in the future survey data can be supplied in a more timely way.
The Department for Infrastructure are the data owners for surface defects repaired, surface defects instructed and road lengths.
ASRB staff are regularly in touch with their data suppliers in DfI Transport and Road Asset Management. This communication can range from informal telephone/email queries to ASRB staff attending formal meetings with DfI Transport and Road Asset Management. A timeline for future data supply will be developed by ASRB in collaboration with data suppliers. This is based on knowledge and insight gained in the development of the first edition of the NI Road Network and Condition Statistics report. Any future problems experienced with the timing of or completeness of the annual tables will be raised by ASRB staff with colleagues within DfI Transport and Road Asset Management as soon as they come to light.
Data Source Information
Road lengths - data on road lengths is extracted from the RMCS.
The road lengths are the digital measurement of roads presented in kilometres. The figures for road lengths only cover adopted roads i.e. public roads which are maintained by DfI Transport and Road Asset Management. Slip roads are not included in motorway route lengths. Data also exclude car parks and footpaths. New road lengths are agreed by both Land and Property Services and DfI Transport and Road Asset Management. The data on roads are a snapshot from the RMCS system at the start of each financial year (1 April).
Surface Defects Instructed and Repaired - Data on surface defects instructed and repaired is derived from the RMCS.
The engineers use a hand held device (tough book) linked to RMCS to record surface defects. The details of the surface defects are recorded by the engineer; surface defects are recorded using a range of defect categories, e.g. pothole, rutted, depressed, cracked, uneven, etc. (Please see the Technical Report for more details).
The defect is also prioritised dependant on severity and risk to the public. An inbuilt script is run within the RMCS system to prioritise the repairs required. Works orders are then issued to the operational maintenance team and/or outside contractors and when these jobs are completed, the works orders are returned to the relevant maintenance office and the surface defects repaired are recorded in the system.
The data extracted are the surface defects repaired within the financial year but users should note that these may have been reported prior to this financial year. Furthermore a small number of defects treated within the financial year may not have been updated on the system until after the end of the financial year.
The data are therefore not extracted from RMCS before the end of April to allow for the late entry of data. ASRB will monitor changes in previous financial years’ data to determine if approximately a four-week lag in extracting the data is sufficient.
DfI Transport and Road Asset Management advise that lower priority surface defects may be recorded for future planning purposes and therefore may not be instructed for repair. In addition, due to funding and resource constraints throughout the year there is a backlog of instructed defects incomplete; the statistics in the report currently focus on the surface defects repaired during each financial year.
Quality Assurance Principles, Standards and Checks Applied by Data Supplier
A new RMCS was introduced in January 2014. RMCS is a mapping based data management system. It is an administrative system with full NI coverage and incorporates various validation checks. As the system was specifically designed and built for roads management, it means it is tailored to meet the needs of DfI.
The system is designed to allow DfI to manage its network in terms of defects, claims, survey data, scheme generation, GIS mapping, lighting system management etc.
Divisional database administrators update and maintain the RMCS database and the data are extracted for ASRB from the system by the Road Asset Maintenance branch at DfI headquarters. Only specific members of staff have password protected access to the system allowing them to edit only the appropriate areas of interest. The system has inbuilt checks to minimise manual entry errors; a suite of management information reports can be run by the RMCS team which assists in identifying any input errors. As a mapping based system, maps, videos and photographs and linking by geo-referencing allows administrators to check the data entered by looking at the visuals and checking the co-ordinates.
There is also an annual IT health check carried out on RMCS by an external provider - this covers areas such as data security, storage management & protection, system access, defence against hackers/outside IT threats etc.
Producer’s Quality Assurance investigations and documentation
ASRB staff initially met with representatives of DfI Transport and Road Asset Management to get a demonstration of the system in operation and to discuss the quality assurance procedures in place.
ASRB applies an additional series of quality assurance checks to the data tables they receive from RMCS.
As mentioned previously, the various data tables are mostly transferred to ASRB on an annual basis. Each table is deposited by DfI Transport and Road Asset Management in a secure container on Content Manager* (CM) that only ASRB staff can access. Each file is then opened and the parameters checked to ensure they are correct before the file is then moved to the relevant folders on the ASRB secure server for further analysis and to prepare for publication.
*Content Manager (CM) is the Northern Ireland Civil Service’s internal software platform for its corporate electronic document and records management.
ASRB carry out a range of checks on the data which include ensuring the titles of tables are checked and table data are checked for any missing values. Figures in tables are compared with data from previous years and totals in tables are checked. Further checks ensure that table formats and content are in line with the original request. Additionally, notes to the tables are checked against previous years and any gaps in notes are identified. Any significant and unexplained increases or decreases which occur are investigated with the data supplier and either confirmed or corrected. Finally, time-series comparisons are prepared as an indication of whether the data are consistent with historical trends or not.
Any anomalies or errors identified as a result of these checks are referred back to DfI Transport and Road Asset Management so if appropriate they can be corrected at source.
Strengths and weaknesses / limitations of the Administrative Data sourced from RMCS
Strengths
- The information is captured on RMCS, an IT system specifically designed to manage and record information on roads.
- The data is for all of Northern Ireland, providing full coverage of the lengths and defects of the adopted roads in NI.
- DfI has comprehensive guidance for staff on how to input details – to ensure that all staff record details accurately and on a consistent basis.
- All staff receive regular training on use of the system and how to access the system, and access to WDM support desk for any trouble shooting. A suite of management information reports ran routinely enable DfI Transport and Road Asset Management to ensure that the figures are accurate and robust.
- As the information on RMCS is used for operational purposes, errors in data/reporting can be identified at the operational stage. Any emerging anomalies or discrepancies are then investigated, and data corrected where required.
- Regular data validation checks and quality assurance checks are run by DfI Transport and Road Asset Management to identify and correct any missing values or inaccuracies in the data.
• Data sourced from RMCS is collected and recorded on a similar basis to other parts of the UK.
- The system is considered to be fit for purpose by those who use and maintain it daily.
- There is no hard cost to ASRB for obtaining the data.
- There is minimal burden on the data suppliers as the data are already collected for operational purposes.
- Efficient use of data already available.
Weaknesses/Limitations
- The RMCS is dependent on staff accurately inputting road details onto RMCS; as with most administrative systems there may be some potential for manual entry errors.
- As the data extracts are from a live system; they are a snapshot at a point in time.
- There may be delays in updating the system which means statistics for the more recent time periods may vary.
Section 2 - QAAD assessment of Public Expenditure data on NI Roads from Account NI
Data supplier
DfI Finance Business Partner.
Operational context and administrative data collection
Account NI provides one streamlined service from purchasing through to payment of invoice for all government departments. Account NI processes over 20,000 invoices per month.
Account NI are keen to work closely with suppliers to the departments in order to provide the best service possible and facilitate prompt payment of invoices. Account NI was introduced to improve the delivery of financial transaction processing services across the NI Civil Service.
Access to the Account NI application is password protected and users can only access specific services relevant to their permissions. Upon accessing the portal users are reminded that they are accessing a computer network owned by the Department of Finance (DoF) and unauthorised access or misuse is forbidden and will constitute an offence under the Computer Misuse Act 1990.
Data Type
Summary data in tabular format.
Data Content
Annual breakdown of data on Public expenditure on NI roads from 2019/20 to the latest financial year.
Data on Public Expenditure are supplied according to the following categories: -
- New construction and improvement (includes major and minor capital road schemes, land costs, street lighting - new installations, sustainable travel schemes such as cycling and pedestrian measures, park & share and improvements to structures such as bridge strengthening).
- Structural Maintenance-Capital (includes resurfacing and reconstruction, asphalt patching, surface dressing, capital investment in drainage infrastructure and capital investment in the structural stability of embankments).
- Structural Maintenance-Resource (includes patching of carriageway and footways).
- Highway Structures (includes maintenance of bridges and retaining walls)
- Routine Maintenance (includes environmental maintenance (grass cutting, weed control, tree maintenance etc.), gully emptying and cleaning, fencing and minor repairs to embankments, cuttings and verges).
- Traffic Maintenance (includes road markings, safety fences & guardrails, energy to illuminate traffic signs and signals and maintenance of traffic signs and signals)
- Winter Maintenance (includes salting and snow clearance)
- Public Lighting (includes maintenance and energy)
- Park & Ride/Share Sites (does not include the cost of Decriminalised Parking Enforcement)
- Other (this is expenditure which is not detailed in the categories above such as some administration overheads, liability costs, payments to the Departments Public-Private Partners (PPP) for their operating and lifecycle costs linked to maintaining the road network lengths constructed by them through Design Build Finance and Operate (DBFO) arrangements, Road Drainage charge from Northern Ireland Water, contractual costs associated with Enforcement of Parking and Moving Traffic offences, purchases of Fleet, IT and other Plant Property and Equipment (PPE) and Active Travel Grants (e.g. Greenway Grants)).
Data Supply and communication with Data Suppliers
Once all the expenditure data have been checked by DfI Finance Business Partner they are provided to ASRB. These are supplied in the autumn but it is hoped that in the future expenditure data can be supplied in a more timely way.
Summary data are provided annually by DfI Finance Business Partner to ASRB via a secure folder on Content Manager (CM) to ASRB by DfI Finance Business Partner in June, after financial year end.
ASRB staff are regularly in touch with DfI Finance Business Partner. This communication can range from informal telephone/email queries to ASRB staff attending formal meetings with them. A timeline for future data supply will be developed by ASRB in collaboration with DfI Finance Business Partner. This is based on knowledge and insight gained in the development of the first edition of the NI Road Network and Condition Statistics report. Any future problems experienced with the timing of or completeness of the annual tables will be raised by ASRB staff with colleagues within DfI Finance Business Partner as soon as they come to light.
Data Source Information
The data source is the Account NI Shared Service solution for all NICS Departments. DfI Finance Business Partner Team extract the data from the Account NI system by means of an Account NI built report.
Quality Assurance Principles, Standards and Checks Applied by Data Supplier
Data on Public Expenditure on NI roads is cross checked to ensure that the balance reported per year matches the output from the Account NI system.
DfI’s accounts are audited externally biannually, and the information used to produce the data for ASRB is cross checked with the final set of accounts.
DfI Internal audit also provide assurance of the correct use of the Account NI system.
The data provided to ASRB reflect the position of the final year end accounts which are signed off by Audit.
The final figures provided to ASRB are held in a secure CM container with restricted access for agreed individuals from DfI’s Finance Business Partner Team and ASRB.
Producer’s quality assurance investigations and documentation
ASRB applies an additional series of quality assurance checks to the data tables they receive from Account NI.
As mentioned previously, the data tables are transferred to ASRB on an annual basis. Each table is deposited by DfI Finance Business Partner on a CM container that only ASRB staff can access. Each file is then opened and the parameters checked to ensure they are correct before the file is then moved to the relevant folders on the ASRB secure server for further analysis and to prepare for publication.
ASRB carry out a range of checks on the data including ensuring that titles of tables are checked and tables are checked for any missing values. Figures in tables are compared with data from previous years and totals in tables are checked. Further checks are made to ensure that table formats and content are in line with the original request. Additional checks are made on the notes to tables against previous years and any gaps in notes are identified. Any significant and unexplained increases or decreases which occur are investigated with the data supplier and either confirmed or corrected. Finally, time-series comparisons are prepared as an indication of whether the data are consistent with historical trends or not.
Any anomalies or errors identified as a result of these checks are referred back to DfI Finance Business Partner so if appropriate they can be corrected at source by the data supplier on their IT systems.
Strengths and weaknesses / limitations of the Administrative Data sourced from Account NI
Strengths
- The information is captured on Account NI, an IT system specifically designed to record expenditure.
- DfI Finance Business Partner has comprehensive guidance for staff on how to input details – to ensure that all staff record details accurately and on a consistent basis.
- Senior managers monitor public expenditure across the NI road network on a continuous basis and help ensure that the figures are accurate and robust. Any emerging anomalies or discrepancies are investigated, and data corrected where required.
- Regular data validation checks and quality assurance checks are run by DfI Finance Business Partner to identify and correct any missing values or inaccuracies in the data.
- The key figures for Public Expenditure on roads are recorded on a similar basis to other countries in the UK.
- Levels of control such as Delegation Limits, Hierarchy authorisation etc., are built into Account NI and these would align with the Departments Accounting Procedures manual.
- The data are sourced from an administrative data system and therefore are a complete record of public expenditure on all roads in NI (i.e., are not based on a sample of roads).
- The system works well and is trusted by the staff who use it. It is reliable and needs minimal maintenance.
- The system is considered to be fit for purpose by those who use and maintain it daily.
- Both internal and external audits are carried out on an annual basis.
- There is no hard cost to ASRB for obtaining the data.
- There is minimal burden on the data suppliers as the data are already collected for operational purposes.
- Efficient use of data already available.
Weaknesses/Limitations
- Account NI is dependent on staff accurately inputting details onto Account NI; as with most administrative systems there may be some potential for manual entry errors.
- As the data extracts are from a live system; they are a snapshot at a point in time.
- There may be delays in updating the system which means statistics for the more recent time periods may vary. Until such times as the Accounts are laid before the Assembly, data is subject to change.
Assessment of Level of Assurance Requirements (RMCS and Account NI)
ASRB assess that the level of risk of quality concerns in these sources is low at present and that the public interest is low. This suggests that the appropriate level of assurance required for these statistics is A1 - Basic Assurance.
This means that ASRB have used the QA toolkit and have reviewed the administrative source and published a summary in this document.
ASRB came to this decision as the RMCS and Account NI data are considered robust, but at published levels are of limited public interest.
ASRB will continue to assess the level of assurance and update if necessary.
ASRB therefore plan to review the QAAD on an annual basis to ensure that the assessment of the level of assurance requirements is accurate and up-to-date.
The next review is scheduled for Summer / Autumn 2024.
Assessment of User Needs (RMCS and Account NI)
The RMCS and Account NI administrative sources in relation to the data included in the “Northern Ireland Road Network and Condition Statistics” are considered to be of high quality. The system processes, the supplier checks and additional validations carried out by ASRB ensure the robustness of the data. Although there are some weaknesses ASRB consider that these data are fit-for-purpose.
Document last updated: December 2023
Next review due: Summer / Autumn 2024
Annex A UK Statistics Authority Quality Assurance of Administrative Data Toolkit
The assessment of the RMCS and Account NI data sources has been carried out in accordance with the UK Statistics Authority Quality Assurance of Administrative Data Toolkit.
Each administrative data source investigated has been evaluated according to the toolkit’s risk and profile matrix (below), reflecting the level of risk to data quality and the public interest profile of the statistics.
|
Level of risk of quality concerns |
Public interest profile |
||
|
Lower |
Medium |
Higher |
|
|
Low |
Statistics of lower quality and lower public interest [A1] |
Statistics of low quality and medium public interest [A1/A2] |
Statistics of low quality and higher public interest [A1/A2] |
|
Medium |
Statistics of medium quality and lower public interest [A1/A2] |
Statistics of medium quality and medium public interest [A2] |
Statistics of medium quality and higher public interest [A2/A3] |
|
High |
Statistics of higher quality and lower public interest [A1/A2/A3] |
Statistics of higher quality and medium public interest [A3] |
Statistics of higher quality and higher public interest [A3] |
A risk/profile matrix for RMCS and Account NI is included below. The matrix reflects the level of risk of data quality concerns and the public interest profile of the statistics determined by a review undertaken by the ASRB Transport Statistics team using the information contained within the UK Statistics Authority’s Administrative Data Quality Assurance Toolkit.
Risk Profile matrix of ASRB Statistics when QAAD Toolkit applied
|
ASRB Statistic |
Data Quality Concern |
Public Interest |
Risk Profile Rating |
|
Surface Defects Repaired (RMCS) |
Low |
Low |
A1 |
|
Surface Defects Instructed (RMCS) |
Low |
Low |
A1 |
|
Road Lengths (RMCS) |
Low |
Low |
A1 |
|
Public expenditure on NI roads (Account NI) |
Low |
Low |
A1 |
A1 = Basic assurance, A2 = Enhanced assurance, A3 = Comprehensive assurance
Further information regarding the allocation of matrix scores can be made available by contacting us using the details at the start of this document.