Background Quality Report – Travel Survey for Northern Ireland (TSNI) Reports

It informs users about the quality of the information upon which they may be drawing conclusions and making decisions.

The report is structured around the five quality dimensions for statistical outputs (from the European Statistics Code of Practice, PDF (458 KB)). The UK Statistics Authority’s Code of Practice for Statistics requires that:

Q3.3 The quality of the statistics and data, including their accuracy and reliability, coherence and comparability, and timeliness and punctuality, should be monitored and reported regularly.

Introduction

The Travel Survey for Northern Ireland (TSNI) began in Northern Ireland as a continuous survey in 1999.  It is conducted over a calendar year (from January to December).  The sample size is relatively small and therefore three years of data would normally be combined to ensure a wide range of analysis can be carried out that is robust.  The first results were published in 2003 for the period 1999-2001.  The latest reports cover the 2021 time period.

For 2020 and 2021 data: There were a number of significant changes to the survey methodology from 2020 in response to the COVID-19 pandemic to ensure the data could continue to be collected safely.  It was decided that due to the methodology changes and because 2020 is an exceptional year, it would not be appropriate to combine 2020 data with previous years.  This was continued for the 2021 data which is also reported as a single year. This has limited the analysis that can be done as the base numbers for the analysis are limited by reporting only one year instead of the usual three years combined.

Information for the survey is collected using two methods: a computer interview and a paper travel diary.  Prior to April 2020, the interview was conducted face to face and the paper travel diary was left with the respondent for completion.  From April 2020, following the suspension of face to face interviewing, the interview was conducted over the telephone.  The paper diary was completed by the interviewer who received details of the respondent’s journeys over the telephone.  From January 2023, respondents were offered a choice of whether they wished to complete the interview over the telephone or in-person.

Analysis, Statistics and Research Branch (ASRB) in the Department for Infrastructure (DfI) receive SPSS databases containing the TSNI data from Central Survey Unit (CSU) in the Northern Ireland Statistics and Research Agency (NISRA) annually.  Up to 2017-2019, the databases covered the latest three-year period.  From 2020 onwards, the databases provided are for a single year.

The production timetable is dependent on when CSU sends the data and the ASRB work programme. Usually, ASRB spend around 1-2 months validating the data and establishing if weights are required. The Headline Report is produced and published around 1-2 months after data are finalised.  The Technical Report follows around 1-2 months later and the In-depth Report around 4-5 months after the Technical Report.  TSNI reports are published on the Department for Infrastructure website on the Travel Survey for Northern Ireland statistics webpage.

Information on the survey methodology and data quality are published in the annual TSNI Technical Report available from the Travel Survey for Northern Ireland statistics webpage.

Relevance

The degree to which the statistical product meets user needs in both coverage and content.

The survey is asked to a representative sample of the Northern Ireland population.  As the annual achieved sample is relatively small, 3 years of data would normally be combined to ensure the analysis carried out is robust.  This ensures that a wide range of analysis can be carried out.  For certain stable groups of the sample where it is established that there is a large enough achieved sample, it is possible to report limited information for shorter (one year) periods of time (see Accuracy and Reliability section for details of 2020 and 2021 data).

The TSNI Headline Report covers key statistics on personal travel such as average number of journeys and average distance travelled by mode of travel and by journey purpose.  This is produced and published as soon as possible after the data are finalised once ASRB have completed their validation process.

The In-depth Report includes the above as well as analysis from the household and individual questionnaires and, where achieved sample sufficient for robust analysis, breakdowns by age and sex, by urban-rural and by Local Government District (LGD).  Since 2016-2018, the report includes the following sections: ‘All travel modes’, ‘Walking’, ‘Cycling’, ‘Public transport’, ‘Driving, driver and passenger trips’, ‘Journey purpose’, ‘Travelling to work’, ‘Shopping’ and ‘Key tables by urban-rural’.

The reports have been adapted over the years to keep them relevant e.g. the inclusion of data to monitor Indicator 25 of the draft Programme for Government 2016-2021, analysis of new questions or options added to the survey, the removal of obsolete classifications, the addition of analysis by LGD following the introduction of the 11 LGDs.  Any changes to a publication from previous years are detailed within the report.

The In-depth Report was redesigned for the 2016-2018 edition following feedback from various user engagement activities (see below).  It was updated to reflect current statistical and data visualisation methods.  In addition, information was reorganised into themed sections so users can easily identify the data they are most interested in.

The TSNI reports are used by a wide variety of users for a range of purposes.  DfI uses the information to inform policy, set objectives and monitor performance in relation to travel in Northern Ireland, for example in the draft Programme for Government.  It provides information on areas such as active travel, safe and sustainable travel, accessible transport and equality monitoring.  TSNI data are also used in DfI’s Draft Road Safety Strategy annual statistical report and DAERA’s NI Environmental Statistics report.  Private companies may use the data to look at travel trends, academics for research and other government Departments and Agencies to inform cross-cutting policy and strategy development.

ASRB ensures that the TSNI reports remain relevant to users through user engagement meetings with key users of the survey.  A comprehensive user consultation was carried out in 2019.  Details of user engagement activities, results from the user consultation survey and a Statement of User Needs based on the feedback received have been published and are available from the TSNI user engagement section of the DfI website.

There is also a customer satisfaction survey, which includes a question on whether the publication meets their needs, sent out each time a report is released to collect feedback from users.

All these sources help to shape the information collected and the content of the reports.  In this way, the needs of the user communities are continuously monitored, and if appropriate and practical, acted upon.  For example:

• The redesign of the In-depth Report based on the results of the 2019 user consultation.
• New questions were added on hybrid working, as this became more commonplace during/after the COVID-19 pandemic, anticipating future user requirements.

Accuracy and Reliability

The proximity between an estimate and the unknown true value.

Data are from a sample survey i.e. not asked of the whole population of Northern Ireland.  The sample for the Travel Survey for Northern Ireland consists of a systematic random sample of addresses, which is designed to provide representative geographic coverage across Northern Ireland.  The sample is selected from the Northern Ireland Statistics and Research Agency Address Register.  Further details are given in the TSNI Technical Report available from the Travel Survey for Northern Ireland statistics webpage.

As estimates from a sample survey depend on the particular sample chosen, they may differ from the true values of the population.  This variance from the true population value is measured using a confidence interval.  The confidence intervals published for TSNI data are 95% confidence intervals.  This means there is a 95% probability that the true population value is contained within the range of values given.  Confidence intervals for the TSNI are published in the Technical Report available from the Travel Survey for Northern Ireland statistics webpage.

As the annual achieved sample is relatively small (785 households and 1,520 persons in 2021), three years of data would normally be combined to ensure the analysis carried out is robust. This ensures that a wide range of analysis can be carried out.  For certain stable groups of the sample where it is established that there is a large enough achieved sample, it is possible to report limited information for shorter (one year) periods of time.

For 2020 and 2021 data: It was decided that due to the methodology changes and because 2020 is an exceptional year, it would not be appropriate to combine 2020 data with previous years.  This was continued for the 2021 data which is also reported as a single year.  This has limited the analysis that can be done as the base numbers for the analysis are limited by reporting only one year instead of the usual three years combined.

When reporting differences in the publication, the data are tested to see if they are statistically significant at the 5% level. A reported increase or decrease means there is at least a 95% probability that there is a genuine difference between results and the difference is not simply explained by random chance or sample error.

A wide range of quality assurance checks are carried out when the TSNI databases are received to ensure data are complete and there are no inconsistent cases. Validation checks are also carried out on every piece of analysis to ensure they are in line with other results from the same data source and with previous trends e.g. any large increases or decreases are investigated. However, it should be noted that due to the COVID-19 pandemic and subsequent changes to survey methodology, 2020 and 2021 results are not directly comparable to previous years and caution should be taken in reaching any conclusions if making these comparisons.

Previous TSNI reports are revised if necessary and appropriate.  All revisions are conducted in line with ASRB’s revisions policy which can be found on the DfI’s Code of Practice for Statistics supporting statements webpage.

Timeliness and Punctuality

Timeliness refers to the time gap between publication and the reference period. Punctuality refers to the gap between planned and actual publication dates.

Data for the Travel Survey for Northern Ireland are for a calendar year e.g. data for 2021 refers to the time period 1^st January 2021 to 31^st December 2021.

Central Survey Unit send the data annually after they have completed their processing (including validation processes, the entry of information from the paper travel diaries etc.).  Up to 2017-2019, the databases covered the latest three-year period.  From 2020 onwards, the databases provided are for a single year.

The report production timetable is dependent on when CSU sends the data.  The 2022 databases were delivered 16 months after the end of the survey, although CSU expect to provide data in a more timely manner going forward.  The TSNI Headline Report is published as soon as possible after the data are finalised once ASRB have completed their validation process.  The TSNI In-depth Report contains more tables and detailed analysis and requires a longer production period.

If CSU supply the data in July, the production timetable should be as follows:

The TSNI Headline Report is usually published September to November depending on the level of validation required and checks on weighting requirements.  This is a time lag of just over 8 to 10 months between the end of the reference period and publication and 2 to 4 months between receipt of the data and publication.

The TSNI In-depth Report is usually published February to April in the following year, a time lag of just over 12 to 15 months between the end of the reference period and publication and around 7 to 9 months between receipt of the data and publication.

Due to the COVID-19 pandemic, the survey and reporting methodology have undergone a number of major changes recently (see2020 and 2021 Technical Reports available from the TSNI statistics webpage).  This has led to a delay in receiving the databases from our data supplier and ASRB have put extra data processing procedures in place e.g. checks to see if weighting is required.  Once the delivery schedule and data received become more consistent, we will investigate if the time lag can be reduced.

A delay to the delivery of the data will have a knock on effect, leading to a delay in the publication of the reports.

Publication dates for the next 12 months are available on the DfI website in the upcoming statistical releases section.  Closer to the date of publication, announcements are uploaded on the statistics release calendar of the gov.uk website (the link is for upcoming DfI publications).

For provisional publication dates, where the usual publication date has been delayed by more than a month, the delay is announced and explained on the TSNI statistics webpage.  If necessary, updates are given regularly.  On occasion, TSNI publications have had their finalised pre-announced date changed but never by more than two weeks.  This has been necessary, to ensure there is sufficient time to complete all the necessary data checks and analysis.  When this has occurred, the change and reason for the change has been published in upcoming statistical releases on the DfI website and the gov.uk website.

Accessibility and Clarity

Accessibility is the ease with which users are able to access the data, also reflecting the format in which the data are available and the availability of supporting information. Clarity refers to the quality and sufficiency of the metadata, illustrations and accompanying advice.

All the TSNI reports are available to download free of charge in PDF format from the Travel Survey for Northern Ireland statistics webpage of the DfI website.  The statistical tables are also available to download in MS Excel format and ODS (Open Document Spreadsheet) format, from each report’s webpage.  If requested, ASRB provide hard copies or other suitable media.

The publications are also accessible through the statistics release calendar on the gov.uk website (the link is for published DfI publications).

Essential information, guidance and notes are available in the User Information and Definitions sections which are included in the TSNI Headline and In-depth reports.  In addition, where necessary, explanatory footnotes are included in the tables.  Following the changes to the survey methodology due to the COVID-19 pandemic, an additional appendix has been added to clarify factors that users should take into consideration when interpreting 2020 and 2021 results.

There is also a TSNI Technical Report published annually which contains details of the survey methodology including any changes, details of any investigations carried out on the data, any changes to the reporting methodology/data analysis, confidence intervals and the survey questionnaire.

The reports contain contact details of the responsible statistician in case further information is required.  Additional ad-hoc analysis, where appropriate, is provided on request.

Coherence and Comparability

Coherence is the degree to which data that are derived from different sources or methods, but refer to the same topic, are similar.  Comparability is the degree to which data can be compared over time and domain.

The TSNI is the only source of information on how, over the region as a whole, people use different forms of transport to meet their travel needs as individuals or family groups in Northern Ireland.  As such there is no exact comparison with any other data sources. However, comparisons are made to ensure the demographic/geographic breakdowns (i.e. sex, age, Local Government District) are in line with the mid-year estimates of population.  Where a similar question is asked in another survey (e.g. car ownership) comparisons are made to check that data are consistent.

All surveys conducted by Central Survey Unit follow harmonisation guidelines produced by the Office for National Statistics (ONS).  This means that, especially for demographic data, questions are asked in the same way and are therefore comparable.

Data are available back to 1999-2001.  However, it should be noted that questions have been added to the survey over time so not all data are available back to 1999-2001.  Up to 2017-2019, comparisons could be made over time as there had been no fundamental change to the survey methodology.  When possible, the latest data are always compared with previous years’ data, where appropriate, and any significant changes are thoroughly checked and validated.

There were significant changes to the survey methodology from April 2020 in response to the pandemic to ensure data could continue to be collected safely.  Therefore 2020 and 2021 results can be compared to each other but are not directly comparable to those of previous years and caution should be taken in reaching any conclusions if making these comparisons.  More information is given in the “Notes on TSNI 2020 and 2021 data” Appendix in the relevant reports.

The survey is the same throughout Northern Ireland which means some limited information is available at Local Government District (LGD) level which are comparable.  However, the availability of these data are dependent on the achieved sample being large enough to carry out robust analysis and generally three years of data combined are required for these breakdowns.

The settlement classifications which are used to classify areas as urban or rural were updated in 2015.  Reports prior to 2013-2015 were based on the previous 2005 settlement classifications.  Checks were carried out in 2013-2015 comparing TSNI data split by 2005 urban-rural definitions with TSNI data split by 2015 urban-rural definitions.  There were found to be no significant differences.

The TSNI is based on the National Travel Survey (NTS), as used in Great Britain up to 2012 (NTS coverage changed to England only from 2013).  Therefore comparative data for Great Britain are only available up to 2012 and comparative data for England are available up to the present.

Trade-offs between Output Quality Components

Trade-offs are the extent to which different aspects of quality are balanced against each other.

It could be considered that there is some trade-off between ‘relevance’ and ‘accuracy and reliability’.  The sample size of the survey is small due to cost constraints.  This means that only limited analysis can take place for single years and for Local Government Districts and other sub-population breakdowns.

Up to 2017-2019, the majority of the analysis available is for three years combined allowing a wider range of analysis.  Following the survey methodology changes from 2020 due to the pandemic, the reported 2020 and 2021 data are only available for single years.  Further details are available in the “Coherence and Comparability” section.  TSNI data are only analysed if the achieved sample is large enough for robust analysis.

Assessment of User Needs and Perceptions

The processes for finding out about users and uses, and their views on the statistical products.

Data presented in this report help to meet the information needs of a wide range of internal and external users.  See “Relevance” section for more details of users and uses of the TSNI reports and data.  In addition to the reports, TSNI data are used to answer a number of ad-hoc queries throughout the year on a wide variety of transport and travel related topics.

ASRB have a mailing list of TSNI users who are notified each time a TSNI publication is issued.  This list is updated each time a report is published to ensure it is up-to-date and that the people on it still wish to receive information about TSNI publications.

A series of user engagement meetings were carried out in 2018 with internal and external users of the TSNI data to better understand the needs of our customers from inside and outside of the Department for Infrastructure.  These meetings established how they use the TSNI data, what is most useful and if there is any further travel data that would be helpful to them.  A summary of the meetings is available from the TSNI user engagement section of the DfI website.

A comprehensive user consultation was carried out in 2019.  A survey was issued asking how TSNI information is accessed, the types of TSNI information used and their opinion on proposals for future TSNI outputs.  The survey received 93 full and partial responses which have been analysed.  The results have been published and are available from the TSNI user engagement section of the DfI website.

There is an on-going ASRB customer satisfaction survey which includes a question on whether ASRB publications (including TSNI reports) meet customers’ needs.  The link to the survey is issued every time users are notified that a TSNI report has been published or an ad-hoc query is answered.  Results are collated on an annual basis.

User needs are prioritised, taking account of resources available and the feasibility of any additional needs identified.  A statement of user needs summarising the results of the various user engagement activities and containing a list of user requirements and actions to be taken is available from the TSNI user engagement section of the DfI website.

Performance, Cost and Respondent Burden

The effectiveness, efficiency and economy of the statistical output.

An annual survey control return is completed for the Travel Survey for Northern Ireland.  The compliance burden is calculated from the median time taken to complete the survey and the number of respondents.

The achieved response rate is a usually a good indication of respondent burden. However, response rates dropped across social surveys in general during COVID-19, particularly when interviewers were unable to call out to households.  The TSNI response rate was 59% in 2017-2019, a time period before the pandemic.  In 2020, which included data collected pre-pandemic and during the pandemic, it was 25%.  In 2021, it was 33%.

Confidentiality, Transparency and Security

The procedures and policy used to ensure sound confidentiality, security and transparent practices.

TSNI data received by ASRB are anonymised (name/address information is not contained in the database).  In addition, the data represent a sample of the population and ASRB cannot identify the sample that was chosen from the population.

The databases are held on a network that is only accessible to ASRB.

Analysis is only carried out if the achieved sample is large enough and the data are presented as averages or percentages.  There are therefore no small cells which need have disclosure control methods applied.

ASRB has produced a statement on its confidentiality processes which can be found in the Code of Practice for Statistics supporting statements on the Department for Infrastructure’s website.