Co-managed by the WCO and the IMF, the first International Survey on Customs Administration (ISOCA) was finalized in June 2020 and a report presenting some of the results was released in November 2021. Whilst a higher number of participants is required for the Survey to provide a global view of the roles Customs administrations play and their practices, this first edition served as a proof of concept, demonstrating the feasibility of collecting data on Customs to support analysis and research at the national and international level.
The International Survey on Customs Administration (ISOCA) was created to collect quantitative and qualitative data on Customs administrations and to enable comparisons to be made between countries sharing common features (e.g. income level or gross domestic product; landlocked or coastal; geographic region; trade volume; population; size of the administration and mandates). It is co-managed by the World Customs Organization (WCO) and the International Monetary Fund (IMF).
ISOCA covers most aspects of an administration, such as Customs revenues by type, details on offences, time required to release goods, cost of collection, performance standards, workforce characteristics, general management, border cooperation arrangements, automatic exchange of information, technologies for goods in transit, mobile teams, duty and tax exemptions.
As such, the data collected through ISOCA can be used to:
- (i) support strategic planning, communication, reporting to national authorities, advocating for investment, and improving resource allocation;
- (ii) understand historical performance;
- (iii) establish performance baselines;
- (iv) develop benchmarks to identify additional powers, trends in support function policies, composition and skills of human resources;
- (v) provide sufficient data to facilitate research; and
- (vi) identify specific needs for technical assistance and capacity building.
The inaugural round of the Survey was launched in June 2019 and finalized in June 2020, and the data collected correspond to fiscal years 2016 and 2017.
There was strong initial commitment by Customs administrations to participate in the Survey, with 102 responding positively to the invitation of the WCO and the IMF (93 of them were WCO Members). However, 30 of them did not complete the Survey and 21 did so only partially. A number of reasons were advanced to explain the lack of response. For example, while data collection began prior to COVID-19, some administrations reported remote work as a challenge in collecting the required data. The high abandon rate can also be explained by the lack of knowledge of the possibilities offered by the data reporting system. For example, some coordinators did not share data entry responsibilities by delegating the filling of the forms online. In the end, only 51 administrations provided submissions meeting minimum quality standards for both fiscal years (46 of them were WCO Members).
The entire data collection and review process took more than a year. Reminders were sent and extensions provided to participants. Innovative online guidance tools and mechanisms were developed to facilitate completion of the Survey, such as the possibility to delegate the responsibility of collecting and entering data in the online data collection platform.
The no-answer rate for some questions requiring numerical responses was extremely high, which may be indicative of a difficulty in some administrations in obtaining the data and suggest limitations in their data architecture, data management and reporting systems.
“Some of the requested data were not recorded in a harmonized manner in our systems and not all were readily available in our Customs Management System. We had therefore to consolidate missing data from different internal and external sources.”
Mauritius ISOCA Coordinators
Despite participation being lower than expected, the data collected represent the most comprehensive available set of data on Customs administrations and their practices. While there is clearly a need for more Customs administrations to participate in the future, this first Survey serves as a “proof of concept” for the collection of Customs administration data.
In November 2021[1], the WCO and the IMF released a report presenting some of the first Survey results. Results were grouped into three categories: performance-related data; profile data; and data on practices and institutional structure. In addition, depending on the issue addressed, Customs administrations were often grouped in one of two ways:
- by standard grouping: small states (population less than 1.5 million, all income levels) / lower income / higher income; and
- by geographic location: island state / landlocked country / other countries.
The report is available online[2] and we invite readers interested in the results to consult it.
“We did not incur any additional cost to participate in the Survey. We were able to provide the data that was sought by the Survey without undue trouble. We do expect that as the group of participants broadens the data will potentially be more useful to us for further analysis and decision-making.”
ISOCA Coordinator from Ireland
The first round of the Survey should be seen as a proof of concept demonstrating the feasibility of collecting Customs administrations’ data
Although the database established during this first edition of the Survey presents opportunities for improvement, the Survey itself served a critical purpose in providing a “proof of concept” for the collection of Customs administrations’ data. It needs to be seen as a “prototype”, or a demonstration model which gives an idea of the kind of aggregate analysis that could be done if more countries were to participate in the Survey.
“Whilst the data collected through the ISOCA Survey could not be used for benchmarking purposes due to the low response rate, the first round of the Survey showed that the data collection process itself was feasible. The National Tax and Customs Administration of Hungary has supported and will continue to support this initiative.”
ISOCA Coordinator from Hungary
A survey similar in design and purpose to ISOCA, the International Survey on Revenue Administration (ISORA), is the product of the collaboration of five international organizations to collect data on tax administration practices, structures and performance directly from tax administrations[3]. Some of the lessons learned in the development of ISORA were taken into account during ISOCA’s development.
One of the lessons is that, despite intensive preparation and efforts, a survey of this type, covering all countries from the smallest to the very largest, inevitably needs live, comprehensive testing. Even though three of the organizations managing ISORA had been administering separate surveys to collect data on tax administrations – which actually informed the design of ISORA – a major revision of the survey was undertaken after two iterations to improve data quality. A simplified and shortened set of questions to be answered on an annual basis was compiled, which was supplemented by a periodic questionnaire. Despite data collection taking place in 2020 while staff in many administrations were working remotely, 156 administrations completed this streamlined third ISORA. The data from the 2020 survey have been released publicly for the first time, and data collection from the fourth survey will soon be complete.
Another lesson is that participation increases during a second round. One of the reasons is that second-time participants found the data reporting system easier to use. Moreover, the organizations managing the survey realized that supporting participating administrations remains essential, even in an established survey.
“Participating in the ISOCA Survey enabled us to self-evaluate our performance in a number of key areas. It is a timely and important initiative, however the data collection was time-consuming because it was the very first exercise of this kind and there is clearly room for improvement.”
ISOCA Coordinator from Georgia
Providing assistance and reviewing data submission is key
Taking into account lessons learned when undertaking ISORA, a WCO-IMF Technical Working Group was established to provide assistance, answer questions and conduct data quality review in liaison with each Survey participant’s representatives. The latter often provided data that did not match data their administrations had already provided elsewhere. Reviewing the data was essential to ensure that any differences compared to other surveys or reports could be explained, and to ensure coherence across datasets and across countries. Elementary data items have to be based on compatible concepts, definitions and classifications so that they can be meaningfully combined and can be compared over time.
More time was required to complete the Survey due to the COVID-19 pandemic but the crisis also shed light on the importance of data
The COVID-19 pandemic increased the length of time needed by countries to provide information required in the Survey. This raises fundamental questions in terms of the sustainability of the data policy when priorities and resources are shifted towards relatively short-term but urgent tasks. We might be tempted to think that, at periods of crisis, there is no time for general assessment and benchmarking. But one should not forget that the implementation of a robust and coherent data strategy can assist the organization in ensuring organizational resilience, in other words, in responding and adapting to incremental change and sudden disruptions.
While delaying the collection of data that were not considered critical to the response to COVID-19, the pandemic has also increased interest in data analysis, highlighting the need to develop data interpretation skills at all staff levels. It has also shed light on the importance of data collection and interpretation, and the danger of incomplete information and biased data, especially in comparative settings[4].
Some administrations need to strengthen data collection capacity and data sharing
ISOCA also highlights the limited interest in sharing data or the lack of hierarchical support to do so. Some very modern and well-equipped administrations did not submit data, or did so in such a limited way that it could not be used by the analysts.
There seems to be a need to develop a stronger data culture in some administrations and to develop capacity to produce key indicators as part of day-to-day operations, enabling them to measure their performance and to guide their strategic, tactical and operational decisions.
“At times it was difficult to complete the Survey and I couldn’t answer some of the questions due to the fact that my Administration did not have an updated database. This Survey enabled me to show my superiors that there is an urgent need to equip ourselves with a number of essential tools to ensure the smooth running and modernization of our Administration”.
ISOCA Coordinator from Niger
Greater awareness of the benefits of the Survey should drive administrations’ engagement in the process.
The WCO and the IMF hope that, in the future, more Customs administrations will join this common effort to collect quality data for the benefit of the entire Customs community. The two Organizations are aware of the need to strengthen their engagement strategy and to ensure that the respondents assume ownership of the process, in view of its anticipated benefits. A stakeholder engagement strategy is also needed to ensure not only data coherence, but also data usability and accessibility.
Although a higher number of participants is required for the Survey to be meaningful, the WCO and the IMF now know that the tool developed to collect the data operates efficiently. The two Organizations are also aware of the need to simplify the ISOCA Survey to balance the need for accurate data and the burden of data collection, and to better exploit the benefits that the online platform offers.
Their objective is to integrate the expectation of Customs administrations regarding the use of data at the design stage of the Survey, thus ensuring that the collected data are relevant and can improve decision-making. Metrics must meet criteria in terms of legitimacy, functionality, and technical validity to ensure their internal adoption and fitness for purpose.
To achieve this, the WCO and the IMF need to garner the support of some Customs administrations. Any administration interested in partnering with them in this process is therefore warmly invited to contact them.
More information
Isoca@wcoomd.org
fadrafit@imf.org
[1] The report can be consulted at http://www.wcoomd.org/en/about-us/partners/publications.aspx and https://data.rafit.org/?sk=8234d071-b89c-4876-8bba-791b9a33ddfe&sId=1635259677768.
[2] http://www.wcoomd.org/en/about-us/partners/publications.aspx or https://data.rafit.org/?sk=8234d071-b89c-4876-8bba-791b9a33ddfe&sId=1635259677768
[3] The five organizations are: the Asian Development Bank (ADB), the Inter-American Center of Tax Administrations (CIAT), the International Monetary Fund (IMF), the Intra-European Organisation of Tax Administrations (IOTA), and the Organisation for Economic Co-operation and Development (OECD). CIAT, IOTA, the IMF and the OECD signed an MOU in 2016. Ahead of the second round of ISORA, the ADB joined the partnership.
[4] George, Bert & Verschuere, Bram & Wayenberg, Ellen & Zaki, Bishoy. (2020). A Guide to Benchmarking COVID‐19 Performance Data. Public Administration Review. 10.1111/puar.13255.
