Written by Tengfei Wang Economic Affairs Officer, Trade, Investment and Innovation Division, UNESCAP Article No. 7 [UNCTAD Transport and Trade Facilitation Newsletter N°75 - Third Quarter 2017]
Following the UN/CEFACT Recommendation No. 42 on Trade and Transport Facilitation Monitoring Mechanism (TTFMM)1, a whole set of baseline study reports of TTFMM in Bangladesh - comprising a main report2 and five subsidiary reports3 - have just been released. Two similar sets of reports for Bhutan and Nepal will be available soon.
This post aims to highlight some key findings and lessons learnt in the process of undertaking the studies. In doing so, I hope my reflections of the studies can provide some useful reference when similar work is carried out in the future. The points I put do not follow a particular order. However, I do want to place emphasis on the last point because the questions and issues probably go beyond the current studies and should be relevant to other international indicators and data collection.
"Worthwhile results come from hard work and careful planning" - John Wooden
Tremendous efforts were put to undertake the studies in Bangladesh, Bhutan and Nepal. Support from responsible ministries, government agencies and national trade facilitation committees of the participating countries are essential to complete the studies. In total, 11 meeting were organized to define the scope of studies, make study plans and check progresses against the plans, review the preliminary results and validate the final outcome. Over 300 individuals participated in the workshops or share their expertise, or both. Nine (9) field trips were taken. Several thousand emails were exchanged. The ADB and ESCAP staff, colleagues from international partners especially International Trade Centre (ITC) and the consults all went the extra mile to ensure the successful deliveries of the project.
The devil is in the details
Business Process Analysis Plus (BPA+)4 is the key methodology used for data collection and analysis in the reports. It is based on the BPA, supplemented with Time Release Study (TRS) and Corridor Performance Measurement and Monitoring (CPMM)5. From my experiences in applying BPA and working with different experts and consultants, my observation is that it is not difficult to understand the principles of BPA but difficult to master it. In particular, it is not easy to get all the small details right. Kindly mind you, do not underestimate these small details. In the end, all these small details such as the drawings, graphs and process description in the context of BPA make a difference in terms of overall quality of the reports. As Henry Royce put it, "small things make perfection, but perfection is no small thing". If you would like to become an expert on BPA, step 1 is to take the BPA online course6. If you invest your time and efforts now, you will receive dividends in the future when you implement a BPA-related project.
Time Release Study is great but a holistic approach is greater
The fact that Time Release Study (TRS) was developed by the World Customs Organization and is enshrined in the WTO's Trade Facilitation Agreement (see Article 7.6) probably speaks for itself the importance of this methodology. Granted, any reduction in border crossing time should be great news for the traders. However, border crossing needs to be reviewed in a broader context of whole trade process. The reports show that border crossing time in some cases account for 1%-2% of total trade time, which means that optimizing border crossing alone is not enough. Other trade procedures must be taken into consideration in order to improve the trade process. This explains why BPA+ methodology, of which TRS is an important component, is applied in these studies to provide more comprehensive observations of different trade procedures and processes.
Another essential piece of the puzzle
The indicators produced under the TTFMM effectively supplement the international trade facilitation and logistics indicators, which are reviewed in the ESCAP-OECD handbook7. The baseline studies produced useful indicators related to specific product import or export, specific trade corridors and border crossing points. These micro-data are detailed and useful to identify bottlenecks and support evidence-based policy making and reforms. For example, the studies show the average speed along the corridors is strikingly low. The average speed is below 30 km/h and, in some cases, about 10 km/h. measures certainly need to be taken to avoid unnecessary stops along the journey.
Two sides of the same coin
Diagnosis in the reports show that most proposed procedures to enhance trade facilitation are covered by the World Trade Organization's Trade Facilitation Agreement (WTO's TFA), indicating the importance of implementing TFA for advancing trade and transport facilitation in a country. On the other hand, the reports substantially add value to implementing TFA because they identifies trade facilitation measures that need to be administered in the short- and long-term, and therefore, support a country to prioritize the implementation if the country faces financial and human capacity constraints. The indicators generated in the reports would lay a solid foundation to monitor the effectiveness of implementation of trade facilitation measures (e.g., the number of documents required drops from 30 to 10 over time.) The reports show that a large number of documents need to submit manually and often repeatedly, which highlights the importance of paperless trade. In this respect, the countries are encouraged to implement paperless trade provisions in existing regional agreements8 and actively participate in the new Framework Agreement on Facilitation of Cross-border Paperless Trade in Asia and the Pacific9.
Data reliability and sustainability of data collection could be the Achilles' heel, and solutions
Probably, any statistics textbook will tell us that data "reliability" and "validity" are fundamental for any meaningful statistical analysis. In the process of designing and conducting the studies, I more and more realized the importance of data reliability. To elaborate, allow me to start with the basic definition of data reliability.
Data Reliability refers to the extent to which results are consistent over time and an accurate representation of the total population under study. If the results of a study can be reproduced under a similar methodology, then the research instrument is considered to be reliable10.
Three errors, among others, may lead to unreliable data:
Subject error (different results on different days)
Subject bias (try to please researcher)
Observer error and bias11
All three errors can be associated with BPA. In conducting BPA, data are collected through desk research and semi-structured interviews. The informants, depending on their perspectives, or the size and type of companies or organizations they work for, may provide different data and information.
The subject error can be relevant to data collection using the CPMM and TRS methodologies. For instance, some incidents may slow down traffic12. The data collected during the incidents may not correctly reflect average corridor and border crossing processes and efficiency.
Similarly, the subject bias may be relevant to CPMM and TRS. CPMM data are mainly collected by the drivers who work on the corridors. Some drivers might not want to report the actual journey or reasons for the delays if their faults caused the delays. In conducting TRS, all individuals involved may behave differently from their normal practice because they know their performance was being watched and recorded.
The studies took all possible mitigation measures to ensure data reliability. For the BPA component, a national validation workshop was organized to bring different stakeholders to review the preliminary results. In addition, the preliminary results were sent to a range of audience and stakeholders for a few rounds to invite any feedback. For CPMM, we asked the drivers to show any printed proof whenever possible to cross check the time of an activity (e.g., the receipt for a tollway shows the time and location of a vehicle). Similarly, for TRS, we tried to look at any historical data stored in the computer to compare the survey results.
Sustainability of data collection is another concern. Data collection under the project was possible mainly because of the resources mobilized under the project or the support from government. The question is how to ensure sustainability of data collection in the long term. After all, "monitoring" means long-term data collection and analysis, rather than a one-off activity.
I believe IT is the only way to ensure data reliability and main sustainability of data collection. For instance, if an IT or automation system can be designed to capture the information on the time and costs related to vehicle movements along the corridors and border crossing. A large amount of data can be automatically collected and recorded, which can be used for CPMM and TRS analysis at any time.
This argument is not hypothetical. In Asia and the Pacific, Singapore, Japan and Republic of Korea can carry out TRS at any time using the data stored in the computerized systems. Other countries in the region may not have the same sophisticated system but may start small. For instance, if the IT systems in these countries can record the time that a vehicle passing a few selected points (such as ports, dry ports, border crossing points), the average speed can be easily calculated. Similarly, if the IT system can record the time when the customs brokers submit the documents and the time the procedure is completed, the TRS of some procedures, if not all, can be easily conducted. The countries need to take a proactive approach in this respect. When designing and developing automation systems, data related to trade and transport facilitation monitoring should be included in the implemented system whenever possible.