Abstract
The Lao Statistics Bureau has launched the development of Statistical Information System on the Environment. With its integrated systems approach, incorporating multiple data sources within a unified analytical framework enabling an understanding of tradeoffs and synergies to monitor sustainable development and formulate integrated, evidence-based policies, the System of Environmental-Economic Accounting (SEEA) plays a central role in Lao PDR strategy for the development of environment statistics, as well as for the mainstreaming of their use in policy processes.
However routinely producing environmental-economic accounts at the national level by the national statistical office requires addressing the technical and institutional barriers to the integration of these different data sources.
In the context of the national statistical systems of a lower middle income country, the case of the development of Material Flow Accounts by the Lao Statistics Bureau with support from the Lao Luxembourg Cooperation Project in Statistics is illustrative of the challenges faced to integrate data from multiple sources into one single database (the number and diversity of stakeholders and domains and the consequent dispersion of data, the difficulty of access, the often poor quality of the data (linked notably to the relative novelty of the statistics involved) etc.) and provides feedback on the solutions applied to overcome the challenges.
Keywords
Introduction/background
In the 9th National Socio-Economic Development Plan (NSEDP, 2021–2025), the Government of Lao PDR (GoL) has reaffirmed its commitment to a paradigm shift towards green growth and sustainability as the orientations that will guide its actions to reach its aspiration to modernize the country and graduate from least-developed country status by 2026. GoL’s ambitious agenda simultaneously entails enhancing its performance, demanding more accountability, curbing corruption and remodeling its systems to utilize more evidence-based and informed decision making.
Lao PDR is endowed with important natural assets and the development of natural resources has been a key driver of economic growth, job creation and poverty reduction and is considered a great opportunity to support a transition towards greener, cleaner and more inclusive development. However, without an improved system to sustainably manage natural capital, the social and environmental costs might become irrecoverable, outweighing the economic benefits of exploitation and reducing future opportunities. The lack or difficulty of access to relevant data, in particular on the environment, constrains informed policy making and results monitoring, inhibiting advances on the green growth and sustainability agenda.
The evidence based policymaking approach to development requires better official statistics and more generally quality and timely data as well as quick access to information (from statisticians, but also from ministries, businesses, universities, mass organizations and individual citizens) The commitment of the GoL to shape a more centralized, coordinated and professional National Statistical System (NSS) for the production of more and better official statistics, is notably reflected in its endorsement of a new Law on Statistics and the adoption of a strategic approach to the development of statistics But the demand is not only for more and better official statistics, but also for improved capacity to analyse, disaggregate and interpret data. Capacity development is required not only for National Statistical Office, Lao Statistics Bureau (LSB) but also for line ministries, provinces and districts.
In this context, the LSB is mandated to coordinate and support other Ministries and Agencies as technical resource for the production of environment statistics to respond to policy processes needs to shape a sustainable development that conciliates economic development, cultural and social progress, with “natural resources preservation and environment protection, natural disasters mitigation and climate change adaptation”.
In order to respond to these needs/to fulfil its mandate, the LSB, with support from the Lao PDR-Luxembourg Cooperation Project in Statistics (LLPS) and in close collaboration with partner Ministries and agencies within the NSS, has initiated the preparation of a Master Plan for the Development of Environment Statistics (MPDES) adopted in 2019 as well as the establishment of a Working Group on Environment Statistics (WGES) to supervise and coordinate its implementation for the development of a Statistical Information System on Environment Statistics (SISES).
As a reference framework to integrate economic and environmental data and allow more comprehensive analysis on the interrelationships between the economy and the environment and their evolution, the System of Environmental-Economic Accounting (SEEA) [1] plays a central role in Lao PDR strategy for the development of environment statistics as well as for the mainstreaming of their use in policy processes. The SEEA uses an integrated systems approach, incorporating multiple data sources within a unified analytical framework, it enables an understanding of tradeoffs and synergies to monitor sustainable development and formulate integrated, evidence-based policies.
However routinely producing environmental-economic accounts at the national level by the national statistical office requires addressing the technical and institutional barriers to the integration of these different data sources.
Hence, at the start of 2019 the Lao NSS (with support of STATEC and UNESCAP) has launched the development of Environmental-Economic Accounts (EA) with an assessment of environment data available, the identification of priority EA, the establishment of dedicated inter-ministerial task forces and the provision of capacity development activities for task forces staff members. Two priority EA have been identified, Material Flow Accounts (MFA) and Physical Energy Flow Accounts (PEFA), and their compilation has started during the second half of 2019 with the development of methodological guidelines and compilation tools [2]. The first compilation of official MFA by the LSB was completed at the end of 2020.
The development of MFA by the LSB with support from the LLPS in Lao PDR is notable for two main reasons:
First the compilation of MFA builds on the development from scratch for LSB of the whole Statistical Information System on Environment Statistics. Second the compilation of MFA requires incorporating within a unified analytical framework multiple data sources dispersed over a large number and diversity of stakeholders and domains.
World Development Report 2021 – Dimensions of integrated national data system.
In the context of the national statistical systems of a lower middle income country, the case of the development of MFA by the LSB with support from the LLPS is illustrative of the challenges faced to integrate data from multiple sources into one single database (the number and diversity of stakeholders and domains and the consequent dispersion of data, the difficulty of access, the often poor quality of the data (linked notably to the relative novelty of the statistics involved) etc.) and provides feedback on the solutions applied to overcome the challenges.
Thus this case study will aim at responding to the following questions: What are the challenges faced by LSB to integrate data from various sources to routinely producing environmental-economic accounts? Among the various phases of the statistical business process and within the whole Lao PDR NSS, what issues have been the most prominent? How did the LSB manage, or did not manage, to overcome these challenges?
The present paper aims at presenting the integration of data sources in environmental-economic accounting in Lao PDR as a case study of a project in which multiple sources data were used, to outline benefits and challenges, and to present some of the potential solutions identified and implemented.
Data integration involves multiple dimensions. First there are many possible types of data integration involving multiple combinations of data sources. Second undertaking data integration projects potentially involves many issues from challenges with data collection, sharing and reuse, managing quality, risks, comparability and metadata requirements, mobilizing adequate human resources building partnerships, to weak regulatory environments, governance framework and policies to awareness, incentives and funding.
In order to organize the results of the case study and to cover as much as possible of these different dimensions we are using a set of criteria inspired by the list of broad topics1 included in the High Level Group for the Modernisation of Official Statistics Guide to Data Integration for Official Statistics [3] as well as the list of functions, pillars and foundations proposed by the World Bank in the World Development Report 2021 [4] to characterize the vision of an integrated national data system (see Fig. 1).
The integrated national data system described in the World Development Report 2021 with multistakeholder data governance and safe, highquality data flow may remain a distant vision for Lao PDR NSS. The dimensions identified in the report provide nevertheless good references to evaluate the integration of multiple data sources in environmental-economic accounting in Lao PDR and draw attention to the challenges, successes and remaining gaps identified in the process.
Hence, the case study of the challenges in the integration of multiple data sources in environmental-economic accounting in Lao PDR is structuring around the following set of broad topics:
Data governance environment Data governance environment includes laws and regulations, policies and strategies, institutions, ICT infrastructure as well as cooperation and coordination mechanisms between multiple stakeholders. Data quality dimensions Data quality dimensions include the quality of the data production process and outputs, based on common standards, data protection, data openness and usage and quality assurance. Enabling foundations The enabling foundations include human capital, trust, funding, incentive, and culture of data use.
Results
The following paragraphs describe successively the situation of the Lao National Statistical System (NSS) in terms of data governance environment, data quality dimensions and enabling foundations, in the context of the integration of survey, administrative and private environmental data sources for the compilation of environmental-economic accounts in Lao PDR.
Data governance environment
The first dimension for the integration of data sources in official statistics is the data governance environment. This section describes briefly the elements of interest in the national legislation, the role of existing institutions and their relation within the Lao NSS, the Strategy for the Development of the NSS (SDNSS), as well as its sectoral sub-strategy focused on the development of environment statistics. Finally, it presents the cooperation mechanisms in place.
Laws and regulations
The GoL has taken important steps to improve the institutional, legislative and regulatory environment for official statistics.
Revised statistical legislation enacted in 2010, and again in 2017 [5], has designated LSB as a center for coordination with line ministries and agencies (the horizontal statistics system) as well as with provinces and other sub-national localities (the vertical statistics system) and conferred on it the right of technical certification and endorsement of official statistics. LSB is headed at the Vice-Ministerial level under the Ministry of Planning and Investment (MPI) and has drawn its budget directly from the Ministry of Finance (MoF) since 2011.
The Law on Statistics is based on the principles of the ASEAN Community Statistical System Code of Practice, which in turn are consistent with the UN Fundamental Principles of Official Statistics. The law provides the following:
Institutional Environment
Mandate for data collection – clear legal mandate exists. Professionalism and Integrity – follows professional considerations on methods and procedures. Confidentiality – information provided by respondents for statistical purposes is safeguarded. Accountability – statistics are presented in a clear and understandable form with sources and methodology. Statistical Cooperation and Coordination – cooperation and coordination contribute to the improvement of the NSS.
Statistical Process
Cost effectiveness and adequacy of resources – resources are sufficient and used effectively. Reduced respondent burden – reporting burden should not be excessive for respondents.
Statistical Output
Commitment to Quality – regularly review and improve the quality of statistical processes and products. Indicators are: relevance, reliability, timeliness, comparability and accessibility.
Strategic framework
The GoL has also adopted a strategic approach to statistical development.
Strategy for the Development of the National Statistical System (SDNSS)
The SDNSS, revised, updated and extended in 2016 to cover the period from 2016 to 2025, provides a strategic framework for statistical work in the Lao PDR [6]. It aims at improving the production, availability and use of quality statistics, and at modernizing and streamlining the national statistical system (NSS) by addressing numerous challenges hindering statistical development.
The Strategy aims:
To improve and strengthen the foundation for the development of statistical production in both quantity and quality. To ensure that the NSS meets international standards and to promote the credibility of the NSS to the public sector, private sector, development partners and funding organisations both domestically and internationally. To ensure comprehensive and sustainable economic, social, and environmental official statistics are made available to users with high quality, timeliness, transparency and credibility. To ensure that the production of official statistics is efficient by using modern information systems and methods; and To involve all stakeholders actively in the formulation of official statistics.
These aims will be pursued by 5 main strategies as follows:
Improving and implementing regulation. Developing the statistical infrastructure (including ICT infrastructure). Management and development of data. Ensuring Statistical Coordination System and Networks Integration. Resource mobilisation to support Human Resources development and the implementation of statistical activities.
Master Plan for the development of environment statistics (MPDES)
The SDNSS is complemented by a Master Plan for the development of environment statistics (MPDES 2019–2025) [7] aiming at the production and dissemination of high-quality official statistics compliant with international standards and guidelines in the domain of environment through the development of a SISES. The MPDES includes five work programmes:
Work program 1: Improve the organization through the creation of institutional units and the allocation of dedicated human resources within LSB and the NSS. Work program 2: Develop the methods and process in line with international standards and classifications. Work program 3: Produce environment statistics, environmental-economic accounts and indicators to respond to national and international needs. Work program 4: Ensure the coordination between producers of official statistics through the creation of a WGES and specific task forces. Work program 5: Increase knowledge and competency in the domain of environment statistics.
The MPDES is adopted at the Ministerial level by both LSB and the Ministry of Natural Resources and Environment (MoNRE).
Cooperation mechanisms
The cooperation with the dataset owners is a key challenge to address. LSB needs to ensure that quality data will be provided with continuity and consistency. In order to facilitate this cooperation and coordination several mechanisms are relevant in the context of the compilation of EA.
Working Group on Environment Statistics (WGES)
In order to facilitate the cooperation around environment statistics production within the NSS, the MPDES established a Working Group on Environment Statistics (WGES) The WGES is led by the LSB and composed of members/focal points from the Ministries and Agencies of the Lao NSS dealing with the collection of environmental data and information.
The aim of the WGES is to oversee and manage the establishment and operation of the Lao SISES to support environmental decision-making and assist in the monitoring and evaluation of the National Social Economic Development Plan, as well as satisfy international requests for information on environment, in particular the SDG initiative.
The WGES ensures the coordination between producers of official environment statistics. The coordination role of the WGES includes the coordination of the efforts of international organizations, a crucial dimension in Lao PDR where donors are particularly active. In the environment statistics and EA domains LSB and the LLPS have notably worked closely with the WB, UNSD and the ESCAP.2
Task force for the development of MFA
Under the supervision of this WGES, the creation of specific sub-groups was envisaged and, in particular of a specific task force for the development of MFA. The task force includes focal points and technical staff of the Ministries and agencies producing the data sources of the MFA.
LSB-MoNRE MoU
The adoption of the MPDES and the creation of the WGES are complemented with the signature of a bilateral memorandum of understanding between LSB and MoNRE to confirm the commitment to jointly implement the MPDES, clarify the distribution of tasks between the parties of the agreement as well as to define the rules and conditions of their collaboration.
Data quality dimensions
Integrating data from different sources and of different types can provide more timely and more detailed statistics, meet new and unmet data needs, lower response burden and address quality and bias issues. But it requires high quality data shared between data producers and various users and resulting from on a statistical production process based on sound methodological foundations and common internationally recognized standards. The quality of the data should also be controlled.
Production/collection process
The data used need to be produced in a relevant, timely, accurate manner, at a sufficient level of granularity to inform policy decisions.
The LSB is mandated to lead the collection of official data through surveys and censuses, but the LSB also collects administrative data from other Ministries and Agencies as well as from private companies. In the domain of environment some statistics were already collected by LSB for the purpose of the statistical yearbook or for the compilation of national accounts but several others were not centralized by LSB at the start of the LLPS in 2017.
Building on the Framework for the Development of Environment Statistics (FDES 2013) [8] and the derived Environment Statistics Self-Assessment Tool (ESSAT) the LLPS project developed a survey to assess the availability and needs related to environment statistics and establish a list of statistics to collect as well as of counterparts’ departments in the relevant Ministries. The results of the surveys showed that most environment data were not collected for and therefore not fit for statistical purpose with no centralized repository, no time series, lack of appropriate formats, uncertain frequency etc.
Based on the information collected during the survey, standardized questionnaires covering the statistics within a core set of environment statistics organized by Ministry and by main topics have been developed in collaboration with partner Ministries.
Once finalized, LSB environment unit’s staff have prefilled the questionnaires with the statistics available within LSB to reduce respondent burden, improve the questionnaires and increase the knowledge on environment statistics classifications, concepts and units within LSB environment unit staff.
Simultaneously, training on environment statistics was delivered to counterpart staff designated in other ministries and agencies covering the main concepts and good practices in the official environment statistics production cycle (especially to those who had received little to no formal training in this domain).
In addition a compilation manual was produced, and specific training on the data collection tools conducted for the respondents.
Despite these preparatory activities, the first wave of data collection within the custodian departments of the partner ministries and agencies required numerous iterations the tasks of verification and cleaning (including data checks based on Environmental Economic Accounting frameworks) took a very significant time to LSB environment unit staff to validate the series and at the end of the process several questionnaires could still not be filled in satisfactorily.
The second wave of data collection permitted to make significant progress but mainly, when possible, at the cost of an expansion of the scope of intervention of LSB environment unit staff up the chain of production of environment statistics:
In most cases, requests for clarification required the direct intervention of the departments which produced the data. Clarification was notably required for the compilation of data on agricultural products due to inconsistencies in the series, but the process was quite straightforward. When possible direct support in the form of “on the job training” to improve compilation from the raw data sources was provided to the custodian departments in order to fill in the questionnaires. Direct support was for instance provided to the Ministry of Energy and Mining (MEM) to extract information from the reports from mining companies. Sometimes an expansion of the scope of the data collection to additional Ministries has been needed: in particular the Ministry of Public Work and Transport (MPWT) was solicited to provide information on road, hydroelectric dams, train and construction in general to improve the data on non-metallic minerals, Ministry of Industry and Commerce (MoIC) for cement production or hydropower production in companies reports. Finally estimations and the definition of action plans to collect the data needed were in some occasions the only responses possible. Due to the unavailability of data, Crop residues (used) and fodder crops, and grazed biomass have been estimated. The data used for estimation of crop residues could be sourced from Ministry of Agriculture and Forestry (MAF) data on “Crops amount produced”. For the estimation of fodder crops and grazed biomass, the number of live animal data from MAF data on “Live animals” could be used.
Following the compilation of environment statistics, and a statistical quality evaluation based on descriptive statistics, quality datasets have been integrated into the environment database together with a metadata repository, including methodological document and dictionary of variables. The environment statistics database was used as main source of statistics for the production of tables, the calculation of indicators and the compilation of environment economic accounts, including MFA.
The first lesson learned from this exercise is that the legal framework was central in making possible the integration of these multiple data sources notably in providing the legal basis for LSB mandate and creating dedicated statistics department or division in each Ministry. However, despite the existence of a dedicated statistics department or division in each of the Ministries involved, the very high degree of decentralization in the production process and storage of administrative environmental data within the NSS constituted a challenge. As a consequence, the list of counterparts’ departments at the various ministries involved in environment statistics production had to be constantly expanded to obtain the data needed. This had a negative impact on timeliness.
The second lesson is that the production process, the quality of the data, the availability of metadata varied a lot depending on the custodian agency. When the reception of data was common and regular, and thus the cooperation between the data producer and LSB well established, data received usually responded better to the needs. Data on imports and exports of goods for instance were not exempted of quality issues but they were easily accessible, with timeliness, in internationally recognized classifications and with a high level of granularity. Hence, the main data source used for the compilation of IMP and EXP was trade transaction data from the Lao custom database. The data included monetary as well as physical units. The reported physical weight could be used directly without any adjustment. When trade information on materials or products in large physical amount was available, that information was used to cross-check the data from the trade database.
The third lesson is that generally very limited metadata were available with the data collected As a heritage of the pyramidal reporting system established in 1975 with a high degree of decentralization to serve the central planning system as a national statistical system, “traditionally” administrative data are collected at the village level, and then successively aggregated at district, provincial and finally at the national level first within the relevant Ministry before being transmitted to the LSB. The result is a lack of granularity as well as information on the data available at the Ministries’ level Data for compiling MF.1.1–Crops, which consists of 10 sub-categories, was usually well recorded in the Lao National Statistics System and the data recorded in the “Questionnaire on Environmental statistics on Crops & Livestock” could be used directly for the compilation of MF.1.1–Crops. However, the level of granularity of the data available was relatively low, metadata available limited, notably information on the production process.
Moreover LSB had only very limited and indirect control in the definition of the concepts and populations used in the collection of the data. Differences in concepts, classifications, populations and collection units were thus treated a posteriori in adapting the existing statistics to the needs.
In some cases the production chain was short enough or the relationship between LSB and the custodian Ministry sufficiently well established for LSB to directly work on the raw data sources with the statistics department of the Ministry to understand the design decisions made and production process applied or even to influence them.
Business process model nomenclature applied to data collection in environment statistics.
The data used for the compilation of metal ores in this LSB-MFA came from two sources, mining statistics collected from MEM, and company reports. In Laos there are three major mining sites, the information reported by the company mentioned previously includes these mines. The data from company reports was preferred to data from MEM because the latter did not for instance include information on gross ores.
But in other cases, the chain of production was too complex and/or remote and the lack of information on the production process could even affect the usability of the external data source for statistical product. There might also be resistance to change the production process of external data sources especially when the current practices have been long-established.
Finally some information provided by partner Ministries and agencies could simply not be converted as statistics because they were not based on actual measurement but on quotas or targets set by the GoL, because there was not sufficient information on the population covered, the concepts and definitions used, or because of the low quality of the data themselves (variability, missing values, inconsistencies etc.).
The development of environment statistics as a new statistical domain has provided a unique opportunity to implement a SISES directly based on international standards. Common standards are key to guarantee the quality of the statistical production but also to facilitate the flow of data between stakeholders at national and international level.
Production standards
Hence the development of the SISES in Lao PDR has been based on a standardized approach building on the Generic Statistical Business Process Model (GSBPM) [9], the Generic Statistical Information Model (GSIM) and the Common Statistical production architecture (CSPA) developed by UNECE.
The GSBPM has also constituted the reference for the implementation of the SISES. LSB environment unit staff and selected ministries and agencies focal points have received introductory training on the GSBPM and all activities conducted during LLPS phase 1 have been contextualized in the statistical production business process.
Business Process Model and Notation (BPMN) has also been introduced as a standard for business process modeling to support the management the data collection process notably for LSB environment unit staff (see Fig. 2).
Finally, the GSIM has also been applied to the Lao SISES to modernise and streamline environment statistics through the provision of generic descriptions of identified 110 standard information objects, in two, “Structures” and “Concepts”, of the four groups presented in the GSIM.
More specifically the information objects selected are the following:
Data resources Data sets Data points Variables Statistical classifications
Environment statistics standards
From a subject matter perspective, two main frameworks were used as references for the development of environment statistics and accounts in Lao PDR.
FDES describes the structure for a catalogue of environmental data and SEEA describes a framework for the structuration of environmental statistics. For the particular case of MFA the reference framework used is Eurostat EW-MFA guideline [10].
Classifications
A key issue in the domain of standards is classifications. Given the limited metadata available for most national classifications and the need for international comparability, the choice was made to use translated international classifications as early and faithfully3 as possible in the compilation process of EA preferably to national classifications.
In cases different classifications were already in use for data sources, the usual process was to use correspondence tables and conversion tools to convert into correct classification code.
Hence to convert imports and exports information from HS classification code to MFA classification code, the correspondence table provided in Eurostat EW-MFA guideline was used. Due to the compatibility of the dataset obtained from the Lao custom database with the code in the correspondence table, the matching was done at the HS code 4-digit level.
The same process could be applied for national classification for which correspondence tables with international classifications (in our case MFA classification) were available. Sometimes intermediary correspondence tables had to be used to develop new correspondences tables. Classification used for Lao PDR national accounts and Central Product Classification (CPC2.1) were for instance used to create intermediary correspondence tables for the conversion from national classification used for Crops & Livestock statistics by MAF into MFA classification.
However, in some occasions the national classification in use had no established correspondence with international classification. In this case correspondence tables had to be developed. This can be challenging especially when classifications are complex or the rules of a classification are difficult to apply. The situation were even more challenging when translation between Laotian and English languages were needed. In some occurrences no recognized translation existed in Laotian for some of the concepts (notably statistical concepts) included in the international classifications. Cooperation between the LSB and subject matter specialists from the custodian Ministry or Agency was in this case crucial to solve the classification problem.
This was notably the case to determine the right correspondence between Industrial chemicals recorded in the statistics systems and the Industrial chemicals included under Non-metallic minerals category (M.F.3) in EW-MFA.
Concepts and definitions
Transforming administrative datasets into statistical datasets also requires dealing with conceptual differences. For instance as seen earlier, two sources, mining statistics collected from MEM, and company reports, were available for the compilation of metal ores in the MFA framework. Generally, in mining statistics, the non-ferrous metal group is reported whether in metal content or concentrate, therefore the conversion from such reported forms to gross ores must be carried out. However, the distinction between metal content and concentrate were not clear and the information on gross ores were not included in the data from MEM. The data from company reports had thus to be used.
Even more problematic, in several occasions in the data sets provided by partner Ministries and agencies no clear distinctions were made between actual measures, quotas or targets. Hence some of the data sources provided by MAF for compiling MF.1.3–Wood were in fact summary of the figures from timber extraction quota reports.
Units
The question of reporting units also appeared in some occasions problematic. Hence, for instance, the existence of several traditional units of fuel wood volume used in different parts of the country made difficult the use of data from Lao Expenditure and Consumption Survey (LECS) for the estimation of fuel wood extraction.
Relationship of monitoring control and evaluation to process.
Data transfer format
Finally the question of the standards used to facilitate the exchange of data was also central in the process of integration of environmental data for the compilation of MFA. Hence the LLPS supported the development of standardized data collection tools to facilitate the transfer of data from partner Ministries and Agencies to the LSB. These questionnaires developed in collaboration with partner Ministries and organized by Ministry and by main topics have been voluntarily kept as simple as possible and developed as Excel files to use a software with which all focal points were familiar The focal points have been encouraged to participate in their elaboration and trained to use them. However at the end of the first phase of the LLPS a significant share of the focal points were still not autonomously filling the questionnaire satisfactorily and data were still received occasionally in the form of print out of summary tables or data series on loose sheets of paper sometimes annotated manually occasioning the additional burden for LSB staff and risk of error associated with manual data entry.
The quality monitoring, control and evaluation within the Lao NSS is based on a reference approach described in a document the Lao Official Statistics Quality Assurance Framework (LOSQAF) [11] Where projects have been defined and while they were being implemented, they usually provided their own quality assurance, but recommendations for quality improvements emanating from quality monitoring and evaluation under the LOSQAF should, even in this case, be implemented as part of project implementation.
LOSQAF
LOSQAF is based on the Generic National Quality Assurance Framework (NQAF) Template, Guidelines and Statistical Glossary that were developed by a United Nations Statistics Commission Working Group in 2012 specifically to assist NSOs, such as the LSB, to build their own quality assurance frameworks (QAFs).
Five types of quality monitoring, control and evaluation are envisaged in the LOSQAF (see Fig. 3):
monitoring of quality and performance indicators during the process; application of quality gates during the process; assisted self-evaluation of the process and its outputs; internal, peer-based evaluation of the process and its outputs; external evaluation of the process and its outputs.
Monitoring of quality and performance indicators during the process
The quality monitoring and evaluation implemented during the collection of administrative data for the compilation of MFA were principally of three kinds.
(a) Assisted self-evaluation
The objectives of assisted self-evaluation were to help the manager and staff responsible for the process to assess its quality and the quality of its outputs, to identify structural weaknesses, and to propose quality improvements.
The initial evaluation of the quality of environment statistics available within the NSS and within LSB for the compilation of MFA, was conducted using the Framework for the Development of Environment Statistics (FDES), Environment Statistics Self-Assessment Tool with support of LLPS experts.
(b) Application of quality gates
The objectives of quality gates were to ensure errors were detected as soon as possible after they occurred, for the underlying causes to be determined, for the errors to be corrected, and (in the case of a repeating process) for the process to be adjusted to prevent or reduce similar errors in the next cycle.
The gates were placed at key points in the process. Problems uncovered by quality gates were addressed at the time that they were discovered.
The quality of environment statistics provided by the partner Ministries and Agencies has been assessed by LSB Environment unit team and LLPS experts as the questionnaires were received. The quality was assessed through descriptive statistics, basic consistency checks and confrontation with the data available within LSB and published by the Ministries and Agencies as well as the information collected during the inventory conducted in 2018. The results of this initial assessment conducted during the data collection process were shared on a continuous basis with LSB counterparts to be discussed with the data producing departments in the partner Ministries and Agencies when necessary at the occasion of data validation meetings organized to eliminate inconsistencies, agree on the correction of the eventual mistakes, clarify the misunderstandings and improve the quality of the data collected in general.
This initial quality assessment was completed with checks based on the MFA framework conducted during the compilation process which provided additional opportunities to check the consistency and plausibility of the data included in the framework. Similar checks have been conducted for the data included in Physical Energy Flow Accounts notably at the occasion of the compilation workshop organized in November 2019 on the subject.
(c) External evaluation of the process and its outputs
An external quality evaluation of the SISES was conducted at the end of the LLPS. The evaluation objectives were to provide an objective view of the quality of the process developed during the project, and hence to identify any structural weaknesses and to propose quality improvements to address them.
More specifically, the evaluation conducted by an expert from the STATEC, the twinning partner of the LLPS, had two main objectives:
Evaluating the quality of the statistical production process and outputs. Evaluating the sustainability of the process through an evaluation of the autonomy of LSB environment unit team members in the various phases of the compilation process.
The quality evaluation aimed at informing statisticians and users on the quality of the statistical series (identify outliers, describe trends, evaluate variability) as well as to build trust in these statistics. The terms of reference of the quality evaluation included the following dimensions:
For statistical purpose, calculation of descriptive statistics: Position indicators (average, minimum, maximum, median, quartile) and dispersion indicators (statistical scope, interquartile range, variance, standard deviation) have been presented and their interpretation have been explained. The quality evaluation based on descriptive statistics was subsequently integrated directly in the questionnaires to be systematically applied on all collected datasets. For informative purpose, a quality grading was performed and quality components were identified to be described in a dedicated report. The results of quality grading was considered informative for users but it was pointed that this grading was not sufficient to guarantee the statistical quality of dataset. The second part of the quality evaluation was dedicated to an audit of various phases of the compilation process. The audit concluded that the collection, verification and cleaning phases of the GSBPM were considered sustainable. The storage phase required additional technical documentation of the processes. Finally the calculation phase and dissemination phases still required additional work.
Data usage
The data collected have been processed and stored in an environmental statistics repository. The objective of the repository was to be as accessible and versatile a source of quality statistics on the environment and for the environment based on as transparent and internationally recognized standards and processes as possible to encourage frequent and widespread use of the data by various users and for a wide range of purposes.
However, the calculation phase and dissemination phase of the SISES were not yet fully operational at the time of writing. Disseminating the datasets on LSB website and on international databases was still an objective for coming years and the publication of the derived EA and indicators was still work in progress.
Enabling foundations
The integration of data sources in official statistics also relies on enabling foundations including human capital, trust, funding, incentives and a culture of data demand and use.
Data demand and a culture of data use
Valuing data is crucial for the right data to be produced. Traditionally the needs from international organizations have stimulated the demand for official statistics in Lao PDR, especially on the environment.
Actually despite the creation of the LSB (as a department of statistics of MPI) in 1975 as a reporting system to serve the central planning system, it is only at the occasion of the 8th National Socio-Economic Development Plan (NSEDP, 2016–2020) that a monitoring framework was formally associated to the plan. The 9th NSEDP (2021–2025) has reinforced this commitment to the paradigm shift started in 2016 towards a policymaking approach to development characterized by an increased focus on evidence-based and informed decision making. The new approach implies higher demand and use of data. It requires better quality and timely official statistics as well as quick access to information not only for the GoL but for all stakeholders. Availability even of quality data is not enough though, as Muller [12] argues, the data “must be analysed and converted into information and knowledge, then shared widely within and between countries and stakeholders”. This is particularly true for the environment, a domain in which the lack of data inhibits advances on the green growth and sustainable development agenda.
Data are increasingly viewed as foundational for creating public value through improved policy making, particularly by high-level management but this view is not yet widespread in Lao PDR society. Data is still not widely used neither in the Lao society at large nor within the GoL. This is true from data describing phenomenon at the scale of the nation to data produced and focused at the scale of organizations or individuals The potential value of data as well as information sharing is not always well understood. Some data and information are even produced but remain unavailable and later become obsolete because they tend to remain with consultants hired to support the NSS in their production This is partially rooted in the current limitations in the dissemination of quality, notably timely, statistics in Lao PDR but also in the need to improve the data literacy of individuals, journalists, and other stakeholders including data producer or even policy makers and legislators.
Incentives
The question of incentives for institutions and individuals to produce, protect, and especially to share data was a key issue in the process of integration of data for the compilation of MFA in Lao PDR.
If not openly, some custodian agencies have shown reservation and even sometimes reluctance to provide access to the data they produced. Thus, not all the data available was effectively made available to the LSB.
Attempts to make datasets available have notably been hindered by economics and ownership-related issues. Relinquishing exclusive ownership might have meant for the producer of data renouncing to the revenue these data could potentially generate. Despite their availability data on land cover could for instance not be centralized by LSB.
Sometimes also, as argued by Ndzabandzaba [14], data custodian might have been unwilling to share because they didn’t understand the value and the importance of sharing data Some information collected in the framework of projects at local level are for instance sometimes not shared and thus not aggregated at the national level.
LSB clear mandate, grounded on the Law on statistics, to centralize environmental data within the Lao NSS, the clear demand from the central government to improve the availability of environmental indicators to monitor the 9th NSEDP and the SDGs in particular, have constituted strong incentives for data sharing but not always sufficient. Adopting compensation schemes could constitute ways to promote data exchanges. The compensations provided in the framework of the LLPS have taken the form of capacity development activities, access to processed better quality statistics, to improved metadata, production statistical analysis or dissemination material.
Trust
For the data to be shared by the different stakeholders there must be trust that the data will be protected and not misused.
Security concerns or protection of reputation, in particular the conception that some data are sensitive and could be used against its producer or Lao PDR government, can be reasons for not sharing data in spite of agreed protocols. As mentioned above, despite the absence of any open refusal, data on timber extraction have been for instance difficult to access for LSB and only information on quotas have been provided by the custodian agency.
The OECD [13] identifies three sets of factors underlying trust in official statistics:
Structural factors related to the independence and professionalism of the Lao NSS and its respect of the UN Fundamental Principles of Official Statistics including protection of data, described above in the data governance section. Statistical factors such as commitment to quality, compliance with internationally recognized standards and sounds methodologies, are detailed in the data quality dimensions section. Reputational factors, finally, public opinion’s on official statistics, how much it is valued and trusted, depends on how much user needs are taken into account Official statistics only add value if they are used and thus respond to users’ needs. It follows that assessing users’ views should be an integral part of the business process of developing, producing, disseminating and reviewing official statistics. ‘Users’ means all stakeholders, from central government on the one hand to members of the public on the other. At this stage of the development of environment statistics and EA in Lao PDR the focus was on public and international users, but the plan is to gradually expand the scope of SISES to integrate better a broader range of users and producers.
Funding
Integrating administrative data is in principle less costly than conducting sample surveys as they are supposedly already collected for administrative purpose.
However administrative data are not always up to the standards necessary to be integrated in official statistics and sometimes even not collected. In this case the costs, at least in terms of human resources and capacity development, to start a data collection from scratch, or even to bring up the data production process within Ministries and agencies up to the expected standard can be important. As described above this is a situation experienced in several environmental domains Lao PDR.
Moreover, even when data is readily available and usable integrating administrative data still requires some budget. The resources needed for integrating data notably include operating budget, IT infrastructure and human resources. Human resources include the time of subject-matter statisticians, methodologists and IT staff but also in our case the costs related to hiring experts to support the LSB and partner Ministries in the development of the processes. The complement to the funding from the GoL notably provided by the Government of Luxembourg through the LLPS permitted to cover most of the additional costs incurred for the integration of the data for the duration of the project. However, donor support could not solve all resources issues:
Additional activities generated by the project put a strain on limited availability of qualified staff within the LSB and the NSS in general. The limiting factor for the implementation of the project becomes the availability of qualified government staff working time. Donor funding cannot replace the government budget as it cannot cover all kind of costs and is by definition more punctual. Hence as they don’t impact salaries and are at least perceived less sustainable, donor funds cannot solve data economics and ownership-related issues described above.
The creation of a department dedicated to environment statistics within LSB should help increase the sustainability of the production of environment statistics in general and EA in particular.
Human capital
As mentioned above the availability of, in our case, qualified government staff within the LSB but also in the partner Ministries and agencies was the crucial factor to be able to successfully integrate data sources for the compilation of environment statistics and EA.
Particular care was taken to identify the talented people with the right set of skills to integrate LSB environment unit and become environment focal point in partner Ministries and Agencies but also to join the task force for the development of EA. This was particularly important as the number of staff to be involved would always remain limited: the whole range of environment statistics and EA had to be covered by only two technical staff at LSB.
In order to develop capacities, the following specific training and workshops were also organized:
Statistical literacy in the domain of environment statistics
Workshop on Environmental-Economic Accounts development in Lao PDR in order to Support Policy Making and SDG Development gathered representaives of departments from MoNRE, MAF, MEM, MLSW, MPI, MoF and MoH. Regional EA workshop (in collaboration with ESCAP) with the objectives to provide general knowledge on FDES, SEEA and SDGI-E and share experiences and best practices at the regional level of NSO involved in the compilation of EA. Environment statistics subject-matter and methodological skills
The training on environment statistics targeting the members of LSB environment statistics unit and the counterparts designated in other ministries and agencies introducing notably environment data needs identification, GSBPM and data production process, quality management, data processing, metadata and databases as well as the main statistics, accounts and indicators frameworks (including FDES and SEEA) on the environment. The training on FDES and SEEA to introduce the two main reference frameworks selected for the production of environment statistics in Lao PDR for LSB environment unit. EA compilation
Training on MFA concepts: introduction for staff to be involved in the compilation of EA. Technical EA workshop on MFA to compile first draft MFA. Technical EA workshops on advanced MFA compilation to finalize the compilation of MFA.
Discussion
The current study aims at highlighting some of the specific challenges encountered by NSOs in developing countries in the integration of different data sources in the production of official statistics. The development of MFA by the LSB with support from the Lao Luxembourg Cooperation Project in Statistics is used as a case study to identify and illustrate the challenges faced to integrate data from multiple sources into one single database and provides feedback on the solutions applied.
As their compilation requires incorporating multiple and broadly dispersed data sources within a unified analytical framework, MFA imply the coordination of a large number and diversity of actors and domains. Moreover, as a domain of statistics being newly centralized in Lao PDR, it cumulates issues related to difficulty of access and often relatively poor quality of the data.
Hence routinely producing these environmental-economic accounts at the national level by the national statistical office requires addressing a large range of technical and institutional barriers to the integration of different data sources. Actually far from being a circumscribed activity, the integration of data for the compilation of MFA has required to touch upon the various domains in the statistical business process within the whole Lao PDR NSS from the data governance environment to statistical quality dimensions to enabling factors such as human capital, incentives or trust.
The study has permitted to identify some domains that did not constitute constraints:
The Legal and strategic framework, including the strategy and mechanisms specifically developed, appeared adequate for the current stage in the process of data integration. As argued by Hughes [15], the law on statistics and SDNSS were in general sufficiently complete and explicit to facilitate data sharing from line ministries and agencies and the institutions set in place appeared to play their role satisfactorily. The whole data governance environment, including IT infrastructure both at the national and NSS level, despite current limitations, was also considered sufficient at this stage for the integration of data as it did not constitute a bottleneck for this specific activity. Lack of funding from donors was not identified as a limiting factor for the integration of data sources for MFA compilation. Actually the support from Luxembourg combined with support from other donors were sufficient to cover the development costs. The quality of the data production process within the LSB, including the compilation of MFA themselves, was also considered adequate for the current needs.
On the other hand, the exercise permitted to uncover some of the key issues that have affected the process of integration.
First human capital and in particular the availability of qualified staff was the most prominent challenge for the LSB. The quality of the statistical production process within ministries constituted an important limitation for the results of the compilation. The quality of administrative datasets may have been sufficient for administrative purposes but was in several occasions not sufficient for statistical purposes. In few occasions the lack of incentive for partner Ministries to provide access to their data, has also been an hindrance. Unfortunately identifying the underlying causes behind the lack of cooperation whether it was due to economic and ownership issues in a context of reliance on funding from international donors, lack of understanding of the value and importance of sharing data in a context of relatively insufficiently developed culture of data use, or a lack of trust in LSB as a recipient of these data, would require further investigations.
In order to sustain the progress made and continue the development of environment statistics and EA, this is in these categories that the key improvements are to be found.
Actually, the Master Plan for the Development of Environment Statistics foresees the continuation and extension of the EA compilation. The development of SEEA will continue with the compilation of additional modules. Various supporting projects will be set up with this goal: UNSD on mining and energy accounts, World Bank on ecosystem, land and water accounts, the Lao-Luxembourg cooperation project on monetary environmental accounts. During the next five years, new surveys must be started to fill the gaps in the scope of environment statistics. The database must be migrated into SQL and compatible with SDMX format. Different activities will also be implemented to disseminate environment statistics and give them more visibility for policy makers, academia and media. The production of green economy indicators must be extended, and their uses must be explained to various users. Finally, a support to the organization of the WGES and the extension of this national experts’ network will be continuously organized.
The recent decision by the GoL to reorganize the LSB with the creation of a department dedicated to environment and security statistics confirmed the commitment of the government and should help solve the issue of government resources in providing guarantees for the availability of staff to work in this domain, as well as increased and sustained financing The second phase of the LLPS will be complementary of these improvements in helping to support the development costs. More experience in cooperation, should contribute to build trust within the WGES. While the development of dissemination products, capacity development activities and support to improvement of administrative data collection and compilation process, should contribute to strengthen incentives to share data within partner Ministries and agencies.
Footnotes
The set of broad topics developed by the Data Integration Project to organise the issues to consider are the following: Business Requirements; Opportunities; Challenges; Risk mitigation; Standard Processes; Recommended methods; ICT considerations; Quality; Standards; Metadata requirements; Related work in other projects/organisations; Skills; Resources; Partnerships; Governance; Promotion and advocacy; Recommendations.
Thus, the LSB has contributed actively to the analytical work on Natural Capital Accounting (part of a global advisory program on Wealth Accounting and Valuation of Ecosystem Services (WAVES)) initiated by the World Bank. In particular this work has involved the valuation of Lao’s natural capital endowment at national level and in one selected landscape.
Though adaptations have been introduced in some occasions to reflect national specific needs.