NDACC Protocol for the Validation of Instruments and Data Analysis Methods
Introduction
The ‘NDACC Protocol for the Validation of Instruments and Data Analysis Methods’ (often referred to as simply the ‘Validation Protocol’) is designed to ensure that archived NDACC data are of as high a quality as possible, within the constraints of measurement technology and retrieval theory at the time the data were taken and analyzed. This is essential for supporting NDACC objectives, including the early detection of changes in atmospheric composition with the accuracy and precision necessary to understand the responsible forcings.
Thus the ’Validation Protocol’ sets the data quality criteria for affiliation of a new measurement for currently recognized NDACC instrumentation types. The associated ‘NDACC Protocol for Measurement Affiliation’ describes the process for affiliation – the preparation and submission of a proposal to qualify a measurement as NDACC affiliated –and treats both newly proposed and existing NDACC instrumentation/measurement types. These two documents work together to fully define the procedures and the quality expectations.
The specific methods whereby individual measurements should be validated have been established by the various NDACC Instrument Working Groups (IWGs) and are outlined in the instrument-specific Validation Appendices that are provided along with this document. In addition, the ‘NDACC Protocol for Instrument Intercomparisons and Campaigns’ should be referenced for details regarding recommended procedures in formal intercomparison campaigns.
Measurements seeking NDACC affiliation and their associated data analysis methods must be evaluated and verified before they are accepted in the NDACC. Validation is a continuing process, and measurements must be continuously monitored throughout their lifetime. This protocol provides guidance on evaluation of a measurement during both stages of its life cycle. Each appendix will address these two aspects as outlined below.
Evaluation of New Measurement Proposals seeking NDACC Affiliation
NDACC’s objectives require high-precision, state-of-the-art measurements. Hence, instrument development and testing as well as measurement verification are high-priority activities to ensure the operational continuity and success of the network.
The procedure for proposing a new measurement for NDACC affiliation is detailed in the ‘NDACC Protocol for Measurement Affiliation’. The appropriate IWGs base their recommendation for acceptance on the following criteria.
Quality Criteria
These criteria are detailed for specific instrument types in the Validation Appendices and address issues such as:
- Evaluation of the instrument design and data analysis
- Instrument and data analysis intercomparisons
- Progress and actions resulting from such intercomparisons
- Absolute or relative calibration techniques
- Use of standard spectroscopic data and agreed methods for the evaluation of uncertainties
Documentation
Once a measurement is accepted, the Instrument Principal Investigator (PI) should provide documentation as detailed in the ‘NDACC Data Protocol for Instrument Principal Investigators.’
Evaluation of Continuing Instruments and and Instrument Measurements
NDACC affiliated measurements must continue to meet data quality standards. As such:
- In cases where a species or parameter is measured by multiple NDACC instruments, the PIs should verify their measurements throughout their NDACC affiliation by comparing measurements obtained using the PI’s instrument with those obtained by other NDACC instruments, instruments in other networks, or satellite instruments. As discussed in the ‘NDACC Instrument Intercomparisons and Campaigns Protocol’, the use of NDACC mobile intercomparators and participation in formal intercomparison campaigns are strongly encouraged.
- During their data acquisition periods, PIs shall have access to all data (both final and preliminary) that have been archived on the NDACC Data Handling Facility (DHF) and will be bound by the ‘NDACC Data Protocol for Instrument Principal Investigators’ for the submission of their own data. All NDACC PIs are encouraged to collaborate actively with each other to enable continuing high quality in the analysis and verification processes. The ‘NDACC Data Protocol for Data Users’ provides guidelines regarding co-authorship and the exchange of results for collaborative projects.
- Changes to Instruments: Any changes to an instrument should (as far as possible) not affect the nature of the results obtained. Following such changes, it should be possible to process the results so that they are strictly comparable with results obtained previously. If this is not feasible, then the change in the data characteristics should be fully documented in the NDACC data archive, as described in the ‘NDACC Data Protocol for Instrument Principal Investigators.’
The Validation Appendices also provide criteria for the continuing evaluation of NDACC Instruments.
The criteria address:
- Requirements for validation of a currently affiliated instrument after an upgrade or replacement
- Requirements for validation of instruments that have moved from the current location to another location
- Investigator responsibilities for continuing instrument verification and data analysis
- Participation in instrument and algorithm intercomparison campaigns
- Intercomparisons with measurements obtained with NDACC traveling instruments, other NDACC site instruments, instruments in other networks, and/or satellite instruments
- Absolute or relative calibration techniques and use of laboratory standards
- Use of standard spectroscopic data and agreed methods for uncertainty evaluation
- Data analysis method updates to incorporate new scientific standards
As stated in the NDACC Protocol for Measurement Affiliation, the PI is primarily responsible for the everyday quality assurance of their own data, and for keeping the archives updated; however, IWGs should conduct validation exercises to the extent that resources and logistics permit. The results of continuing evaluation procedures are to be recorded in the NDACC archive and reported to the Steering Committee (SC). Measurement and analysis deficiencies found during such evaluations should be corrected, if possible, and any affected data in the archive flagged appropriately. If the PI informs the IWG Representatives that the correction of deficiencies is not presently possible, the instrument may be designated in the Measurements and Analysis Directory as currently inactive until corrective measures are possible. The PI will be a partner in the determination with the IWG and the NDACC SC.
Evaluation of New Generic Instrumentation or Measurement Method for NDACC
The evaluation procedure for the acceptance of a new generic instrument type or measurement method for use within the NDACC is addressed in the NDACC Protocol for Measurement Affiliation.
Current online Version: November 14, 2025
Revision: November 14, 2025
Changed title from ‘Validation Protocol’ to ‘The NDACC Protocol for the Validation of Instruments and Data Analysis Methods’. Clarified the distinction between this and the ‘NDACC Measurements Protocol’ (also now renamed) and moved some text between the two to comply with that distinction. This is fully focused on data quality and validation standards. Removed details on documentation as these are detailed in the ‘NDACC Protocol for Instrument Principal Investigators’. The history of revisions is added.
Revision: March 7, 2017
References to other protocols in the introduction are reworded. Removes explicit reference to intercomparisons being blind. Two sections: ‘Changes to Instruments and Data Processing’, and ‘Contents of NDACC Data Archive’ are removed.
Revision: March 24, 2009
Update name of network to NDACC. Removes reference to Primary/Complementary distinction. Adds section on changes to instrumentation and data processing. Adds section on contents of the NDACC archive.
Original Version: October 9, 1996