MADIS Quality Control
Overview
MADIS observations are quality-controlled during data processing and these results are made available to users. Observations in the MADIS database are stored with a series of flags indicating the quality of the observation from a variety of perspectives (e.g. temporal consistency and spatial consistency), or more precisely, a series of flags indicating the results of various quality control (QC) checks. MADIS users and their applications can then inspect the flags and decide whether or not to use the observation.
The QC procedures are, for the most part, provided by the NWS Techniques Specification Package (TSP) 88-21-R2 (1994).
Two categories of QC checks, static and dynamic, are described in the TSP for a variety of observation types, including most of the observations available in the different MADIS datasets. The static QC checks are single-station, single-time checks which, as such, are unaware of the previous and current meteorological or hydrologic situation described by other observations and grids. Checks falling into this category include validity, internal consistency, and vertical consistency. Although useful for locating extreme outliers in the observational database, the static checks can have difficulty with statistically reasonable, but invalid data. To address these difficulties, the TSP also describes dynamic checks which refine the QC information by taking advantage of other available hydrometeorological information. Examples of dynamic QC checks include position consistency, temporal consistency, and spatial consistency.
The TSP also describes single-character "data descriptors" for each observation, which are intended to provide an overall opinion of the quality of the observation by combining the information from the various QC checks. Algorithms used to compute the data descriptor are a function of the types of QC checks applied to the observation, and the sophistication of those checks. Level 1 QC checks are considered the least sophisticated, level 3 the most sophisticated checks.
QC Processing and Data Structures
- Meteorological Surface
- Road Weather Information System (RWIS) & Clarus
- Radiosonde
- NOAA Profiler Network
- Hydrological Surface
- Weather Telematics
- Aircraft Based
- Multi-Agency Profiler (CAP)
- Radiometer
- Satellite Wind
- Satellite Sounding
- Snow
- Gap-filling Upper Air
- RSAS Surface Analysis
QC Summary Message Description
MADIS meteorological surface variables are quality-controlled by the Quality Control and Monitoring System, which collects statistics on observational errors encountered for NWS sea-level pressure, potential temperature, dewpoint, surface wind, and altimeter setting. The system provides the total number of observations for each variable, the number of observations that failed the QC checks, the station names for the failed observations, and the error and threshold values for each of the failed observations. The error is defined as the difference between the QC analysis value and the observed value, as computed in the spatial consistency check.
To view a current list of surface provider QC Summary Messages available in an hourly, daily, weekly, or monthly output, please click here.
Statistics are calculated for all stations. Stations from different networks are kept statistically separate. Specifically, the following stratifications are maintained: "ASOS", "OTHER-MTR", (non-ASOS METARs), "MARITIME", "SAO", "CRN", "USHCN-M", and "NEPP". Mesonets are stratified by provider. For example, "MAP" for Multi-Agency Profiler. The statistics are available in the form of hourly QC messages. The following figure shows the MAP hourly message for 8 July 2004 at 1600 UTC.
MAP HOURLY QUALITY CONTROL MESSAGE (PAGE 1 OF 1)
*********************************************************************************
* 8-JUL-2004 * SLP * POT TEMP * DEW PNT * DD * FF * ALT *
* 1600 UTC * (MB) * (DEG F) * (DEG F) * (DEG) * (KNTS) * (MB) *
*********************************************************************************
* TOTAL OBS * 0 * 5 * 5 * 5 * 5 * 5 *
* QST OBS * 0 * 1 * 0 * 0 * 0 * 5 *
* PERCENT QST * 0.00 * 20.00 * 0.00 * 0.00 * 0.00 * 100.00 *
*********************************************************************************
* ASTOR * * * * * *1007( 2)*
* CCDNH * * * * * *1002( 2)*
* CCOCA * * 7( 6)* * * *1000( 4)*
* PSENH * * * * * *1002( 2)*
* PYMMA * * * * * *1002( 2)*
* * * * * * * *
* * * * * * * *
* * * * * * * *
* * * * * * * *
*********************************************************************************
ERROR = ANALYSIS - OBSERVATION OB ERROR ( ERROR THRESHOLD )
Hourly QC message for Multi-Agency Profiler stations on 8 July 2004 at 1600 UTC. The stations listed had variables that were found bad by the spatial consistency check.
Statistics for the total number of observations ("TOTAL OBS"), the total number of observations that failed the QC checks ("QST OBS"), and the percentage of failed observations ("PERCENT QST") are given at the top of each page of the hourly message. "QST" represents "questionable" observations. Errors and threshold values for the failed observations are listed in the columns. In ASOS hourly messages, the stations are also stratified by NWS region.
Note that threshold values are not given for the surface wind errors in the above figure. Wind observations are tested by computing observation errors and threshold values for each of the u and v components of the wind. However, observation errors are converted to polar coordinates before display in the QC messages. If either of the u or v components fail, both direction and speed errors are computed.
Stations listed in the QC messages are either in error due to hardware or software failure, or are unrepresentative of the observation scale and, as such, are susceptible to diurnal, mesoscale, and terrain effects. To help distinguish between the two, daily, weekly, and monthly (4-week) summaries of the hourly QC messages are also provided. The summaries include the percentage of failed observations and the average error and rms error of the failed observations for individual stations and for all stations combined. The following figure shows the daily NOAA Profiler Network (NPN) QC message for 16 May 1997.
NPN DAILY QUALITY CONTROL MESSAGE (PAGE 1 OF 1)
*********************************************************************************
*16-MAY-1997* SLP * POT TEMP * DEW PNT * DD * FF * ALT *
* 00-23 UTC * (MB) * (DEG F) * (DEG F) * (DEG) * (KNTS) * (MB) *
*********************************************************************************
* TOTAL OBS * 336 * 336 * 301 * 336 * 336 * 0 *
* QST OBS * 0 * 0 * 11 * 0 * 0 * 0 *
*PERCENT QST* 0.00 * 0.00 * 3.65 * 0.00 * 0.00 * 0.00 *
*********************************************************************************
* PRCO2 * * *21/-20/ 85* * * *
* * * * * * * *
* * * * * * * *
* * * * * * * *
* * * * * * * *
* * * * * * * *
* * * * * * * *
* * * * * * * *
* * * * * * * *
*********************************************************************************
ERROR= ANALYSIS-OBSERVATION RMS ERROR/MEAN ERROR/PERCENT QST
Daily QC message for NOAA Profiler Network (NPN) surface stations on 16 May 1997.
As with the hourly messages, all stations in the domain are used to calculate the statistics reported at the top of each page, but only stations that have failed the QC checks (at least once in the reporting period) are listed in the individual statistics. Stations with large percentages of failed observations are most likely experiencing hardware or software failures. For example, the QC message in the figure shows PRCO2 (Purcell, OK) as reporting bad dewpoint temperature observations 85% of the time. The rms errors for the station are also identical to the absolute value of the mean error, an indication that a persistent bias exists in the observations. The fact that the error is negative further indicates that the observations are biased high. With the information, the Profiler Control Center in Boulder was able to determine that the surface dewpoint sensor at Purcell had failed. The sensor has since been fixed, and the percentage of dewpoint observations failing the QC checks is back to zero.
Daily, weekly, and monthly summaries include only those stations with observations that have failed more than 25% of the time.
Last updated 25 July 2018
