QA/QC procedure
Automated procedure for telemetry data
The QA/QC procedures below are used to automatically process the wave data from Datawell Directional Waverider buoys. The aim of the procedures is to produce good valid data that has passed through a series of algorithms that are easily understood and well documented. The algorithms may not remove all unwanted data but user feedback suggested that people wanted all the data with a relatively "light touch" in terms of QC. However, in certain circumstances (e.g. large changes in Tpeak between wind and swell seas) may cause good data to be flagged out by this automated process. Post-recovery data will undergo a fuller set of QA/QC procedures before release and it is therefore advisable to download post-recovery data from the WaveNet website where available.
Datawell Directional Waverider Data via ORBCOMM and Iridium and HF telemetry
Whilst the data rate for HF telemetry is continuous, with full spectra from the buoy being generated every 30 minutes, for ORBCOMM and Iridium the rate can be set as required with the maximum frequency of data every 30 minutes. Both ORBCOMM and Iridium systems use handshaking and error checking to ensure that data corruption is extremely rare. ORBCOMM coverage is not continuous and therefore there may still be the occasional time delay for the data to reach the Cefas server for processing. The Iridium network has global coverage and the telemetry is near-real time with maximum data latencies of only a few minutes.
a) Calculation of wave parameters
For valid data messages, wave parameters are calculated using algorithms provided by Datawell to decode ORBCOMM and Iridium telemetry data and spectra. The ORBCOMM and Iridium spectra have 13 and 27 frequency bins respectively. The frequencies of these bins can vary from message to message as they are calculated to best represent the specific dataset. The files from the HF telemetry system contain the SPT 64-bin full buoy spectra (SPT), which are identical to those downloaded following recovery of the buoy. The wave parameters, also decoded from the messages, are stored in the database for display in the tables, graphs and plots.
b) Range checking of data
The data are then checked against the following maximum and minimum values:
| Parameter | Units | Minimum value | Maximum value |
|---|---|---|---|
| Wave Height (Hsig) | Metres | 0 | 20 |
| Tpeak | Seconds | 1.6 | 20 to 25* |
| Tz | Seconds | 1.6 | 20 |
| Wave direction | Degrees | 0 | 360 |
| Wave spread | Degrees | 0 | 90 |
| Temperature | Degrees Celsius | -5 | 46 |
Data falling outside the ranges are stored to the database and flagged, but not displayed in the tables, graphs or plots.
c) Removal of spikes
The automated de-spiking of the data checks if the absolute change from the previous data point exceeds a maximum 'rate of change' for assigned to the parameter. The following values are used:
| Parameter | Units | Maximum rate of change per hour |
|---|---|---|
| Wave Height (Hsig) | Metres | 3 to 6* |
| Tpeak | Seconds | 6 to 8* |
| Tz | Seconds | 3 |
| Temperature | Degrees Celsius | 3 |
Data falling outside the 'rate of change' criteria are stored to the database and flagged, but not displayed in the tables, graphs or plots.
Third Party Wave data via Global Telecommunications System (GTS)
a) Calculation of wave parameters
The UK Met Office routinely loads files containing third party parameter-only data in SHIP code format onto a dedicated FTP server at Cefas. It is understood that these data have already undergone basic QA prior to transfer. The GTS Data Importing software regularly processes the files and decodes wave parameters, which are then stored in the database for display in the tables, graphs and plots.
b) Range checking of data
The data are then checked against the following maximum and minimum values:
| Parameter | Units | Minimum value | Maximum value |
|---|---|---|---|
| Wave Height (Hsig) | Metres | 0 | 20 to 25* |
| Tz | Seconds | 1.6 | 20 to 30* |
| Temperature | Degrees Celsius | -5 | 35 |
Data falling outside the ranges are stored to the database and flagged, but not displayed in the tables, graphs or plots.
c) Removal of spikes
The automated de-spiking of the data checks if the absolute change from the previous data point exceeds a maximum 'rate of change' for assigned to the parameter. The following values are used:
| Parameter | Units | Maximum rate of change per hour |
|---|---|---|
| Wave Height (Hsig) | Metres | 3 to 6* |
| Tz | Seconds | 3 to 4* |
| Temperature | Degrees Celsius | 3 |
Data falling outside the 'rate of change' criteria are stored to the database and flagged, but not displayed in the tables, graphs or plots.
* Data from buoys/platforms to the west of the UK generally have the higher maximum value and ‘rate of change’ applied.
