How is The Weather Company (TWC) data quality-controlled?
The Weather Company (TWC) uses a wide variety of processes to quality-control their data as it is being ingested into their systems. At a high level it does the following:
- Range Checks: Checks for data that is outside the bounds of normal readings.
- Stuck Sensor Checks: Readings that repeat consecutively multiple times over a given time period.
- Cluster Check: Computer algorithms to compare readings from a station to other surrounding stations for valid data checks.
- Additional gold star criteria:
- The weather station has observations less than an hour old.
- The weather station has not been failed by any of the data quality checks.
- The weather station has not been failed in the past 5 days.
- The weather station is not a Netatmo station.
TWC data is quality control tested before it makes it into their various cleaned, gridded products that Granular uses and is not adjusted after the fact.
It is 60 degrees outside and 20 stations on the map are showing temperatures around that, but there’s that one that shows a temperature of 75 degrees. That station may or may not be a gold star station, but the data is quality controlled before it makes it into the cleaned, gridded products Granular uses. In addition, the TWC gridded data is composed of many other sources of weather data for a given location besides a single weather station. If that station keeps reporting drastically different numbers for a given period of time, it would also lose its gold star status.