Validation with ground-based data
Snow cover days derived from MODIS data (SCDMODIS) were validated with in-situ snow measurements at meteorological stations in Lithuania, Latvia and Estonia. There is no single statistical index which would describe all aspects of satellite data performance. Some indices (such as accuracy) are very sensitive to category which is the most common, while others (such as frequency bias) measure only relative frequency. Thus several indices should be used along to make valuable evaluation.
Comparison of annual SCDMODIS with ground data showed that MODIS tends to overestimate the number of snow cover days (Table 1). The correlation coefficients in the Baltic capitals were between 0.85 and 0.89, indicating good agreement between satellite measurements and in-situ data. The accuracy of SCDMODIS was 0.90–0.92, showing that most snowy days were determined correctly. The probability of detection (POD) at Vilnius and Tallinn was 0.83–0.85. At Riga it was 0.67, indicating that only 67 % of snow cover days observed in-situ were determined by satellite data. The threat score (TS) in these three locations varied from 0.61 to 0.72. This demonstrates that satellite-derived snow cover days corresponded well with ground-based observations.
One useful index for satellite-based product evaluation is Climatological Skill Score (SSclim). High positive SSclim values show an improvement over the climatological estimations based only on in-situ data. SSclim was high in Tallinn and Vilnius (0.71 and 0.79 respectively) and much lower (0.35) for Riga. Though SSclim values were low for Riga, MODIS is still better at determining snow cover than station climatological estimations.
Table 1: Statistical indices of MODIS-based annual SCD validation with in-situ data.
| Riga | Tallinn | Vilnius | |
|---|---|---|---|
| SCDinsitu | 89,2 | 111,5 | 93,1 |
| Mean absolute difference d, SCDMODIS – SCDinsitu | 20,9 | 10,7 | 8,9 |
| Standard deviation | 15,4 | 8,0 | 6,8 |
| Correlation coefficient | 0,87 | 0,85 | 0,89 |
| Relative difference, % | 1,40 | -0,48 | 1,40 |
| SSclim | 0,35 | 0,71 | 0,79 |
| Accuracy (Hit rate) | 0,90 | 0,91 | 0,92 |
| FAR (False alarm rate) | 0,13 | 0,15 | 0,15 |
| POD (Probability of detection) | 0,67 | 0,85 | 0,83 |
| TS (Threat score) | 0,61 | 0,74 | 0,72 |
Exercise 9
Based on statistical indices in Table 1, at which location, overall performance of MODIS was poorest?
The correct answer is: a) Riga.
In Riga standard deviation and mean absolute difference is highest and POD, SSclim, TS is lowest. Riga is at the coast of the Baltic Sea and snow cover in winter is often ephemeral and frequent thaws makes hard to correctly determine days with snow cover using satellite data.