Advances in Geosciences Performance of multi-model AEMET-SREPS precipitation probabilistic forecasts over Mediterranean area

Spanish Meteorological Agency (AEMET) runs a daily experimental multi-model Short-Range Ensemble Prediction System (AEMET-SREPS). The role of the system horizontal resolution (0.25 degrees) on the performance of 24-h precipitation probabilistic forecasts, and its relation with mesoscale events, are assessed comparing the performance over the Mediterranean area and over an European Atlantic area. Gridded high resolution rain observations and standard verification measures have been used at different precipitation thresholds, while studying the dependency on seasons for a one year period (May 2007 to June 2008). As a general result, performance over the Mediterranean area is higher than over the Atlantic one, albeit some relative loss of skill is found in autumn, when mesoscale convective organization is assumed to play a more important role. So it is suggested that AEMET-SREPS system precipitation predictability over the Mediterranean in autumn could be expected to improve if the horizontal and vertical resolution is increased in order to take into account the effect of meso-beta scale, especially important for convective organization.


Introduction
The multi-model Short-Range Ensemble Prediction system (AEMET-SREPS, García-Moya et al., 2009) is a daily experimental Limited Area Model (LAM) Ensemble Prediction System (EPS) focused on the short range (up to 72 h) with a 0.25 degree horizontal resolution and 40 vertical levels, developed at the Spanish Meteorological Agency (AEMET).To take implicitly model errors into account, five different LAMs are used (COSMO (COSMO), HIRLAM (HIRLAM Consortium), HRM (DWD), MM5 (NOAA) and UM-NAE Correspondence to: A. Callado (acalladop@aemet.es)(UKMO)), and in order to sample initial and boundary condition uncertainties, each model is integrated using data from four different global deterministic models (GFS (NCEP), GME (DWD), IFS (ECMWF) and UM (UKMO)), therefore the system comprises 20 members.
In order to assess the role of the system horizontal resolution (0.25 • ×0.25 • longitude and latitude, around 25 km) in the forecast performance for mesoscale events, 24-h probabilistic precipitation forecasts (see Table 1 for experiment details) have been compared over the Mediterranean area and an Atlantic area, because the former is expected to be more related to mesoscale events and the latter to synoptic scale flow.Moreover, a more general Total European area including the two previous ones has also been compared.The main aim is to assess whether the performance of AEMET-SREPS system, due to its 25 km meso-alpha horizontal resolution, can be improved over the Mediterranean area, where the meteorological mesoscale events (and hence horizontal resolution) play a more important role than in an Atlantic area.
Performance over the three areas has been assessed focusing on 24-h accumulation precipitation (from t+6 to t+30 hour lead times, and from t+30 to t+54), comparing with gridded observations and taking into account the seasonal variations along one year period (May 2007 to June 2008) at 1, 5, 10 and 20 mm precipitation thresholds.Standard probabilistic verification methods (see following section for details) have been followed (Brier, 1950;Murphy, 1973;Zhu et al., 2002;Jolliffe and Stephenson, 2003;Candille and Talagrand, 2005).

Verification methodology
Observed precipitation data from High Resolution networks over Europe have been used as reference for a one year period from May 2007 to June 2008.Observations are collected at ECMWF from member and cooperating states, and available Published by Copernicus Publications on behalf of the European Geosciences Union.gridded (after a basic quality control) using an up-scaling method (Cherubini and Ghelli, 2002) which reduces (i) the impact of spatial density of observations and (ii) the potential lack of statistical significance due to spatial dependence between close ones.To improve the quality control, only those boxes with at least five observations inside have been used.The grid resolution fits that one of AEMET-SREPS system and on each box the average value is taken as the observed precipitation.Figure 1 shows the raw observations and the gridded ones for Mediterranean and Atlantic areas; notice that both areas present roughly the same number of boxes (around 355) in order to achieve similar sample sizes and statistical significance.
A standard probabilistic verification experiment following ECMWF recommendations (Nurmi, 2003) has been carried out (see Table 1), assessing probability forecast quality with common properties like reliability, resolution and discrimination, using standard performance measures and the corresponding graphs: Attributes Diagram (Hsu and Murphy, 1986), reliability and resolution components of Brier Score (BS; Brier, 1950) decomposition (Murphy, 1973), Brier Skill Score (BSS) decomposition (Candille and Talagrand, 2005), Relative-Operating-Characteristic (ROC) curve (Jolliffe and Stephenson, 2003) and Relative Value diagrams (RV; Zhu et al., 2002).The assessment has been done at four rainfall thresholds 1, 5, 10 and 20 mm.For BSS and RV the sample climatology is used as reference (Mason, 2004).

Results and discussion
As an overview of the sample climatology, Fig. 2a and b show, respectively, the observed precipitation distribution (probability density function; PDF) along different precipitation intervals, and the complementary cumulative distribution function (CDFC) along different precipitation thresholds, i.e. the frequencies of occurrence (base rates), both for  summer and autumn seasons.The sample is in agreement with the climatological features of mesoscale precipitation over Mediterranean and European Atlantic areas (e.g.Mehta and Yang, 2008).In general terms, Mediterranean area with respect to the Atlantic one shows less precipitation cases, but more cases of high precipitation (hence a more frequent 20 mm binary event), especially in autumn, probably related to more convective activity.Figures 3 (attributes, ROC and RV for summer at 20 mm), 4 (the same for autumn) and 5 (BSS time series for 1 mm and 20 mm) show a summary of verification results for 24-h probabilistic precipitation forecast.In general terms, AEMET-SREPS is a reliable and skilful system for the three selected areas.Specifically, the forecasts over Mediterranean area for all seasons, except for autumn, and for all thresholds, except for 1 mm, show better performance than over Atlantic area (see summary results in Table 2).Similar results are obtained for both t+6 to t+30 (shown) and t+30 to t+54 (not shown) accumulation precipitation periods.The Total Europe area shows, in terms of verification scores, an average behaviour between Mediterranean and Atlantic areas, both included in it.These results, together with the sample climatology distribution, reveal that a better system performance is related with predominant regimes: in the Atlantic area, where large-scale precipitation is predominant, a better performance than on Mediterranean area is found for the lower threshold (1 mm); whereas in the Mediterranean area, where convective precipitation prevails, the results overcome that of Atlantic area for the higher thresholds (20 mm).According to the sample climatology, the 20 mm threshold shows higher frequency in summer and autumn, related on average to convective precipitation.For spring, summer and winter it can be seen that Mediterranean results overcome Atlantic ones in terms of reliability, resolution and skill in 5, 10 (not shown) and 20 mm (shown) thresholds.But for Mediterranean area there is a relative loss of skill in autumn with respect to the Atlantic area, higher in 10 and 20 mm thresholds, which is not in agreement with the previous explanation.Therefore it is suggested that this relative loss is probably due to the fact that in autumn the convective precipitation on the Mediterranean Sea influence is more related to mesoscale organization, not properly resolved by the system.The lack of a proper simulation of the mesoscale organization of the convection is a well known deficiency of NWP models at these horizontal scales (e.g.Palmer, 1997 and2001)   neither convective parameterization nor explicit convection driven by dynamics and synergies of both of them are capable to resolve it properly.These results point out that in AEMET-SREPS system the representation of mesoscale meteorological events needs improvement, especially around the Mediterranean basin in autumn when the mesoscale organization of the convection plays a more relevant role.And not only the mesoscale convective organization, but probably also the orographicrelated processes like the orographic enhancement of the precipitation have to be improved.So it is suggested that the skill of AEMET-SREPS system around the Mediterranean in autumn could be expected to improve if the horizontal and vertical resolution of each LAM member of it is increased in order to take the meso-beta scale into account.Theoretically, three new configurations could be possible with different horizontal resolutions: (a) first one at 12 km close to the hydrostatic limit, maybe still being possible to use hydrostatic NWP models; (b) second at 4 km taking certainly yet non-hydrostatic NWP models, but with the dichotomy if use (and how to use) or not use convective parameterization; and (c) finally at 1 km, without convective parameterization, but raising out the issue of how to get suitable initial and boundary conditions for LAMs at this quite high horizontal resolution, with the probable consequence of moving to smaller NWP model integration areas due to limited computer resources, and then expecting that boundary conditions could dominate the simulations.
Figure 1.A random day observation locations (blue points in the smaller map) for raw 24-hour precipitation data from High Resolution networks over Europe provided by ECMWF and the corresponding up-scaling to 0.25ºx0.25ºlongitude and latitude boxes when at least five observations are available, for Mediterranean area (red boxes) and Atlantic area (green boxes).
Fig. 3. (a) Attributes diagram, (b) Relative-operatingcharacteristics (ROC) curve and (c) Relative Value(RV) diagram to respectively assess reliability, discrimination and value with respect to sample climatology for the 24-h precipitation over 20 mm probability forecast in 2007 boreal summer (June-July-August (JJA)) over Mediterranean area (red lines), Atlantic area (green lines) and Total Europe area (blue lines).

Table 1 .
Objective verification experiment settings and characteristics.

Table 2 .
AEMET-SREPS probabilistic precipitation forecasts performance.Summary over Mediterranean area with respect to Atlantic area in terms of standard scores for EPSs evaluation.