The 2022 EFAS partner survey was sent to all partners in January 2023. This survey was intended to assess the general satisfaction with the EFAS service, products and performance during the year 2022. The 2022 survey was anonymous, and a total of 42 responses were received, which was similar to the number of responses received during the 2017-2021 surveys (Figure 1). 

Results from the survey showed that… 

• … the interest in EFAS has steadily increased since the first survey was conducted in 2017
• … the overall satisfaction with EFAS was very positive
• … the EFAS performance and forecast skill is rated as high or very high by the majority of respondents
• … EFAS provides good quality online documentation and resources
• … there is a high level of engagement with the EFAS partners
• … the EFAS map viewer interface has adequate content and functionalities, but it should be improved
• … the ability to access and download EFAS hydrological data should be better highlighted 

The survey responses are summarized according to the following categories:

• Demographics
• Overall Satisfaction
• Skill, Model Performance, and Trust
• EFAS Services
• EFAS Products and Added Value
• Feedback and Collaboration
• Future Developments

Demographics

The distribution of country origin for the survey respondents was similar in 2020-2022, but the percentage of respondents from the "others" category was highest in 2021.

As in 2021, most of the participants in the 2022 survey were either an EFAS partner or both a data provider and an EFAS partner. However, there were fewer EFAS third party partners and more data providers who responded to the 2022 survey compared to the 2021 survey.

Overall Satisfaction

No major changes in the overall satisfaction were reported in comparison to the previous five years. As with the previous surveys, the responses in 2022 were very positive for overall interest, satisfaction, and performance. A noteworthy trend in the survey responses is that there has been increasing interest in EFAS since 2017.

Skill, Model Performance and Trust

In 2022, the majority of the survey participants rated the EFAS model performance and forecast skill as high (55% of responses) or very high (7% of responses). However, 38% of respondents gave a neutral rating compared to 17% of respondents in 2021.

EFAS Services

In 2022, the EFAS Wiki, User Guide, News and Events, and Resources were the most-read informative resources with 81-88% of participants reading them at least sometimes. In contrast, the three reports were the least read resources with only 52-69% of participants reading them at least sometimes. Overall, there was a decrease in the proportion of participants who read the reports in 2022 compared to 2021; for the 2022 survey, 48% and 45% of the participants responded that they never read the report or did not know that it existed for the annual meteorological report and the detailed assessment report, respectively.

However, when the survey was sent to the participants, the 2021 annual hydrological and 2021 meteorological report had just been published and the 2021 Detailed Assessment Report had not yet been published. Thus, it is very reasonable that the survey uncovered somewhat lower interest for these reports as only the 2020 reports were available in 2022 (it is here to note that the delayed publication of the hydrological, meteorological, and detailed assessment report for the reference year was caused by exceptional circumstances; the reports for the reference year 2022 will be published in mid-2023, according to the regular schedule). Note that "Always" option was removed for the 2022 survey.

Survey participants provided similar responses in 2022 as in previous years when asked to identify their main benefits of being an EFAS partner. Like in 2020 and 2021, the most selected options in 2022 were notifications, forecasts, and learning practices during annual meetings. Respondents have also consistently indicated that observed and forecasted precipitation and partnership are important benefits of being an EFAS partner. It is here noted that the option "Training by DISS" was not provided after 2020.

New options introduced starting from the 2021 survey were: (1) Europe-wide overview on ongoing and forecasted events, and (2) To be able to see/monitor the situation/forecast of neighbouring basins. Although these two options were highly ranked in the 2021 survey, these options were not as highly ranked by the participants in the 2022 survey. One new option, pre-tasking of satellite images in case of severe events, was included in the 2022 survey, and 29% of the respondents indicated that this was among the main benefits of their EFAS partnership. For the 2022 survey, two participants also selected the option to specify other benefits and wrote that their benefits included (1) learning practices based on using EFAS IS and (2) additional products like social media, seasonal outlooks, satellite flood monitoring products.

EFAS provides complementary flood forecast information to relevant stakeholders to support flood risk management at the national, regional and global level. From 2021 to 2022, there was an increase in the percentage of respondents/organizations who compare EFAS forecasts with those from other providers; in 2022, 67% of participants responded "yes" compared to 44% in 2021. 

Participants who answered "Yes" in the 2022 survey were then asked to specify which systems they use and how those systems compare with EFAS. A summary of these systems is provided below:

• National System (8 participants) 
  • EFAS resolution at present is too coarse to forecast well for rivers in England and Wales.
• Local System (5 participants)
• Own System (8 participants 
  • We check EFAS against our own more highly resolved and locally adjusted system with station updates including water regulations in some places. We rate our own system higher compared to EFAS, but always double-check if EFAS is sending notifications.
  • We use our own forecasting modelling system (rainfall-runoff and hydrodynamic model). Our results are much better (timing, values, resolution, forecasted event description, performance).
• AEMET (2 participants)
• ALADIN
• AROME
• BSMEFFG (2 participants)
• Flood Early Warning System (FEWS)
• FFGS
• HARMONIE-AROME
• HypeWEB 
  • EFAS is far more precise.
• ICON ECMWF
• LHW Elbe WAVOS /SOBEK forecasts 
  • In the moment we just compare the results of normal or lower flow conditions and there the results compare well
• Numerical Models
• Pantha Rhei Model
• Spanish SAIH
• W-FLOW
• WMO SEEFFGS

Partners were asked to rank their preference on different forms of training with a rank of 1 indicating their first choice and a rank of 4 indicating their last choice. Overall, the partners in 2022 preferred to receive training in the form of:

• short online tutorials (average rank = 2.19; first choice of 14 partners) followed by 
• regular webinars (average rank = 2.31, first choice of 10 partners), whereas 
• workshops during the annual meeting (first choice of 11 partners) and 
• online documentation (first choice of 7 partners)

were the least preferred with average ranks of 2.68 and 2.76, respectively. Overall, however, these responses were similar to those received during the 2020 and 2021 surveys.

The survey results indicate that the participants were quite divided on whether or not they preferred receiving training in the form of workshops during the annual meeting. In total, 11 participants selected workshops as their first choice compared to 10 who chose regular webinars. However, there were 16 partners who ranked the workshops as their least preferred method (rank = 4) and this caused the average rank of the workshops to be worse than that of the webinars. 

The participants were then asked in which topic(s) they would like to receive training. Overall, flash-flood forecasts were the most popular option with 76% of participants indicating that they would like training on the topic. Approximately half of the participants were also interested in receiving training on the riverine forecasts (55%), impact-based forecasts (52%), and probabilistic forecasts (48%). One participant selected the “Other” option and wrote that they would like to receive training on LISFLOOD.

EFAS Products and Added Value

The ratings of the added value of the different EFAS notifications in 2022 were generally similar to those of the previous survey years. For 2021, survey participants rated the added value of the notifications very highly; in 2022, there was a slight decrease in the ratings of the added value. In general, however, most survey participants rate the added value of the different notifications as either high or very high, and there have not been major changes in the responses between survey years.

Satisfaction with each of the EFAS centres in 2022 was predominantly positive and similar to that of previous years. Note that the "Don't Know What They Do" option was introduced in the 2022 survey.

Ratings for the usability or added value of the EFAS products in 2022 were generally similar to those in 2021 and higher than those in 2020. Participants were especially positive about the added value of the MapViewer in EFAS-IS and the added value of the flash flood forecast layer. The added value of the sub-seasonal and seasonal forecast and the new social media activity layers were ranked lowest.

Overall, awareness of the different EFAS products and features in 2022 was similar to that of the previous surveys. From 2020-2022, the highest awareness was about the training material on the EFAS-IS. From 2021 to 2022, there was a large increase in awareness that the LISFLOOD hydrological model used in EFAS is now open source. Awareness was also generally high for: (1) pre-tasking of satellite imagery for monitoring in case of a flood event, and (2) the possibility to request training from the EFAS Dissemination Centre. In contrast, awareness was lowest about the possibility to download EFAS hydrological data from the Climate Data Store and the ability to access EFAS hydrological data through FTP.

Feedback and Collaboration

Survey participants were asked if they generally provide feedback to notifications. In total, 62% of the participants responded "yes" and 36% responded "no".

The survey participants who responded that they do not generally provide feedback to notifications were then asked to clarify why, and their responses are summarized below:

• No/few relevant notifications/events in their area few notifications/events (6 participants)
• Not implemented in their working procedure (2 participants)
• Lack of time/busy during flood events (2 participants) 
• Not a partner
• Because we are the national civil protection authority and in most cases we are not feedback of the events, as well as being a third party partner
• How can we estimate that a notification is a hit or a miss when the information is a probability (calculated on a different hydrograph than the post-processed one) or an areal percentage?
• I don't have habit to send feedback, because the notifications are too general and very uncertain
• I'm an EFAS OoD; so not on the receiving end of these directly

Some of the participants expressed their willingness to write short articles about EFAS, to conduct short verification analyses, or to evaluate new versions of EFAS before the operational phase. Note that the "Not Sure" and "N/A" options were removed for the 2022 survey.

Future Developments

As in 2020 and 2021, survey participants were asked in 2022 if they would like to receive quality flags for their hydrological/meteorological data and their data in a standard format (meaning harmonised time step and unit, quality controlled). For the 2022 survey, these questions were merged into one question with only options for "Yes" and "No". In total, 25 of the 42 survey participants in 2022 were EFAS data providers, and 60% responded "Yes" to the question. Note that the figure only shows the results for the 25 data providers, and this is why the red + blue areas is less than 100%.

Lastly, partners were also asked if there was anything else they would like to communicate to the EFAS team. These responses are summarized below:

• "Please reconsider the color scale because people who have color blindness have problem in EFAS map viewer reading."
• "I am strongly interested in knowing advances related to the implementation of reservoir performance in the hydrological model."
• "I am pleased to be a part of EFAS service. Thank you."
• "We would be interested in a training course in forecast/ensemble post-processing techniques."
• "As a Civil Protection organization, I think EFAS is very useful in meteorological and hydrological forecast and impact study, but the results are fewer in flash floods events. I hope it would be changes with TAMIR."
• "It would be desirable that the documentation is not only available in English but also in other languages."
• "We used to send mean daily discharge data, but it appears to be not very appropriate to make a good forecast (simulated discharges were far from reality, though the post-processed hydrograms that take real-time measurements into account were better). So, we discussed a lot with MDCC and hydroDCC, and finally we sent historical hourly discharge and rainfall data + real time hourly data, instead of daily data. It would be great to have a recalibration of our EFAS stations now, as detailled data is available. I hope it will drastically enhance the quality of the forecasts. I must say that is has been a pleasure to work with MDCC and HydroDCC, we had a lot of pleasant exchanges. HydroDCC told me that the integration of hourly discharge data should be finished by January (2023). MDCC already finished the integration in 2022."
• "The whole process that the team does is exceptional. Thank you for that!"
• "The planned upgrade - EFASNext - will make EFAS products much more relevant/useful to use operationally because LISFLOOD resolution will be high enough to forecast for rivers in England and Wales."
• "After the intense development of products in EFAS, one strong focus should now be on organizing better the information in the EFAS viewer, which is currently organized according to the projects that have been supporting EFAS and not according to the products. That is, from a user perspective it is not easy to navigate the viewer in a rapid way (there is always a need to consult the help sections, even accessing the viewer with some frequency. In other words the learning curve for using EFAS viewer is very slow."
• "I think it is important to continue to improve the EFAS/CEMS flood service through regular discussions with downstream users of the system/tool. It is very unclear how EFAS forecasts are uptaken into national flood warning services. In some countries, I don't think these are highly valued, as the national models are more reliable. But hopefully this will change in the near future, with improved EU-level products!"
• "We were a little bit confused about some of the questions in this survey."