Blind testing of shoreline evolution models

TLDR

Beaches worldwide continuously adjust to changing wave and tide conditions, yet robust multi‑year shoreline predictions remain problematic even for short‑term scenarios. This study aimed to evaluate 19 shoreline models, including traditional and machine‑learning approaches, through a modelling competition using 18 years of daily shoreline and beach‑rotation data from Tairua beach, New Zealand. The competition calibrated models on 1999–2014 data and tested them on unseen 2014–2017 data, comparing individual model performance and an ensemble to assess predictive uncertainty. While most models reproduced calibration-period changes, they struggled with extreme, rapid oscillations and their predictive skill declined during the forecast period, especially for machine‑learning algorithms, whereas the ensemble outperformed individual models and highlighted the value of research‑coordinated competitions for advancing predictive capabilities.

Abstract

Abstract Beaches around the world continuously adjust to daily and seasonal changes in wave and tide conditions, which are themselves changing over longer time-scales. Different approaches to predict multi-year shoreline evolution have been implemented; however, robust and reliable predictions of shoreline evolution are still problematic even in short-term scenarios (shorter than decadal). Here we show results of a modelling competition, where 19 numerical models (a mix of established shoreline models and machine learning techniques) were tested using data collected for Tairua beach, New Zealand with 18 years of daily averaged alongshore shoreline position and beach rotation (orientation) data obtained from a camera system. In general, traditional shoreline models and machine learning techniques were able to reproduce shoreline changes during the calibration period (1999–2014) for normal conditions but some of the model struggled to predict extreme and fast oscillations. During the forecast period (unseen data, 2014–2017), both approaches showed a decrease in models’ capability to predict the shoreline position. This was more evident for some of the machine learning algorithms. A model ensemble performed better than individual models and enables assessment of uncertainties in model architecture. Research-coordinated approaches (e.g., modelling competitions) can fuel advances in predictive capabilities and provide a forum for the discussion about the advantages/disadvantages of available models.