Urban Flood Forecasting

Stage by Stage

1. 1
   Stage 1

   Data Ingestion & Validation

   ExampleGen ingests rainfall time-series and geospatial flood extent data from Google Cloud Storage and partitions it into training and evaluation splits. StatisticsGen, SchemaGen, and ExampleValidator then run in concert to compute dataset statistics, infer a data schema, and detect anomalies — blocking malformed or drifted data from reaching the training stage.

2. 2
   Stage 2

   Feature Engineering

   The Transform component applies spatial normalization and temporal windowing to prepare inputs for the CNN+LSTM architecture. Critically, the same transformation graph is applied at both training and serving time — eliminating training-serving skew and ensuring that the preprocessing the model was trained on is identical to what it receives during real-time inference.

3. 3
   Stage 3

   Hyperparameter Tuning

   The Tuner component integrates Keras Tuner to search across CNN filter sizes, LSTM hidden units, learning rate, and dropout rate. The best hyperparameter trial is automatically passed to the Trainer component — removing manual tuning from the retraining loop and ensuring every pipeline run uses an optimized configuration.

4. 4
   Stage 4

   CNN+LSTM Training

   The Trainer component trains the CNN+LSTM model using the tuned hyperparameters and transformed features. The CNN encodes spatial rainfall patterns and their interaction with topographic features, the LSTM captures temporal accumulation dynamics, and a custom synchronization layer accounts for the temporal lag between storm events and inundation — the key architectural innovation that drives the 89% NSE.

5. 5
   Stage 5

   Evaluation & Model Blessing

   The Evaluator uses TensorFlow Model Analysis (TFMA) to assess the trained model against NSE, MSE, and spatial accuracy thresholds. Only models that meet or exceed all thresholds receive a blessing. The Resolver component then compares the newly trained model against the current production champion — auto-promoting it only if performance genuinely improves.

6. 6
   Stage 6

   Serving & Deployment

   The Pusher component deploys blessed models to a Vertex AI Serving endpoint, making the flood prediction system available for real-time inference. Model versions are tracked in the Vertex AI Model Registry — enabling challenger vs. champion comparisons, versioned rollbacks, and full audit trails for every production deployment.

89%

Nash–Sutcliffe Efficiency

Nash-Sutcliffe Efficiency (NSE) — validated against Calgary's historical flood events including the 2013 disaster

<15cm

Avg. Training MSE

Average mean squared error on flood depth prediction across training runs — well within operational tolerance

Vertex AI

Production Deployment

Full TFX pipeline deployed on GCP Vertex AI — automated retraining, model registry, and real-time serving endpoint

• High-resolution flood hazard maps from rainfall alone

  The system generates spatially explicit flood extent and depth predictions across Calgary's Bow and Elbow River watersheds using only rainfall as input — no soil moisture data, no river gauge calibration, no physics-based solver outputs required. This dramatically reduces the data burden for operational deployment.

• Climate scenario stress-testing to 2100

  The model was applied to RCP 8.5 rainfall projections for 2025, 2050, 2080, and 2100 — producing future flood hazard maps that reveal how high-risk zones expand and deepen under high-emission climate pathways. These outputs directly inform long-horizon infrastructure investment and capital planning decisions for the client.

• Fully automated production pipeline on GCP

  The TFX pipeline runs end-to-end without manual intervention — ingesting new data, validating schema, engineering features, tuning hyperparameters, training, evaluating, and deploying in a single automated DAG. New rainfall data triggers a retraining run; only models that improve on the production champion are promoted to the serving endpoint.

• Validated against Calgary's historic flood events

  The model was trained and validated on flood events from 2010–2020 including the catastrophic 2013 Southern Alberta flood. Successful replication of observed flood extent and depth from those events — at NSE 89% — confirms the system's reliability for both current operational use and future scenario projection.

• Full reproducibility and lineage across every pipeline run

  Every artifact, execution, transformation, and evaluation result is logged in the ML Metadata Store — creating a complete audit trail from raw data to deployed model. Any pipeline run can be fully reconstructed, any model version can be rolled back, and every prediction can be traced to the exact data and code that produced it.