Predicting wind speeds of tropical storms with Z by HP
Spring 2021
Product may differ from images depicted.
As a Z by HP Data Science Global Ambassador, Heng Cher Keng's content is sponsored and he was provided with HP products.
For 2021 I have set myself the target of taking part in 25 data science competitions in Kaggle, DataDriven, AiCrowd, etc. As a Z by HP Data Science Global Ambassador, I was fortunate enough to receive a HP Z8 G4 data science workstation to take part in competitions.
My first competition is “Wind-dependent Variables: Predict Wind Speeds of Tropical Storms,” hosted by Radiant Earth Foundation at DrivenData.
Every competition tells a story and this one is about:
“Two roads diverged in a wood, and I – I took the one less traveled by, and that has made all the difference.”
Robert Frost
American Poet
The organisation of the blog is as follows:
● Problem and solution: Summary of the data science problem that I am solving and my proposed solution.
● Experience on HP Z8 G4 workstation: How new computation power has empowered me to try something different.
Let’s begin the story!
Problem and Solution
Problem Statement:
Tropical cyclones are one of the costliest natural disasters globally. Hurricanes can cause upto 1,000 deaths and $50 billion in damages in a single event. The task for this competition is to build a model to estimate the maximum sustained surface wind speed (storm intensity) from new satellite images of tropical storms. Past history of images and wind speeds can also be used if available.
Data:
Single-band (10.3 microns) satellite images of 496 storms are provided. These are captured by the Geostationary Operational Environmental Satellites (GOES) orbiting around the Earth. Each storm consists of about 20 to 700 images, measuring 336 x336 pixels. In total, the train set has 70,257 images and the test set has 44,377 images.
Solution:
My solution is an ensemble of LSTM and transformer image-based regression models.
Figure.1. shows the construction of the LSTM encoder-decoder model. A imagenet pre-trained resnet-18d is used to encode the input image into a 512-dim embedding. Next, a 2-layer bidirectional LSTM is used to encode the past history images, time stamps and wind speeds into a context vector. Finally, another 2-layer LSTM is used for sequence decoding in the forecast interval. Given the encoder context vector, the decoder predicts the current wind speed using the current input image, time stamp and the last predicted wind speed.
Figure.1 LSTM encoder-decoder model
Figure.2. shows the construction of the transformer encoder-decoder model. Similarly, resnet-18d is used to encode the input image. Sinusoidal positional encoding is used for the time stamp. A 2-layer multi-head dot-product attention (MHA) encoder encodes the history into a set of “output weighted summed values”.
Another 2-layer MHA decoder predicts the wind speed using current input image as time stamp as query, and encoder output as key and value.
A triangular mask is used for self-attention in the decoder. This is a standard practice to prevent the decoder from “cheating” and accessing any future image and time stamp.
Note that unlike the LSTM, I make a subtle but important modification. The previous predicted wind speed is not used in the transformer decoder. Experimental results show that this reduces predicted error propagation error. The side effects are that the transformer’s prediction could also become more bumpy and less smooth.
Fortunately, results of LSTM and transformer are complimentary, giving rise to quite significant improvement when the different models are combined in the ensemble.
1. First train the baseline CNN encoder using a single image only.
2. Freeze the CNN encoder and train LSTM or transformer only from learning rate 0.001 to 0.0001. Here I am using the RADAM optimiser with lookahead implementation.
3. Finally, unfreeze the CNN encoder and train end-to-end with learning rate 0.0001.
Such freezing and unfreezing steps can prevent LSTM and transformers from overfitting.
Finally, Figure.3 is an example of input images and history wind speed and the predicted results.
Have a Question?
Contact Sales Support.
Follow Z by HP on Social Media
Need Help in Choosing the Best
Z Workstation Solutions?
Need Support for Your Z Workstation?
Disclaimers
Product may differ from images depicted.
Product images are for illustration purposes only, product availability and colours may vary by country.
© HP Development Company, L.P. The Information contained herein is subject to change without notice. Warranties for HP products and services are set out in the express warranty statements accompanying such products and services. In addition, our products and services come with guarantees that cannot be excluded under the Australian Consumer Law. Subject to the foregoing, nothing herein should be construed as constituting an additional warranty. HP shall not be liable for technical or editorial errors or omissions contained herein.