Natural hazard prediction and damage assessment using multimodal satellite and geospatial data in self-supervised XAI model

Natural hazards, including bushfires and floods, are a constant threat to world’s ecosystem. The occurrence of fire events, for example, depends on multiple factors such as the type of fuel present in the vegetation, temperature of land etc.

Data gathered by different satellite systems can be a key in predicting future fire events and damage assessment. With the availability of large amounts of data, Artificial Intelligence (AI) systems can be developed to help minimize the damage caused by these natural hazards. With data growing exponentially, annotating it is becoming a next big challenge. However, due to the heterogeneity of remote sensing measurements and tasks, there is no single go-to dataset that could serve the purpose of benchmarking. Moreover, annotating this type of data is also a challenge.

To deal with this issue the development of supervised learning algorithm is being shifted towards self supervised learning. These systems operate with high precision and make real time decisions with little to no human intervention. However, the black-box nature of these models gives rise to another set of questions such as which classifier is best for a given condition, what factors are controlling the accuracy of the model. we aim to develop an explainable and interpretable selfsupervised deep learning model that can infuse multi-modal data gathered from satellites and open data sources.

Development of an interpretable decision support system can help organizations take timely actions to prevent the spread of a natural disaster such as forest fires. Our approach would involve training an interpretable vision transformer in a self-supervised manner that could fuse information from multi-modal satellite and geospatial data. For this purpose, Data can be acquired from a range of satellite data available. Datasets of PRISMA, MODIS, Sentinel-2 and Landsat can contribute to a great deal in the development of AI based systems for hazard detection, area mapping and damage assessment.


Project Leader:
Professor Flora Salim, University of New South Wales

PhD Student:
Hira Saleem, University of New South Wales