Improving discrimination between clouds and optically thick aerosol plumes in geostationary satellite data

Daniel Robbins; Caroline Poulsen; Steven Siems; Simon Proud

doi:10.5194/amt-15-3031-2022

Improving discrimination between clouds and optically thick aerosol plumes in geostationary satellite data

Daniel Robbins
, Caroline Poulsen
, Steven Siems
, Simon Proud

School of Earth Atmosphere and Environment

Research output: Contribution to journal › Article › Research › peer-review

8 Citations (Scopus)

Abstract

Cloud masking is a key initial step in the retrieval of geophysical properties from satellite data. Despite decades of research, problems still exist of over- or underdetection of clouds. High aerosol loadings, in particular from dust storms or fires, are often classified as clouds, and vice versa. In this paper, we present a cloud mask created using machine learning for the Advanced Himawari Imager (AHI) aboard Himawari-8. In order to train the algorithm, a parallax-corrected collocated data set was created from AHI and Cloud-Aerosol Lidar with Orthogonal Polarization (CALIOP) lidar data. Artificial neural networks (ANNs) were trained on the collocated data to identify clouds in AHI scenes. The resulting neural network (NN) cloud masks are validated and compared to cloud masks produced by the Japanese Meteorological Association (JMA) and the Bureau of Meteorology (BoM) for a number of different solar and viewing geometries, surface types and air masses. Here, five case studies covering a range of challenging scenarios for cloud masks are also presented to demonstrate the performance of the masking algorithm. The NN mask shows a lower false positive rate (FPR) for an equivalent true positive rate (TPR) across all categories, with FPRs of 0.160 and 0.259 for the NN and JMA masks, respectively, and 0.363 and 0.506 for the NN and BoM masks, respectively, at equivalent TPR values. This indicates the NN mask accurately identifies 1.13 and 1.29 times as many non-cloud pixels for the equivalent hit rate when compared to the JMA and BoM masks, respectively. The NN mask was shown to be particularly effective in distinguishing thick aerosol plumes from cloud, most likely due to the inclusion of the 0.47 and 0.51 μm bands. The NN cloud mask shows an improvement over current operational cloud masks in most scenarios, and it is suggested that improvements to current operational cloud masks could be made by including the 0.47 and 0.51 μm bands. The collocated data are made available to facilitate future research.

Original language	English
Pages (from-to)	3031-3051
Number of pages	21
Journal	Atmospheric Measurement Techniques
Volume	15
Issue number	9
DOIs	https://doi.org/10.5194/amt-15-3031-2022
Publication status	Published - 17 May 2022

Access to Document

10.5194/amt-15-3031-2022Licence: CC BY

Cite this

@article{bafce1f51908438ab8b259594d48bc53,

title = "Improving discrimination between clouds and optically thick aerosol plumes in geostationary satellite data",

abstract = "Cloud masking is a key initial step in the retrieval of geophysical properties from satellite data. Despite decades of research, problems still exist of over- or underdetection of clouds. High aerosol loadings, in particular from dust storms or fires, are often classified as clouds, and vice versa. In this paper, we present a cloud mask created using machine learning for the Advanced Himawari Imager (AHI) aboard Himawari-8. In order to train the algorithm, a parallax-corrected collocated data set was created from AHI and Cloud-Aerosol Lidar with Orthogonal Polarization (CALIOP) lidar data. Artificial neural networks (ANNs) were trained on the collocated data to identify clouds in AHI scenes. The resulting neural network (NN) cloud masks are validated and compared to cloud masks produced by the Japanese Meteorological Association (JMA) and the Bureau of Meteorology (BoM) for a number of different solar and viewing geometries, surface types and air masses. Here, five case studies covering a range of challenging scenarios for cloud masks are also presented to demonstrate the performance of the masking algorithm. The NN mask shows a lower false positive rate (FPR) for an equivalent true positive rate (TPR) across all categories, with FPRs of 0.160 and 0.259 for the NN and JMA masks, respectively, and 0.363 and 0.506 for the NN and BoM masks, respectively, at equivalent TPR values. This indicates the NN mask accurately identifies 1.13 and 1.29 times as many non-cloud pixels for the equivalent hit rate when compared to the JMA and BoM masks, respectively. The NN mask was shown to be particularly effective in distinguishing thick aerosol plumes from cloud, most likely due to the inclusion of the 0.47 and 0.51 μm bands. The NN cloud mask shows an improvement over current operational cloud masks in most scenarios, and it is suggested that improvements to current operational cloud masks could be made by including the 0.47 and 0.51 μm bands. The collocated data are made available to facilitate future research.",

author = "Daniel Robbins and Caroline Poulsen and Steven Siems and Simon Proud",

note = "Funding Information: The CALIOP L2 1 km cloud-layer data were obtained from the NASA Langley Research Center Atmospheric Science Data Center. This research/project was undertaken with the assistance of resources and services from the National Computational Infrastructure (NCI), which is supported by the Australian Government. Funding Information: Acknowledgements. The CALIOP L2 1 km cloud-layer data were obtained from the NASA Langley Research Center Atmospheric Science Data Center. This research/project was undertaken with the assistance of resources and services from the National Computational Infrastructure (NCI), which is supported by the Australian Government. Publisher Copyright: {\textcopyright} 2022 Daniel Robbins et al.",

year = "2022",

month = may,

day = "17",

doi = "10.5194/amt-15-3031-2022",

language = "English",

volume = "15",

pages = "3031--3051",

journal = "Atmospheric Measurement Techniques",

issn = "1867-1381",

publisher = "European Geosciences Union",

number = "9",

}

TY - JOUR

T1 - Improving discrimination between clouds and optically thick aerosol plumes in geostationary satellite data

AU - Robbins, Daniel

AU - Poulsen, Caroline

AU - Siems, Steven

AU - Proud, Simon

N1 - Funding Information: The CALIOP L2 1 km cloud-layer data were obtained from the NASA Langley Research Center Atmospheric Science Data Center. This research/project was undertaken with the assistance of resources and services from the National Computational Infrastructure (NCI), which is supported by the Australian Government. Funding Information: Acknowledgements. The CALIOP L2 1 km cloud-layer data were obtained from the NASA Langley Research Center Atmospheric Science Data Center. This research/project was undertaken with the assistance of resources and services from the National Computational Infrastructure (NCI), which is supported by the Australian Government. Publisher Copyright: © 2022 Daniel Robbins et al.

PY - 2022/5/17

Y1 - 2022/5/17

N2 - Cloud masking is a key initial step in the retrieval of geophysical properties from satellite data. Despite decades of research, problems still exist of over- or underdetection of clouds. High aerosol loadings, in particular from dust storms or fires, are often classified as clouds, and vice versa. In this paper, we present a cloud mask created using machine learning for the Advanced Himawari Imager (AHI) aboard Himawari-8. In order to train the algorithm, a parallax-corrected collocated data set was created from AHI and Cloud-Aerosol Lidar with Orthogonal Polarization (CALIOP) lidar data. Artificial neural networks (ANNs) were trained on the collocated data to identify clouds in AHI scenes. The resulting neural network (NN) cloud masks are validated and compared to cloud masks produced by the Japanese Meteorological Association (JMA) and the Bureau of Meteorology (BoM) for a number of different solar and viewing geometries, surface types and air masses. Here, five case studies covering a range of challenging scenarios for cloud masks are also presented to demonstrate the performance of the masking algorithm. The NN mask shows a lower false positive rate (FPR) for an equivalent true positive rate (TPR) across all categories, with FPRs of 0.160 and 0.259 for the NN and JMA masks, respectively, and 0.363 and 0.506 for the NN and BoM masks, respectively, at equivalent TPR values. This indicates the NN mask accurately identifies 1.13 and 1.29 times as many non-cloud pixels for the equivalent hit rate when compared to the JMA and BoM masks, respectively. The NN mask was shown to be particularly effective in distinguishing thick aerosol plumes from cloud, most likely due to the inclusion of the 0.47 and 0.51 μm bands. The NN cloud mask shows an improvement over current operational cloud masks in most scenarios, and it is suggested that improvements to current operational cloud masks could be made by including the 0.47 and 0.51 μm bands. The collocated data are made available to facilitate future research.

AB - Cloud masking is a key initial step in the retrieval of geophysical properties from satellite data. Despite decades of research, problems still exist of over- or underdetection of clouds. High aerosol loadings, in particular from dust storms or fires, are often classified as clouds, and vice versa. In this paper, we present a cloud mask created using machine learning for the Advanced Himawari Imager (AHI) aboard Himawari-8. In order to train the algorithm, a parallax-corrected collocated data set was created from AHI and Cloud-Aerosol Lidar with Orthogonal Polarization (CALIOP) lidar data. Artificial neural networks (ANNs) were trained on the collocated data to identify clouds in AHI scenes. The resulting neural network (NN) cloud masks are validated and compared to cloud masks produced by the Japanese Meteorological Association (JMA) and the Bureau of Meteorology (BoM) for a number of different solar and viewing geometries, surface types and air masses. Here, five case studies covering a range of challenging scenarios for cloud masks are also presented to demonstrate the performance of the masking algorithm. The NN mask shows a lower false positive rate (FPR) for an equivalent true positive rate (TPR) across all categories, with FPRs of 0.160 and 0.259 for the NN and JMA masks, respectively, and 0.363 and 0.506 for the NN and BoM masks, respectively, at equivalent TPR values. This indicates the NN mask accurately identifies 1.13 and 1.29 times as many non-cloud pixels for the equivalent hit rate when compared to the JMA and BoM masks, respectively. The NN mask was shown to be particularly effective in distinguishing thick aerosol plumes from cloud, most likely due to the inclusion of the 0.47 and 0.51 μm bands. The NN cloud mask shows an improvement over current operational cloud masks in most scenarios, and it is suggested that improvements to current operational cloud masks could be made by including the 0.47 and 0.51 μm bands. The collocated data are made available to facilitate future research.

UR - https://www.scopus.com/pages/publications/85130475393

U2 - 10.5194/amt-15-3031-2022

DO - 10.5194/amt-15-3031-2022

M3 - Article

AN - SCOPUS:85130475393

SN - 1867-1381

VL - 15

SP - 3031

EP - 3051

JO - Atmospheric Measurement Techniques

JF - Atmospheric Measurement Techniques

IS - 9

ER -

Improving discrimination between clouds and optically thick aerosol plumes in geostationary satellite data

Abstract

Access to Document

Other files and links

SAEF: Securing Antarctica’s Environmental Future – An Evidence-Based, Informatics Approach

ARC Centre of Excellence for Climate Extremes

Cite this

Improving discrimination between clouds and optically thick aerosol plumes in geostationary satellite data

Abstract

Access to Document

Other files and links

Projects

SAEF: Securing Antarctica’s Environmental Future – An Evidence-Based, Informatics Approach

ARC Centre of Excellence for Climate Extremes

Cite this