Satellite image processing

Raw sensor output is not an image and an image is not a measurement. The processing chain between them is where most of the scientific and commercial value is added, and where most systematic errors originate.

The chain

Two corrections do most of the work, and both are where errors that look like real signals are introduced.

Radiometric correction

The detector does not measure light uniformly. Each element has its own dark current, gain and nonlinearity, and these drift with temperature and age.

Flat fielding deserves emphasis because its failure mode is deceptive. An error produces a smooth gradient across the image, which looks like a real geophysical pattern rather than like noise. Analysts have repeatedly misinterpreted calibration gradients as environmental signals.

Absolute calibration drifts, so it is maintained by imaging vicarious calibration sites: stable, uniform, well-characterised surfaces such as certain desert regions and dry lakebeds whose reflectance is measured on the ground. Cross-calibration against other satellites viewing the same target provides an independent check.

Geometric correction and orthorectification

Turning image coordinates into ground coordinates requires knowing where the satellite was, where it was pointing, and what the terrain looks like.

A point at elevation $h$ imaged at off-nadir angle $\theta$ appears displaced by approximately $h\tan\theta$ from its true ground position. At 20° off-nadir, a 1000 m hill is displaced by 364 m. Orthorectification removes this using a digital elevation model.

Geolocation accuracy is limited by three things together: the orbit determination, the attitude knowledge, and the elevation model. The attitude term is why GNC emphasises that knowledge matters more than control accuracy. A satellite that points imprecisely but knows precisely where it pointed can still geolocate correctly; the reverse cannot be fixed on the ground.

Bundle adjustment using ground control points or tie points between overlapping scenes improves this substantially, and modern systems achieve geolocation of a few metres without any ground control.

Atmospheric correction

The sensor measures top-of-atmosphere radiance. What is usually wanted is surface reflectance, and the atmosphere between them scatters, absorbs and adds path radiance of its own.

Without correction, two images of the same unchanged scene on different days differ, because the atmosphere changed. Any change detection on uncorrected data measures weather as much as ground change.

Aerosol loading is the hardest input, because it varies rapidly in space and time and is poorly observed. It is the dominant residual uncertainty in surface reflectance products.

Cloud masking

For optical imaging, a large fraction of acquired pixels are cloud, and cloud must be identified before anything else is done with the data.

The problem is harder than it appears. Thin cirrus is semi-transparent and partially contaminates the underlying signal without obviously appearing as cloud. Cloud shadows corrupt reflectance without being bright. And bright surfaces, snow, ice, sand and salt flats, are readily confused with cloud by threshold methods.

Classical approaches use spectral thresholds exploiting cloud brightness and temperature. Learned approaches perform better on the difficult cases, particularly thin cirrus and snow discrimination, and cloud masking is one of the clearest successes of machine learning in the field, because ground truth is plentiful and the consequence of an error is bounded.

Change detection

The application that motivates most of the correction chain.

Every box before the difference operation exists to prevent a false positive. Misregistration of even one pixel produces spurious change along every edge in the scene, which is why co-registration accuracy is often specified more tightly than absolute geolocation.

Coherent change detection with SAR is a different and more sensitive technique. Because SAR records phase, comparing two acquisitions detects changes that alter the scattering geometry at sub-wavelength scale, which is far below what the resolution alone would suggest. See Earth observation.

Onboard against ground

The traditional architecture downlinks raw or lightly compressed data and processes on the ground, where computation is cheap and the full chain can be applied.

The economics favour onboard processing precisely where the output is much smaller than the input. Discarding a clouded scene onboard saves the entire downlink cost of that scene, and reporting a ship detection instead of the image containing it reduces gigabytes to kilobytes. See Edge computing in space.

The counterargument is that onboard processing is irreversible. Data discarded in orbit cannot be recovered, and a cloud mask that wrongly rejects a scene has destroyed an observation nobody can retake. This argues for onboard processing that prioritises rather than deletes wherever storage allows.

References

• Schowengerdt, R. A. Remote Sensing: Models and Methods for Image Processing, 3rd ed., Academic Press, 2007.
• Toutin, T. “Geometric processing of remote sensing images”, International Journal of Remote Sensing, 25:1893, 2004.
• Vermote, E. F. et al. “Second Simulation of the Satellite Signal in the Solar Spectrum, 6S”, IEEE TGRS, 35:675, 1997.
• Zhu, Z. and Woodcock, C. E. “Object-based cloud and cloud shadow detection in Landsat imagery”, Remote Sensing of Environment, 118:83, 2012.
• CCSDS 122.0-B and 123.0-B, image data compression recommended standards.