Summary EarthPT a foundation model for Earth Observation arxiv.org
3,680 words - PDF document - View PDF document
One Line
EarthPT is a powerful pretrained transformer model for Earth Observation that accurately predicts future reflectance values and remote sensing indices, with the aim of demonstrating its wide utilization and impact.
Slides
Slide Presentation (9 slides)
Key Points
- EarthPT is an Earth Observation (EO) pretrained transformer model developed for accurately predicting future pixel-level surface reflectances.
- EarthPT is trained in an autoregressive self-supervised manner and has a large number of parameters (700 million).
- The model outperforms simple phase-folded models based on historical averaging in forecasting the evolution of the Normalised Difference Vegetation Index (NDVI).
- EarthPT embeddings hold semantically meaningful information and can be used for downstream tasks such as land use classification.
- The abundance of EO data allows for scaling EarthPT and similar models without data-imposed limits.
- EarthPT has the potential to make accurate long-term forecasts and mitigate future events associated with environmental threats.
- The model can be used for land cover classification and has applications in various sectors such as agriculture and insurance.
- Future work includes deriving a specific scaling law for EO datasets and training larger models using more data.
Summaries
40 word summary
EarthPT is a pretrained transformer model for Earth Observation (EO) that accurately predicts future reflectance values. It can forecast surface-level optical reflectance and remote sensing indices months into the future. The researchers aim to demonstrate its wide utilization and impact.
162 word summary
EarthPT is a pretrained transformer model designed specifically for Earth Observation (EO) use-cases. It is a 700 million parameter decoding transformer that can accurately predict future pixel-level surface reflectances across the 400-2300 nm range. The model
The researchers aim to demonstrate that a transformer model trained on Earth Observation (EO) data can scale and have wide utilization and impact, similar to natural language models. They show that their model, EarthPT, accurately forecasts reflectance values in the future,
The EarthPT foundation model for Earth Observation is capable of forecasting surface-level optical reflectance and common remote sensing indices at the pixel level, months into the future. It uses indicators such as Normalised Difference Water Index (NDWI), Bare Soil Index (
This text excerpt includes a list of references to various research papers and articles related to topics such as training compute-optimal large language models, reinventing RNNs for the transformer era, learning curves, scaling laws for neural language models, chain-of-th
The summary is not provided.