Master's Thesis

Short version

In this project, you will develop and evaluate novel ways to build foundation models for the earth system that are capable of processing multi-resolution and multi-temporal data from diverse sensors during pretraining and finetuning in a computationally efficient way. The project is executed as onsite project and provides access to IBM HPC infrastructure.

Description

Significant progress in the development of highly adaptable and reusable artificial intelligence (AI) models is expected to have a major impact on earth science. Foundation models are pre-trained on large unlabeled datasets via self-supervision and subsequently fine-tuned with small, labeled datasets for various downstream tasks. The abundance of unlabeled data from multiple satellites and ground stations makes earth system modeling an ideal domain for pre-training large-scale foundation models. However, recent foundation models for the earth system exploit only a fraction of the available data types, typically from a single satellite, reanalysis data product and at constant resolutions at single time steps. Consequently, the scientific community is increasingly interested in developing approaches to build interoperable (i.e., multi-modal) models for the Earth system that efficiently exploit multi-sensor, multi-resolution, and multi-temporal data.

Goal

This project aims to explore transformer models to result in interoperable, multi-resolution foundation models that encode information from various data sources. This is an essential step in further enhancing AI models for the Earth system to unlock diverse environmental applications.

Requirements

• Bachelor's degree
• Python, PyTorch, DDP, GitHub/Lab
• Problem-solving mindset: ability to explore technical literature, cross-referencing relevant information, synthesis of novel ML architectures
• Ways to stand out: hands-on experience with HPC systems and job scheduling, prior encounters with large-scale transformer architectures, especially with positional encodings, etc.
  

Diversity

IBM is committed to diversity at the workplace. With us you will find an open, multicultural environment. Excellent flexible working arrangements enable all genders to strike the desired balance between their professional development and their personal lives.

How to apply

If you are interested to apply for this position, please submit your application via the following link.

References
[Prithvi EO] Foundation Models for Generalist Geospatial Artificial Intelligence .
[Prithvi WxC] Prithvi WxC: Foundation Model for Weather and Climate .
[4M] 4M-21: An Any-to-Any Vision Model for Tens of Tasks and Modalities .
[DiffusionSat] DiffusionSat: A Generative Foundation Model for Satellite Imagery .