TerraTorch 1.0: Making geospatial AI simpler. TT 1.0, an AI Alliance affiliate project, facilitates the creation and comparison of geographic models.
TerraTorch (TT) 1.0, a significant advancement in geographic artificial intelligence. TerraTorch was created to satisfy the requirements of researchers studying geospatial artificial intelligence who want to refine foundation models and use them with their data. By integrating and standardizing foundation models as reusable backbones, it streamlines this process and makes it possible to seamlessly combine them with cutting-edge decoders and heads.
They are part of the optimization goal since different tasks (such as regression, classification, segmentation, object detection, etc.) call for different heads, and decoders perform differently on various datasets, backbones, and head combinations. TerraTorch, which was created especially for geospatial, meteorological, climate, and Earth observation applications, has quickly developed into a potent framework for foundation model inference and fine-tuning.
The notion of “necks” intermediate layers that adjust between incompatible model components was recently proposed by TerraTorch, which increases architectural design flexibility. The framework not only enhances repeatability and integration but also lowers user error by reducing the requirement for additional glue code because all neural network components are modularized and available through a TerraTorch execution configuration YAML file as well as the Python API.
Lastly, this configuration-based method seamlessly integrates with the TT Iterate plugin to provide neural network architecture search (NAS) and hyperparameter optimization (HPO). This tool helps users fine-tune hyperparameters for better model performance and optimize the choice of backbones, decoders, and heads. TerraTorch is also essential for benchmarking foundation models in accordance with the GeoBench standard, which guarantees thorough assessment and comparison of models in geospatial AI.
Geospatial foundation models can be easily adapted to EO applications with TerraTorch. The integrated metrics facilitate the comparison of models and speed up model validation. The broader community is responding favorably to the fine-tuning of Prithvi-EO-2.0 by the researchers using TerraTorch for a number of applications, such as tracking the intensity of floods, wildfires, and burns, landslip identification, and crop segmentation.
To assist customers get started quickly while retaining flexibility for fine-tuning, TT 1.0 offers fingertip access to a variety of foundation models with opinionated performance optimizations (default settings based on best practices) that are still changeable. This facilitates the exploration and use of cutting-edge geospatial AI models by enabling quick experimentation and deployment.
TerraTorch joins the AI Alliance
TerraTorch’s approval as an AI Alliance Affiliate Project is a major highlight of this edition. This partnership ensures alignment with best practices in open AI development and promotes innovation in geospatial machine learning by integrating TerraTorch into a wider community of AI researchers and practitioners.
Iterate: HPO and NAS for geospatial AI
The Iterate plugin for NAS and HPO is a crucial component of TT 1.0. By automating the process of finding the optimal architectures and hyperparameters running on top of Optuna and optionally utilising MLFlow and Ray Tune, Iterate streamlines the process of choosing and optimizing models. Furthermore, Iterate facilitates benchmarking foundation models in accordance with the GeoBench standard, guaranteeing a thorough assessment of geospatial AI models in comparison to actual datasets.
A comprehensive collection of foundation models and data modules
With its extensive library of foundation models and reusable data modules, TT 1.0 makes using geospatial AI models simpler than before. TT 1.0 can be used by users for pre-training, fine-tuning, and inference. It can be set up using a customizable YAML configuration file or run with code that is compatible with PyTorch Lightning.
Deep integration with TorchGeo, CLAY, and IBM open-source models
TT 1.0 easily interfaces with TorchGeo to increase its functionality, giving it access to all models and data modules that the TorchGeo ecosystem supports. Furthermore, TerraTorch‘s position as a top geospatial AI framework is strengthened by the native support for numerous open-source geospatial foundation models, such as the Granite family, the Prithvi family, Clay, SatMAE, Satlas, DeCur, and DOFA.
- TT 1.0 facilitates image classification, multivariate multiple regression, and semantic segmentation with adjustable heads.
- Additionally, it allows parameter efficient fine-tuning (PEFT) with ViT-Adapter and LoRA. TT 1.1 will have VPT accessible.
- Furthermore, any Timm or SMP model can be used in TT 1.0, providing the user complete control over the model type they choose for a given task.
TerraTorch: The engine behind IBM’s Geospatial Studio for scalable AI inference and fine-tuning
IBM’s Geospatial Studio relies on TerraTorch, which powers scalable AI inference and fine-tuning for geospatial applications. It serves as the platform’s foundation, facilitating smooth data preprocessing, model training, and real-time satellite imagery analysis in a single setting.
Because Geospatial Studio is open-source, TerraTorch leverages community-driven innovations to reduce vendor lock-in, increase transparency, and spur innovation. By doing this, Geospatial Studio is not restricted to IBM’s stack and can immediately take use of all the latest advancements in foundation models and AI best practices.
Terratorch models and vLLM
An optimized open-source library for LLM inference, vLLM offers dynamic batching, sophisticated memory management, and Paged Attention for latency-free, effective streaming.
By using TerraTorch to facilitate the production and consumption of geographical data, one can are extending vLLM beyond text to multi-modal AI. Onboarding IBM Research’s Prithvi-EO-2.0, the first non-text input/non-text output model in vLLM, has been a significant milestone. By introducing non-text post-processing steps in vLLM, which in the case of geospatial models can employ TerraTorch for computer vision tasks, current work aims to expedite this move. The groundwork for vLLM being genuinely multimodal is being laid by this.
What comes next?
TT 1.0’s release is only the first step. To guarantee TerraTorch stays at the forefront of geospatial AI innovation, and are dedicated to making constant enhancements, increasing support for new models, datasets, jobs (such as object detection), and benchmarking standards.
It extend an invitation to the geospatial and AI communities to investigate TT 1.0, help shape its future, and contribute to its development. Give it a try now to join this thrilling adventure!
