Multiscale, multimodal and multitemporal remote sensing


Remote sensing of the environment:
Multiscale, multimodal and multitemporal remote sensing

Climate change, preservation of biodiversity, detection and monitoring of pollution, mitigation of natural disasters (assessment of vulnerability, detection of early warnings, crisis situation management...) are key issues with a very high societal impact. In order to address these challenges, new resources are increasingly available. Satellite remote sensing is currently undergoing a technical revolution with the appearance and blooming development of very high-resolution sensors. It allows the observation of the planet with a wide coverage and a very high spatial resolution, at a reduced cost and with a very short revisit time. The multiple modalities that are available (optical data, radar...) allow to access various physical characteristics on the ground. In parallel to the growing availability of satellite based remote sensing data, airborne based platforms and light unmanned aerial vehicles (drones) are a tremendous opportunity to provide additional complementary information, with a great flexibility. As a result, unprecedented quantities of digital data are available, with heterogeneous scales and modalities. In order to exploit the previously described Big Data to full extent, achieving the potential offered by the new generations of remote sensing sensors, and to actually meet crucial needs, some induced pivotal scientific and technical challenges must be tackled. This is the core goal of this Chair : advanced information processing methods will be developped to handle heterogeneous and massive sources of information. This includes cutting edge deep learning techniques, manifold and graph-based representations and data fusion.
At the crossroads between remote sensing, geosciences, mathematics, machine learning, signal and image processing, this project aims at bridging the gap between increasingly pressing needs and the tremendous potential offered by remote sensing technologies, in the broadest sense.


Modeling and monitoring of the plant biodiversity in the Alpine ecosystem,

  • Conception of a cubesat with embedded AI to monitor the environment , with the support of Teledyne e2V (kick-off in march 2020 and Mission Definition Review passed in nov 2020) (PI: Mathieu Barthélémy, CSUG)
  • Organization of an international scientific challenge in partnership with SolarAid, HPE, European and Italian Space Agencies, and IEEE Geoscience and Remote Sensing Society (challenge to be held in 2021)
  • Methodological developments in AI and remote sensing data in a number of contexts

Chair events

Organization of a special session on “Incorporating Physics into Deep Learning” at the 2020 IEEE IGARSS (scheduled to be held in Hawai in July 2020 and eventually held on-line in sept 2020).

Roundtable « les technologies au service de la planète », Festival Transfo (Grenoble, sept 2020)

Scientific publications

Cross-Attention in Coupled Unmixing Nets for Unsupervised Hyperspectral Super-Resolution
ECCV 2020

Joint & Progressive Subspace Analysis (JPSA) with Spatial-Spectral Manifold Alignment for Hyperspectral Dimensionality Reduction
IEEE Transactions on Cybernetics

Hyperspectral Images Super-Resolution via Learning High-Order Coupled Tensor Ring Representation
IEEE Transactions on Neural Networks and Learning Systems

Multimodal GANs: Towards Crossmodal Hyperspectral-Multispectral Image Segmentation
IEEE Transactions on Geoscience and Remote Sensing

Graph-Induced Aligned Learning on Subspaces for Hyperspectral and Multispectral Data
IEEE Transactions on Geoscience and Remote Sensing

Deep Encoder-Decoder Networks for Classification of Hyperspectral and LiDAR Data
IEEE Geoscience and Remote Sensing Letters

More Diverse Means Better: Multimodal Deep Learning Meets Remote Sensing Imagery Classification
IEEE Transactions on Geoscience and Remote Sensing

X-ModalNet: A Semi-Supervised Deep Cross-Modal Network for classification of Remote Sensing Imagery
ISPRS Journal of Photogrammetry and Remote Sensing

The treasure vault can be opened: large scale genome skimming works equally well for herbarium as silica gel dried material.

Assessing the reliability of predicted plant trait distributions at the global scale.
Global Ecology & Biogeography

Model complexity affects species distribution projections under climate change.
Journal of Biogeography

Macroecology in the age of big data – where to go from here?
Journal of Biogeography

A standard protocol for reporting species distribution models.

Research topics

Extraction of latent variables through common subspace learning, multimodal manifold learning and alignment, graphs in very high dimensions.

  • This should result in splitting the information into a common (shared) part and a modality-(and/or scale, and/or date) specific part.
  • These components (shared and specific, respectively) should be merged components for a complete representation and understanding.
  • This will allow to perform transfer knowledge (from one date to another, one modality to another…) or to reconstruct missing data from a set of incomplete acquisitions.
  • A special attention will be given to the processing and analysis of hyperspectral data.
  • Multimodal autoencoders, (Very large) graph matching, (Local) low dimensional manifold models…