Tutorial's Motivation

​

Machine learning (ML) has proliferated the progress in learning informative representations for high-dimensional data. While effective for learning the statistical associations (e.g., correlation) in data, ML cannot learn the intricate causal interactions across data coordinates. Causal inference transcends learning statistical associations and enables cause-effect reasoning. Such reasoning is often facilitated via external interventions (experiments).  ​In several fast-growing fields (e.g., robotics) the causal variables are often inaccessible and lie in an unobservable latent space. The emerging field of causal representation learning (CRL) aims to learn the latent causal structures by integrating representation. CRL, subsequently, facilitates causal reasoning, intervention, and planning. This tutorial will provide a thorough overview of the recent advances in CRL.

​

Tutorial's Outline

​

In this tutorial, we will explore the latest advancements in the emerging field of causal representation learning (CRL). We will begin by introducing the foundational concepts and motivations behind combining representation learning with causal inference. Following a brief history of CRL, we will describe its primary objectives and the theoretical challenges. We will then review the key approaches developed to address these challenges. Our primary focus will be on interventional CRL, and we will discuss other advances in CRL with multi-view observations, CRL with interventions on latent variables, and CRL applied to temporal data. Additionally, we will highlight real-world application opportunities, discuss the challenges in scaling CRL to practical use cases, and discuss open questions for CRL related to both theoretical and empirical viewpoints.

​

Content

​

The half-day tutorial will be structured as follows, with presenters denoted by their initials.

​

Part 1:  Introduction and Background (45 minutes)

• Representation learning & its success!

• Towards strong generalization

• Formalizing CRL

• What does causality signify?

• Background on causal inference and discovery

• CRL's theoretical objectives

• Why is CRL challenging?

• Identifiability and algorithm design

• Taxonomy of CRL approaches

• Interventional CRL​ ​

​

Part 2: Interventional CRL (75 minutes)

• Why interventional CRL?

• Deterministic Interventions

• Stochastic interventions and score functions

• Score-based CRL

  • General transformations​

  • Linear transformations

  • Multi-node interventions​

​

Part 3: Empirical Aspects and Recent Developments (75 minutes)

• Score estimation

  • ​parametric models

  • non-parametric models

  • score-matching models.

  • Environment extrapolation

• Empirical results

• Finite-sample analysis

• Multiview CRL

• Temporal CRL

• Contrastive learning

• Multi-domain CRL

• Counterfactual CRL

• Mechanism sparsity​​

​

Part 4: Discussions and Closing Thoughts (15 minutees)