LiNGAM - Discovery of non-gaussian linear causal models

Brief description

LiNGAM is a new method for estimating structural equation models or linear causal Bayesian networks. It is based on using the non-Gaussianity of the data.

Several authors (Spirtes et al. 1993; Pearl 2000) have recently formalized concepts related to causality using probability distributions defined on directed acyclic graphs. This line of research emphasizes the importance of understanding the process which generated the data, rather than only characterizing the joint distribution of the observed variables. The reasoning is that a causal understanding of the data is essential to be able to predict the consequences of interventions, such as setting a given variable to some specified value.

One of the main questions one can answer using this kind of theoretical framework is: 'Under what circumstances and in what way can one determine causal structure based on data which is not obtained by controlled experiments but by passive observation only?'. In many cases it is impossible or too expensive to perform controlled experiments, and hence methods for discovering likely causal relations from uncontrolled ('observational') data would be very valuable.

We have developed a method, abbreviated LiNGAM, for identifying Linear, Non-Gaussian, Acyclic causal Models based on purely observational, continuous-valued data. This method can be seen as an extension of the standard SEM (Structural Equation Model; see, for instance, Bollen 1989) framework. The key aspect of our method is that when the data is non-gaussian it is possible to identify more of the generating structure than is possible in the traditional Gaussian setting. The basic method assumes that (a) there are no hidden confounders, and (b) all (or all but one) of the error terms are non-gaussian. Under these conditions it can be shown that the full generating model can be identified in the limit of an infinite sample.

Original article introducing LiNGAM

S. Shimizu, P. O. Hoyer, A. Hyvärinen, and A. J. Kerminen.
A linear non-gaussian acyclic model for causal discovery.
Journal of Machine Learning Research, 7:2003-2030, 2006.

Estimation methods and code

The model introduced in the paper above can be estimated in different ways:

ICA-based LiNGAM 

The estimation method in the original 2006 JMLR paper above is distributed as a complete Matlab/Octave code package. (We also distribute R code as part of the bayeslingam package and as part of the VAR-LiNGAM package.) Also implemented in TETRAD, R package: pcalg by Kalisch et al. Python code. 

DirectLiNGAM

S. Shimizu, T. Inazumi, Y. Sogawa, A. Hyvärinen, Y. Kawahara, T. Washio, P. O. Hoyer and K. Bollen.
DirectLiNGAM: A direct method for learning a linear non-Gaussian structural equation model.
Journal of Machine Learning Research, 12(Apr): 1225--1248, 2011.
Python code
R code by Genta Kikuchi

Pairwise LiNGAM

A. Hyvärinen and S. M. Smith.
Pairwise likelihood ratios for estimation of non-Gaussian structural eauation models.
Journal of Machine Learning Research 14:111-152, 2013.
Matlab code
Python code

Various extensions

The basic model in the 2006 JMLR paper has been later extended in different ways. For an extensive and updated list, see this page. Here are some of them:

Hidden common causes

P. O. Hoyer, S. Shimizu, A. J. Kerminen, and M. Palviainen.
Estimation of causal effects using linear non-gaussian causal models with hidden variables.
International Journal of Approximate Reasoning, 49: 362-378, 2008.
Small Matlab code package.

Time series analysis

A. Hyvärinen, K. Zhang, S. Shimizu, and P. O. Hoyer.
Estimation of a structural vector autoregression model using non-Gaussianity.
Journal of Machine Learning Research, 11:1709-1731, 2010.
R code
Python code

Mixtures of Gaussian and non-Gaussian variables

P. O. Hoyer, A. Hyvärinen, R. Scheines, P. Spirtes, J. Ramsey, G. Lacerda, and S. Shimizu.
Causal discovery of linear acyclic models with arbitrary distributions.
Proc. 24th Conf. on Uncertainty in Artificial Intelligence (UAI2008), pp. 282-289, Helsinki, Finland, 2008.
TETRAD

Cyclic model

G. Lacerda, P. Spirtes, J. Ramsey and P. O. Hoyer.
Discovering cyclic causal models by independent components analysis.
Proc. 24th Conf. on Uncertainty in Artificial Intelligence (UAI2008), pp. 366-374, Helsinki, Finland, 2008.
TETRAD

Nonlinear models

K. Zhang and A. Hyvärinen.
On the identifiability of the post-nonlinear causal model.
Proc. 25th Conf. on Uncertainty in Artificial Intelligence (UAI2009), pp. 647-655, Montreal, Canada, 2009.
code

P. O. Hoyer, D. Janzing, J. Mooij, J. Peters, and B. Schölkopf.
Nonlinear causal discovery with additive noise models.
Advances in Neural Information Processing Systems 21 (NIPS2008), pp. 689-696, 2009.
code
Python code

Survey articles

S. Shimizu.
LiNGAM: Non-Gaussian methods for estimating causal structures.
Behaviormetrika, 41(1): 65--98, 2014 (Special Issue on Causal Discovery).

K. Zhang and A. Hyvärinen.
Nonlinear functional causal models for distinguishing cause from effect.
Statistics and Causality: Methods for Applied Empirical Research, pp. 185-202, 2016. 

S. Shimizu and P. Blöbaum. Recent advances in semi-parametric methods for causal discovery. In Direction Dependence in Statistical Models: Methods of Analysis (W. Wiedermann, D. Kim, E. Sungur, and A. von Eye, eds.), pages xx–xx. Wiley, 2020. 

Updated by Shohei Shimizu in Jun 2022.

Created at the Neuroinformatics group in University of Helsinki around 2006.