Least-Squares Dimensionality Reduction (LSDR)

Description

Least-Squares Dimensionality Reduction (LSDR) is a supervised dimensionality reduction method. LSDR adopts a squared-loss variant of mutual information as an independence measure and estimates it using the density-ratio estimation method uLSIF. Thanks to this formulation, all tuning parameters such as the Gaussian width and the regularization parameter can be automatically chosen based on a cross-validation method. Then LSDR maximizes this independence measure (making the complementary features conditional independent of outputs) by a natural gradient algorithm.

Download

MATLAB implementation of LSDR: LSDR.tgz ("demo_LSDR.m" is the first file to execute).

Examples

LSDR-regression LSDR-classification

References

Suzuki, T. & Sugiyama, M.
Sufficient dimension reduction via squared-loss mutual information estimation.
Neural Computation, vol.25, no.3, pp.725-758, 2013.
[ paper ]

Masashi Sugiyama (sugi [at] cs.titech.ac.jp)

Sugiyama Laboratory, Department of Computer Science, Graduate School of Information Science and Engineering, Tokyo Institute of Technology,
2-12-1-W8-74, O-okayama, Meguro-ku, Tokyo, 152-8552, Japan.
TEL & FAX: +81-3-5734-2699