Unconstrained Least-Squares Importance Fitting (uLSIF)


Unconstrained Least-Squares Importance Fitting (uLSIF) is an algorithm to directly estimate the ratio of two density functions without going through density estimation. The solution of uLSIF as well as the leave-one-out score can be computed analytically, thus uLSIF is computationally very efficient and stable. Furthermore, the uLSIF solution tends to be sparse, which contributes to reducing the computation time.

uLSIF is based on the squared loss, while a variant based on the Kullback-Leibler loss called KLIEP is available from here.


MATLAB implementation of uLSIF: uLSIF.tgz R implementation of LSIF/uLSIF: Link to Takafumi Kanamori's web page



Kanamori, T., Hido, S., & Sugiyama, M.
A least-squares approach to direct importance estimation.
Journal of Machine Learning Research, vol.10 (Jul.), pp.1391-1445, 2009.
[ paper ]

Kanamori, T., Hido, S., & Sugiyama, M.
Efficient direct density ratio estimation for non-stationarity adaptation and outlier detection,
In D. Koller, D. Schuurmans, Y. Bengio, L. Botton (Eds.), Advances in Neural Information Processing Systems 21, pp.809-816, Cambridge, MA, MIT Press, 2009.
(Presented at Neural Information Processing Systems (NIPS2008), Vancouver, B.C., Canada, Dec. 8-13, 2008.)
[ paper, poster ]

Masashi Sugiyama (sugi [at] cs.titech.ac.jp)

Sugiyama Laboratory, Department of Computer Science, Graduate School of Information Science and Engineering, Tokyo Institute of Technology,
2-12-1-W8-74, O-okayama, Meguro-ku, Tokyo, 152-8552, Japan.
TEL & FAX: +81-3-5734-2699