Exponential Machines

Alexander Novikov, Mikhail Trofimov, Ivan Oseledets
5/12/2016
stat.MLcs.LG

Official code: Bihaqo/exp-machines (43 stars, MIT, last commit 2025-03-04).

Abstract

Modeling interactions between features improves the performance of machine learning solutions in many domains (e.g. recommender systems or sentiment analysis). In this paper, we introduce Exponential Machines (ExM), a predictor that models all interactions of every order. The key idea is to represent an exponentially large tensor of parameters in a factorized format called Tensor Train (TT). The Tensor Train format regularizes the model and lets you control the number of underlying parameters. To train the model, we develop a stochastic Riemannian optimization procedure, which allows us to fit tensors with 2^160 entries. We show that the model achieves state-of-the-art performance on synthetic data with high-order interactions and that it works on par with high-order factorization machines on a recommender system dataset MovieLens 100K.

View on arXivView PDF

Code Implementations(1)

Bihaqo/exp-machinesOfficial100%
MIT

Exponential Machines implementation

4314Python, DockerfileApr 26, 20161 years agoMIT

Similar Papers

tensorFM: Low-Rank Approximations of Cross-Order Feature Interactions

83% match2026

AutoFIS: Automatic Feature Interaction Selection in Factorization Models for Click-Through Rate Prediction

81% match1 repo1832020

Beyond Interaction Capacity: Estimator Scaling with Recursive Models for CTR Prediction

81% match2026

xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systems

81% match1 repo7522018

DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systems

81% match2 repos8.0k2020

Tensor Convolutional Network for Higher-Order Interaction Prediction in Sparse Tensors

80% match2025

Cite this paper

@article{novikov2016exponential,
  title  = {Exponential Machines},
  author = {Alexander Novikov and Mikhail Trofimov and Ivan Oseledets},
  year   = {2016},
  eprint = {1605.03795},
  archivePrefix = {arXiv},
  url    = {https://arxiv.org/abs/1605.03795}
}

Add a badge to your README

Shows whether official code is linked for this paper.

Code of Paper badge
[![Code of Paper](https://codeofpaper.com/badge/1605.03795.svg)](https://codeofpaper.com/paper/1605.03795)

Discussion