Efficient Bayesian Model Selection in PARAFAC via Stochastic Thermodynamic Integration

Abstract : Parallel factor analysis (PARAFAC) is one of the most popular tensor factorization models. Even though it has proven successful in diverse application fields, the performance of PARAFAC usually hinges up on the rank of the factorization, which is typically specified manually by the practitioner. In this study, we develop a novel parallel and distributed Bayesian model selection technique for rank estimation in large-scale PARAFAC models. The proposed approach integrates ideas from the emerging field of stochastic gradient Markov Chain Monte Carlo, statistical physics, and distributed stochastic optimization. As opposed to the existing methods, which are based on some heuristics, our method has a clear mathematical interpretation, and has significantly lower computational requirements, thanks to data subsampling and parallelization. We provide formal theoretical analysis on the bias induced by the proposed approach. Our experiments on synthetic and large-scale real datasets show that our method is able to find the optimal model order while being significantly faster than the state-of-the-art.
Document type :
Journal articles
Complete list of metadatas

Cited literature [3 references]  Display  Hide  Download

https://hal.archives-ouvertes.fr/hal-01779074
Contributor : Thanh Huy Nguyen <>
Submitted on : Thursday, April 26, 2018 - 12:13:16 PM
Last modification on : Monday, March 25, 2019 - 4:16:02 PM
Long-term archiving on : Thursday, September 20, 2018 - 5:22:48 AM

Identifiers

  • HAL Id : hal-01779074, version 1

Citation

Thanh Huy Nguyen, Umut Simsekli, Gael Richard, Ali Cemgil. Efficient Bayesian Model Selection in PARAFAC via Stochastic Thermodynamic Integration. IEEE Signal Processing Letters, 2018. ⟨hal-01779074⟩

Share

Metrics

Record views

111

Files downloads

85