Neural Collaborative Filtering (2017)
March 25, 2024In collaborative filtering, Matrix Factorization (MF) approximates the user-item interaction matrix by multiplying the latent matrices for users and items. An estimated iteraction is the inner product of the latent vectors for the user and the item. Neural Collaborative Filtering (NCF) replaces the inner product with a neural neural network to capture the complex structure of user interaction data. There are three instances of NFC: Generalized Matrix Factorization (GMF), Multi-Layer Perceptron (MLP), Neural Matrix Factoriation (NeuMF).
GMF inputs the element-wise product of a user’s and an item’s latent vectors into a linear layer.
MF can be seen as a special case of GMF.
Let \(\textbf{v}^U_u\) and \(\textbf{v}^I_{i}\) be the one-hot encoding of a user and an item, and \(\textbf{P}^\top\) and \(\textbf{Q}^\top\) be latent feature matrices.
GMF estimates the interaction between user \(u\) and item \(i\) by projecting the element-wise product of latent vectors:
$$
\hat{y}_{ui}=a_{\text{out}}(\textbf{h}^\top (\textbf{p}_u \odot \textbf{q}_i))
$$
where \(\textbf{p}_u\) is \(\textbf{P}^\top \textbf{v}_u^U\) and \(\textbf{q}_i\) is \(\textbf{Q}^\top \textbf{v}_i^I\).
MLP concatenates the latent vectors of a user and an item, and inputs the vector to a MLP network: $$ \begin{align*} \textbf{z}_1&=\phi_1(\textbf{p}_u, \textbf{q}_i)=\begin{bmatrix}\textbf{p}_u\\\textbf{q}_i\end{bmatrix}\\ \phi_2(\textbf{z}_1)&=a_2(\textbf{W}^\top_2\textbf{z}_1+\textbf{b}_2)\\ &\dots\\ \phi_L(\textbf{z}_{L-1})&=a_L(\textbf{W}^\top_L\textbf{z}_{L-1}+\textbf{b}_L)\\ \hat{y}_{ui}&=\sigma(\textbf{h}^\top \phi_L(\textbf{z}_{L-1})) \end{align*} $$
NeuMF concatenates the outputs of GMP and MLP, then feeds the concatnated vectors to a linear layer:
$$ \begin{align*} \phi^{\text{GMF}}&=\textbf{p}^G_u\odot \textbf{q}^G_i\\ \phi^{\text{MLP}}&=a_L(\textbf{W}^T_L(a_{L-1} (\dots a_2(\textbf{W}^\top_2\begin{bmatrix}\textbf{p}^M_u\\\textbf{q}^M_i\end{bmatrix}+\textbf{b}_2)\dots)) + \textbf{b}_L)\\ \hat{y}_{ui}&=\sigma (\textbf{h}^\top \begin{bmatrix}\phi^{\text{GMF}}\\\phi^{\text{MLP}}\end{bmatrix}) \end{align*} $$