Case Study: A Feed-forward Architecture for Sentence Meaning Inference

Abstract

In Section 11.6 we introduced the sum of pairwise word similarities as a strong baseline for the short document similarity task. Given two sentences, the first one with words $w_1^1,....,w_{\ell_1}^1$ and the second one with words $w_1^2,....,w_{\ell_2}^2$, each word is associated with a corresponding pre-trained word vector $w_{1:\ell}^1,w_{1:\ell_2}^2$, and the similarity between the documents is given by:

$$\sum_{i=1}^{\ell_1}\sum_{j=1}^{\ell_2}\text{sim}\left(w_i^1;w_j^2\right).$$