Abstract: One of possible ways of obtaining continuous-space sentence representations is by training neural machine transla-tion (NMT) systems. The recent attention mechanism however removes the single point in the neural network from which the source sentence representation can be extracted. We propose several variations of the attentive NMT ar-chitecture bringing this meeting point back. Empirical evaluation suggests that the better the translation quality, the worse the learned sentence representations serve in a wide range of classification and similarity tasks.
Authors: Ondřej Cífka, Ondřej Bojar (Charles University)