Abstract: We introduce a scalable Bayesian preference learning method for identifying convincing ar- guments in the absence of gold-standard rat- ings or rankings. In contrast to previous work, we avoid the need for separate methods to perform quality control on training data, pre- dict rankings and perform pairwise classiﬁca- tion. Bayesian approaches are an effective so- lution when faced with sparse or noisy train- ing data, but have not previously been used to identify convincing arguments. One issue is scalability, which we address by develop- ing a stochastic variational inference method for Gaussian process (GP) preference learn- ing. We show how our method can be ap- plied to predict ar-gument convincingness from crowdsourced data, outperforming the previ- ous state-of-the-art, particularly when trained with small amounts of unreliable data. We demonstrate how the Bayesian approach en- ables more effective active learning, thereby reducing the amount of data required to iden- tify convincing arguments for new users and domains. While word embeddings are princi- pally used with neural networks, our results show that word embed-dings in combination with linguistic features also beneﬁt GPs when predicting argument convincingness.
Authors: Edwin Simpson, Iryna Gurevych (Technische Universitat Darmstadt)