Abstract: We report on a comparative style analysis of hyperpartisan (extremely one-sided) news and fake news. A corpus of 1,627 articles from 9 political publishers, three each from the mainstream, the hyperpartisan left, and the hyperparti-san right, have been fact-checked by professional journalists at BuzzFeed: 97% of the 299 fake news articles identiﬁed are also hyperpartisan. We show how a style analysis can distinguish hyperpartisan news from the mainstream (F1 = 0.78), and satire from both (F1 = 0.81). But stylometry is no silver bullet as style-based fake news detection does not work (F1 = 0.46). We further reveal that left-wing and right-wing news share signiﬁcantly more stylistic similarities than either does with the mainstream. This result is robust: it has been conﬁrmed by three different modeling ap-proaches, one of which employs Unmasking in a novel way. Applications of our results include partisanship detection and pre-screening for semi-automatic fake news detection.
Authors: Martin Potthast, Johannes Kiesel, Kevin Reinartz, Janek Bevendorff, Benno Stein (Bauhaus-Universität Weimar)