Reformulating Unsupervised Style Transfer as Paraphrase Generation | Research Paper Walkthrough
CrossMind.ai logo
Details
#EMNLP2020 #TextStyleTransfer #ParaphraseGeneration #Roberta Text Style Transfer refers to a class of algorithms that manipulate original text, in order to adopt the syntactic and semantic notion of another text. For ex. Training a model that could re-write the text from Shakespeare's writing style like a twitter post, maybe even vice-versa. Style transfer in Natural langauge processing is an emerging and active area of research. Authors reformulate the problem of style transfer as a paraphrase generation problem. Authors attemp to solve it in an unsupervised way. This paper is part of Proceedings of 2020 EMNLP Conference. ⏩ Support by subscribing to the channel - https://youtube.com/channel/UCoz8NrwgL7U9535VNc0mRPA?sub_confirmation=1 ⏩ Abstract: Modern NLP defines the task of style transfer as modifying the style of a given sentence without appreciably changing its semantics, which implies that the outputs of style transfer systems should be paraphrases of their inputs. However, many existing systems purportedly designed for style transfer inherently warp the input's meaning through attribute transfer, which changes semantic properties such as sentiment. In this paper, we reformulate unsupervised style transfer as a paraphrase generation problem, and present a simple methodology based on fine-tuning pretrained language models on automatically generated paraphrase data. Despite its simplicity, our method significantly outperforms state-of-the-art style transfer systems on both human and automatic evaluations. We also survey 23 style transfer papers and discover that existing automatic metrics can be easily gamed and propose fixed variants. Finally, we pivot to a more real-world style transfer setting by collecting a large dataset of 15M sentences in 11 diverse styles, which we use for an in-depth analysis of our system ⏩ OUTLINE: 0:00 - Brief on Unsupervised Learning, Style Transfer, Paraphrase Generation 4:35 - Abstract 6:16 - Walkthrough of Algorithm Diagram 9:10 - Creating pseudo-parallel training data 10:17 - Style transfer via inverse paraphrasing 11:48 - Paraphraser implementation using GPT-2 13:34 - Promoting diversity by filtering data 15:54 - Evaluating the style transfer systems 18:57 - Normalisation distribution effect due to paraphrasing 19:22 - Concluding remarks and Thanks :) ⏩ Paper Title: Reformulating Unsupervised Style Transfer as Paraphrase Generation ⏩ Paper: https://arxiv.org/pdf/2010.05700.pdf ⏩ Author: Kalpesh Krishna, John Wieting, Mohit Iyyer ⏩ Organisation: University of Massachusetts Amherst, Carnegie Mellon University ⏩ Project Page: http://style.cs.umass.edu ⏩ IMPORTANT LINKS: 1. Beyond BLEU: Training Neural Machine Translation with Semantic Similarity - https://www.aclweb.org/anthology/P19-1427.pdf 2. Research paper Walkthrough - https://www.youtube.com/watch?v=ykClwtoLER8&list=PLsAqq9lZFOtWUz1WEoJ3GXw197LD7BxMc ********************************************* ⏩ Youtube - https://youtube.com/channel/UCoz8NrwgL7U9535VNc0mRPA ⏩ Blog - https://prakhartechviz.blogspot.com ⏩ LinkedIn - https://linkedin.com/in/prakhar21 ⏩ Medium - https://medium.com/@prakhar.mishra ⏩ GitHub - https://github.com/prakhar21 ********************************************* Please feel free to share out the content and subscribe to my channel :) ⏩ Subscribe - https://youtube.com/channel/UCoz8NrwgL7U9535VNc0mRPA?sub_confirmation=1 Tools I use for making videos :) ⏩ iPad - https://tinyurl.com/y39p6pwc ⏩ Apple Pencil - https://tinyurl.com/y5rk8txn ⏩ GoodNotes - https://tinyurl.com/y627cfsa ⏩ Microphone - https://tinyurl.com/yxup9ptz About Me: I am Prakhar Mishra and this channel is my passion project. I am currently pursuing my MS (by research) in Data Science. I have an industry work-ex of 3 years in the field of Data Science and Machine Learning with a particular focus in Natural Langauge Processing (NLP). #techviz #datascienceguy #nlp #machinelearning #GPT #naturallanguageprocessing #unsupervised

Comments
loading...