Ex2: Neural Data Augmentation via Example Extrapolation (Research Paper Walkthrough)

May 02, 2021

|

41 views

prakharmishra137

Details

#seq2seq #nlp #dataaugmentation Data augmentation in Machine Learning is used to increase the amount of data by adding slightly modified copies of already existing data or newly created synthetic data from existing data. This paper performs neural Example Extrapolation (Ex2) given a handful of exemplars sampled from some distribution. Please feel free to share out the content and subscribe to my channel :) ⏩ Subscribe - https://youtube.com/channel/UCoz8NrwgL7U9535VNc0mRPA?sub_confirmation=1 ⏩ Abstract: In many applications of machine learning, certain categories of examples may be underrepresented in the training data, causing systems to underperform on such "few-shot" cases at test time. A common remedy is to perform data augmentation, such as by duplicating underrepresented examples, or heuristically synthesizing new examples. But these remedies often fail to cover the full diversity and complexity of real examples. We propose a data augmentation approach that performs neural Example Extrapolation (Ex2). Given a handful of exemplars sampled from some distribution, Ex2 synthesizes new examples that also belong to the same distribution. The Ex2 model is learned by simulating the example generation procedure on data-rich slices of the data, and it is applied to underrepresented, few-shot slices. We apply Ex2 to a range of language understanding tasks and significantly improve over state-of-the-art methods on multiple few-shot learning benchmarks, including for relation extraction (FewRel) and intent classification + slot filling (SNIPS). ⏩ OUTLINE: 0:00 - Abstract 01:01 - Illustration of the Approach 02:10 - Overview of T5 model + Illustration Continued 04:45 - Approach - Overview 05:41 - Formal Definitions - Slicing Data 06:59 - Formal Definitions - Few-shot versus many-shot 09:26 - Example Extrapolation (Ex2) - Task Definition 11:07 - Emperical experiment for value of K 12:26 - Training Procedure 15:50 - Using Ex2 for data augmentation 16:25 - My thoughts ⏩ Paper Title: Neural Data Augmentation via Example Extrapolation ⏩ Paper: https://arxiv.org/abs/2102.01335 ⏩ Author: Kenton Lee, Kelvin Guu, Luheng He, Tim Dozat, Hyung Won Chung ⏩ Organisation: Google Research ⏩ IMPORTANT LINKS Blog on the same: https://ravidesetty.medium.com/neural-data-augmentation-via-example-extrapolation-4642c0646e63 Full Playlist on BERT usecases in NLP: https://www.youtube.com/watch?v=kC5kP1dPAzc&list=PLsAqq9lZFOtV8jYq3JlkqPQUN5QxcWq0f Full Playlist on Text Data Augmentation Techniques: https://www.youtube.com/watch?v=9O9scQb4sNo&list=PLsAqq9lZFOtUg63g_95OuV-R2GhV1UiIZ Full Playlist on Text Summarization: https://www.youtube.com/watch?v=kC5kP1dPAzc&list=PLsAqq9lZFOtV8jYq3JlkqPQUN5QxcWq0f Full Playlist on Machine Learning with Graphs: https://www.youtube.com/watch?v=-uJL_ANy1jc&list=PLsAqq9lZFOtU7tT6mDXX_fhv1R1-jGiYf Full Playlist on Evaluating NLG Systems: https://www.youtube.com/watch?v=-CIlz-5um7U&list=PLsAqq9lZFOtXlzg5RNyV00ueE89PwnCbu ********************************************* ⏩ Youtube - https://www.youtube.com/c/TechVizTheDataScienceGuy ⏩ Blog - https://prakhartechviz.blogspot.com ⏩ LinkedIn - https://linkedin.com/in/prakhar21 ⏩ Medium - https://medium.com/@prakhar.mishra ⏩ GitHub - https://github.com/prakhar21 ⏩ Twitter - https://twitter.com/rattller ********************************************* Please feel free to share out the content and subscribe to my channel :) ⏩ Subscribe - https://youtube.com/channel/UCoz8NrwgL7U9535VNc0mRPA?sub_confirmation=1 Tools I use for making videos :) ⏩ iPad - https://tinyurl.com/y39p6pwc ⏩ Apple Pencil - https://tinyurl.com/y5rk8txn ⏩ GoodNotes - https://tinyurl.com/y627cfsa #techviz #datascienceguy #ai #researchpaper #naturallanguageprocessing #t5

0:00 Abstract 01:01 Illustration of the Approach 02:10 Overview of T5 model + Illustration Continued 04:45 Approach - Overview 05:41 Formal Definitions - Slicing Data 06:59 Formal Definitions - Few-shot versus many-shot 09:26 Example Extrapolation (Ex2) - Task Definition 11:07 Emperical experiment for value of K 12:26 Training Procedure 15:50 Using Ex2 for data augmentation 16:25 My thoughts

Category: Research Paper

Comments

loading...

Reactions (0) | Note

📝 No reactions yet

Be the first one to share your thoughts!

Reactions(0)

Note

loading...

Recommended

ICAPS 2014: Mike Phillips on "PA*SE: Parallel A* for Slow Expansions"

ICAPS | Jul 3, 2014

ICAPS 2014: Vidal Alcázar on "Analyzing the Impact of Partial States..."

ICAPS | Jul 3, 2014

ICAPS 2014: Aijun Bai on "Thompson Sampling Based Monte-Carlo Planning in POMDPs"

ICAPS | Jul 3, 2014

ICAPS 2014: Satish Kumar on "A Tree-Based Algorithm for Construction Robots"

ICAPS | Jul 16, 2014

ICAPS 2014: Peter Ondruska on "The Route Not Taken: Driver-Centric Estimation..."

ICAPS | Jul 21, 2014