Ex2: Neural Data Augmentation via Example Extrapolation (Research Paper Walkthrough)
May 02, 2021
|
42 views
Details
#seq2seq #nlp #dataaugmentation
Data augmentation in Machine Learning is used to increase the amount of data by adding slightly modified copies of already existing data or newly created synthetic data from existing data. This paper performs neural Example Extrapolation (Ex2) given a handful of exemplars sampled from some distribution.
Please feel free to share out the content and subscribe to my channel :)
⏩ Subscribe - https://youtube.com/channel/UCoz8NrwgL7U9535VNc0mRPA?sub_confirmation=1
⏩ Abstract: In many applications of machine learning, certain categories of examples may be underrepresented in the training data, causing systems to underperform on such "few-shot" cases at test time. A common remedy is to perform data augmentation, such as by duplicating underrepresented examples, or heuristically synthesizing new examples. But these remedies often fail to cover the full diversity and complexity of real examples. We propose a data augmentation approach that performs neural Example Extrapolation (Ex2). Given a handful of exemplars sampled from some distribution, Ex2 synthesizes new examples that also belong to the same distribution. The Ex2 model is learned by simulating the example generation procedure on data-rich slices of the data, and it is applied to underrepresented, few-shot slices. We apply Ex2 to a range of language understanding tasks and significantly improve over state-of-the-art methods on multiple few-shot learning benchmarks, including for relation extraction (FewRel) and intent classification + slot filling (SNIPS).
⏩ OUTLINE:
0:00 - Abstract
01:01 - Illustration of the Approach
02:10 - Overview of T5 model + Illustration Continued
04:45 - Approach - Overview
05:41 - Formal Definitions - Slicing Data
06:59 - Formal Definitions - Few-shot versus many-shot
09:26 - Example Extrapolation (Ex2) - Task Definition
11:07 - Emperical experiment for value of K
12:26 - Training Procedure
15:50 - Using Ex2 for data augmentation
16:25 - My thoughts
⏩ Paper Title: Neural Data Augmentation via Example Extrapolation
⏩ Paper: https://arxiv.org/abs/2102.01335
⏩ Author: Kenton Lee, Kelvin Guu, Luheng He, Tim Dozat, Hyung Won Chung
⏩ Organisation: Google Research
⏩ IMPORTANT LINKS
Blog on the same: https://ravidesetty.medium.com/neural-data-augmentation-via-example-extrapolation-4642c0646e63
Full Playlist on BERT usecases in NLP: https://www.youtube.com/watch?v=kC5kP1dPAzc&list=PLsAqq9lZFOtV8jYq3JlkqPQUN5QxcWq0f
Full Playlist on Text Data Augmentation Techniques: https://www.youtube.com/watch?v=9O9scQb4sNo&list=PLsAqq9lZFOtUg63g_95OuV-R2GhV1UiIZ
Full Playlist on Text Summarization: https://www.youtube.com/watch?v=kC5kP1dPAzc&list=PLsAqq9lZFOtV8jYq3JlkqPQUN5QxcWq0f
Full Playlist on Machine Learning with Graphs: https://www.youtube.com/watch?v=-uJL_ANy1jc&list=PLsAqq9lZFOtU7tT6mDXX_fhv1R1-jGiYf
Full Playlist on Evaluating NLG Systems: https://www.youtube.com/watch?v=-CIlz-5um7U&list=PLsAqq9lZFOtXlzg5RNyV00ueE89PwnCbu
*********************************************
⏩ Youtube - https://www.youtube.com/c/TechVizTheDataScienceGuy
⏩ Blog - https://prakhartechviz.blogspot.com
⏩ LinkedIn - https://linkedin.com/in/prakhar21
⏩ Medium - https://medium.com/@prakhar.mishra
⏩ GitHub - https://github.com/prakhar21
⏩ Twitter - https://twitter.com/rattller
*********************************************
Please feel free to share out the content and subscribe to my channel :)
⏩ Subscribe - https://youtube.com/channel/UCoz8NrwgL7U9535VNc0mRPA?sub_confirmation=1
Tools I use for making videos :)
⏩ iPad - https://tinyurl.com/y39p6pwc
⏩ Apple Pencil - https://tinyurl.com/y5rk8txn
⏩ GoodNotes - https://tinyurl.com/y627cfsa
#techviz #datascienceguy #ai #researchpaper #naturallanguageprocessing #t5
0:00 Abstract 01:01 Illustration of the Approach 02:10 Overview of T5 model + Illustration Continued 04:45 Approach - Overview 05:41 Formal Definitions - Slicing Data 06:59 Formal Definitions - Few-shot versus many-shot 09:26 Example Extrapolation (Ex2) - Task Definition 11:07 Emperical experiment for value of K 12:26 Training Procedure 15:50 Using Ex2 for data augmentation 16:25 My thoughts
0:00 Abstract 01:01 Illustration of the Approach 02:10 Overview of T5 model + Illustration Continued 04:45 Approach - Overview 05:41 Formal Definitions - Slicing Data 06:59 Formal Definitions - Few-shot versus many-shot 09:26 Example Extrapolation (Ex2) - Task Definition 11:07 Emperical experiment for value of K 12:26 Training Procedure 15:50 Using Ex2 for data augmentation 16:25 My thoughts
Comments
loading...
Reactions (0) | Note
📝 No reactions yet
Be the first one to share your thoughts!
Reactions(0)
Note
loading...
