Formal Description of Prompting: Systematic Survey of Prompting Methods in NLP (P.1)

Formal Description of Prompting: Systematic Survey of Prompting Methods in NLP (P.1)

Oct 13, 2021
|
33 views
Details
#promptlearning #survey #naturallanguageprocessing This systematic survey organizes research work around the new learning paradigm in natural language processing, which is "prompt-based learning". In this first part, I have covered the first two sections of the paper majorly discussing the evolution of the NLP learning space and the Formal description of prompting. In the upcoming videos, I plan to cover the remaining sections as well. Firstly, apologies for missing out on one affiliation during the introduction. It was (National University of Singapore) ⏩ Abstract: This paper surveys and organizes research works in a new paradigm in natural language processing, which we dub "prompt-based learning". Unlike traditional supervised learning, which trains a model to take in an input x and predict an output y as P(y|x), prompt-based learning is based on language models that model the probability of text directly. To use these models to perform prediction tasks, the original input x is modified using a template into a textual string prompt x' that has some unfilled slots, and then the language model is used to probabilistically fill the unfilled information to obtain a final string x, from which the final output y can be derived. This framework is powerful and attractive for a number of reasons: it allows the language model to be pre-trained on massive amounts of raw text, and by defining a new prompting function the model is able to perform few-shot or even zero-shot learning, adapting to new scenarios with few or no labeled data. In this paper we introduce the basics of this promising paradigm, describe a unified set of mathematical notations that can cover a wide variety of existing work, and organize existing work along several dimensions, e.g.the choice of pre-trained models, prompts, and tuning strategies. To make the field more accessible to interested beginners, we not only make a systematic review of existing works and a highly structured typology of prompt-based concepts, but also release other resources, e.g., a website this http URL including constantly-updated survey, and paperlist. Sign-up for Email Subscription - https://forms.gle/duSwrYAGw6zUhoGf9 ⏩ OUTLINE: 0:00 - Abstract and Introduction 06:40 - 4 paradigms in NLP learning space 09:19 - Prompting basics + Terminology and Notations 12:15 - Prompt addition (in prompt-based learning in NLP) 13:45 - Answer search (in prompt-based learning in NLP) 15:03 - Answer mapping (in prompt-based learning in NLP) 15:37 - Example of input, template, and answer for different tasks in prompt-based learning in NLP 17:21 - Design Consideration for Prompting 19:14 - Content for Next video ⏩ Paper Title: Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing ⏩ Paper: https://arxiv.org/abs/2107.13586v1 ⏩ Author: Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, Graham Neubig ⏩ Organisation: Carnegie Mellon University, National University of Singapore Please feel free to share out the content and subscribe to my channel :) ⏩ Subscribe - https://youtube.com/channel/UCoz8NrwgL7U9535VNc0mRPA?sub_confirmation=1 ********************************************** If you want to support me financially which is totally optional and voluntary ❤️ You can consider buying me chai ( because I don't drink coffee :) ) at https://www.buymeacoffee.com/TechvizCoffee ❤️ Support using Paypal - https://www.paypal.com/paypalme/TechVizDataScience ********************************************** ⏩ Youtube - https://www.youtube.com/c/TechVizTheDataScienceGuy ⏩ LinkedIn - https://linkedin.com/in/prakhar21 ⏩ Medium - https://medium.com/@prakhar.mishra ⏩ GitHub - https://github.com/prakhar21 ⏩ Twitter - https://twitter.com/rattller ********************************************* Tools I use for making videos :) ⏩ iPad - https://tinyurl.com/y39p6pwc ⏩ Apple Pencil - https://tinyurl.com/y5rk8txn ⏩ GoodNotes - https://tinyurl.com/y627cfsa #techviz #datascienceguy #researchpaper #prompting #machinelearning

0:00 - Abstract and Introduction 06:40 - 4 paradigms in NLP learning space 09:19 - Prompting basics + Terminology and Notations 12:15 - Prompt addition (in prompt-based learning in NLP) 13:45 - Answer search (in prompt-based learning in NLP) 15:03 - Answer mapping (in prompt-based learning in NLP) 15:37 - Example of input, template, and answer for different tasks in prompt-based learning in NLP 17:21 - Design Consideration for Prompting 19:14 - Content for Next video
Comments
loading...