#bert #gpt-2 #textsummarization #researchpaperwalkthrough
In this video, we will go through an interesting work that tries to automatically summarise covid-19 related (low resource) research articles using Transformer models such as BERT and GPT-2 in a abstractive setting.
Please feel free to share out the content and subscribe to my channel and get notified when i upload next :)
⏩ Subscribe - https://youtube.com/channel/UCoz8NrwgL7U9535VNc0mRPA?sub_confirmation=1
⏩ Abstract: With the COVID-19 pandemic, there is a growing urgency for medical community to keep up with the accelerating growth in the new coronavirus-related literature. As a result, the COVID-19 Open Research Dataset Challenge has released a corpus of scholarly articles and is calling for machine learning approaches to help bridging the gap between the researchers and the rapidly growing publications. Here, we take advantage of the recent advances in pre-trained NLP models, BERT and OpenAI GPT-2, to solve this challenge by performing text summarization on this dataset. We evaluate the results using ROUGE scores and visual inspection. Our model provides abstractive and comprehensive information based on keywords extracted from the original articles. Our work can help the the medical community, by providing succinct summaries of articles for which the abstract are not already available.
⏩ OUTLINE:
0:00 - Abstract and Introduction
2:20 - Summarisation and Low Resource Problem
5:12 - Extractive Summarisation Approach (Baseline)
7:01 - Understanding K-means and K-medoids Clustering Algorithm
13:50 - Training Strategy for Abstractive Summarisation GPT-2
17:52 - Intuition of the Training Strategy
21:15 - My thoughts
⏩ Paper Title: Automatic Text Summarization of COVID-19 Medical Research Articles using BERT and GPT-2
⏩ Paper Link: https://arxiv.org/abs/2006.01997
⏩ Paper Authors: Virapat Kieuvongngam, Bowen Tan, Yiming Niu
⏩ Organization: Rockefeller University
⏩ Code: https://github.com/VincentK1991/BERT_summarization_1
⏩IMPORTANT LINKS
K-Medoid Clustering - https://www.coursera.org/lecture/cluster-analysis/3-4-the-k-medoids-clustering-method-nJ0Sb
*********************************************
⏩ Youtube - https://youtube.com/channel/UCoz8NrwgL7U9535VNc0mRPA
⏩ Blog - https://prakhartechviz.blogspot.com
⏩ LinkedIn - https://linkedin.com/in/prakhar21
⏩ Medium - https://medium.com/@prakhar.mishra
⏩ GitHub - https://github.com/prakhar21
*********************************************
Please feel free to share out the content and subscribe to my channel :)
⏩ Subscribe - https://youtube.com/channel/UCoz8NrwgL7U9535VNc0mRPA?sub_confirmation=1
Tools I use for making videos :)
⏩ iPad - https://tinyurl.com/y39p6pwc
⏩ Apple Pencil - https://tinyurl.com/y5rk8txn
⏩ GoodNotes - https://tinyurl.com/y627cfsa
About Me:
I am Prakhar Mishra and this channel is my passion project. I am currently pursuing my MS (by research) in Data Science. I have an industry work-ex of 3 years in the field of Data Science and Machine Learning with a particular focus in Natural Langauge Processing (NLP).
#transformermodels #languagemodels #techviz #datascienceguy #naturallanguageprocessing #nlp #documentsummarization
