EMNLP 2019

GlossBERT_ BERT for Word Sense Disambiguation with Gloss Knowledge (Research Paper Walkthrough)

Apr 12, 2021

|

62 views

|

prakharmishra137

Sentence Classification

Word Sense Disambiguation

Machine Learning

Details

#bert #wsd #wordnet This research uses BERT for Word Sense Disambiguation task by modeling the entire problem as sentence classification problem by using the Gloss knowledge. They achive SOTA results on benchmark datasets. ⏩ Abstract: Word Sense Disambiguation (WSD) aims to find the exact sense of an ambiguous word in a particular context. Traditional supervised methods rarely take into consideration the lexical resources like WordNet, which are widely utilized in knowledge-based methods. Recent studies have shown the effectiveness of incorporating gloss (sense definition) into neural networks for WSD. However, compared with traditional word expert supervised methods, they have not achieved much improvement. In this paper, we focus on how to better leverage gloss knowledge in a supervised neural WSD system. We construct context-gloss pairs and propose three BERT-based models for WSD. We fine-tune the pre-trained BERT model on SemCor3.0 training corpus and the experimental results on several English all-words WSD benchmark datasets show that our approach outperforms the state-of-the-art systems. Please feel free to share out the content and subscribe to my channel :) ⏩ Subscribe - https://youtube.com/channel/UCoz8NrwgL7U9535VNc0mRPA?sub_confirmation=1 ⏩ OUTLINE: 0:00 - Abstract 01:46 - Task Definition 02:11 - Data Collection approach 02:30 - WordNet Overview 03:35 - Sentence construction method table overview 05:27 - BERT(Token-CLS) 06:41 - GlossBERT 07:52 - Context-Gloss Pair with Weak Supervision 08:55 - GlossBERT(Token-CLS) 09:20 - GlossBERT(Sent-CLS) 09:44 - GlossBERT(Sent-CLS-WS) 10:09 - Results ⏩ Paper Title: GlossBERT: BERT for Word Sense Disambiguation with Gloss Knowledge ⏩ Paper: https://arxiv.org/abs/1908.07245v4 ⏩ Code: https://github.com/HSLCY/GlossBERT ⏩ Author: Luyao Huang, Chi Sun, Xipeng Qiu, Xuanjing Huang ⏩ Organisation: Fudan University ⏩ IMPORTANT LINKS Full Playlist on BERT usecases in NLP: https://www.youtube.com/watch?v=kC5kP1dPAzc&list=PLsAqq9lZFOtV8jYq3JlkqPQUN5QxcWq0f Full Playlist on Text Data Augmentation Techniques: https://www.youtube.com/watch?v=9O9scQb4sNo&list=PLsAqq9lZFOtUg63g_95OuV-R2GhV1UiIZ Full Playlist on Text Summarization: https://www.youtube.com/watch?v=kC5kP1dPAzc&list=PLsAqq9lZFOtV8jYq3JlkqPQUN5QxcWq0f Full Playlist on Machine Learning with Graphs: https://www.youtube.com/watch?v=-uJL_ANy1jc&list=PLsAqq9lZFOtU7tT6mDXX_fhv1R1-jGiYf Full Playlist on Evaluating NLG Systems: https://www.youtube.com/watch?v=-CIlz-5um7U&list=PLsAqq9lZFOtXlzg5RNyV00ueE89PwnCbu ********************************************* ⏩ Youtube - https://www.youtube.com/c/TechVizTheDataScienceGuy ⏩ Blog - https://prakhartechviz.blogspot.com ⏩ LinkedIn - https://linkedin.com/in/prakhar21 ⏩ Medium - https://medium.com/@prakhar.mishra ⏩ GitHub - https://github.com/prakhar21 ⏩ Twitter - https://twitter.com/rattller ********************************************* Please feel free to share out the content and subscribe to my channel :) ⏩ Subscribe - https://youtube.com/channel/UCoz8NrwgL7U9535VNc0mRPA?sub_confirmation=1 Tools I use for making videos :) ⏩ iPad - https://tinyurl.com/y39p6pwc ⏩ Apple Pencil - https://tinyurl.com/y5rk8txn ⏩ GoodNotes - https://tinyurl.com/y627cfsa #techviz #datascienceguy #ai #researchpaper #naturallanguageprocessing

0:00 Abstract 01:46 Task Definition 02:11 Data Collection approach 02:30 WordNet Overview 03:35 Sentence construction method table overview 05:27 BERT(Token-CLS) 06:41 GlossBERT 07:52 Context-Gloss Pair with Weak Supervision 08:55 GlossBERT(Token-CLS) 09:20 GlossBERT(Sent-CLS) 09:44 GlossBERT(Sent-CLS-WS) 10:09 Results

Category: EMNLP 2019

Comments

loading...

Reactions (0) | Note

📝 No reactions yet

Be the first one to share your thoughts!

Reactions(0)

Note

loading...

Recommended

ICAPS 2012: Session VIb on "Planning and Scheduling for Shipping and Transportation"

ICAPS | Aug 18, 2014

ICAPS 2012: Awards & Best Dissertation Presentations

ICAPS | Aug 19, 2014

ICAPS 2014: Miquel Ramírez on "Directed Fixed-Point Regression-Based Planning..."

ICAPS | Sep 22, 2014

Representation Learning: A Review and New Perspectives

AISTATS 2014 | Nov 12, 2014

Playing Atari with Deep Reinforcement Learning | Two Minute Paper

Two Minute Papers | Mar 7, 2015