Transformers are meanwhile famous in natural language processing. New research applies the BERT Transformer architecture on amino-acid sequences and discovers biological properties of proteins! 🧬 Because why not? 😊
"BERTology Meets Biology" paper explained!
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
🔥 Optionally, pay us a coffee to boost our Coffee Bean production! ☕
Patreon: https://www.patreon.com/AICoffeeBreak
Ko-fi: https://ko-fi.com/aicoffeebreak
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
Transformer explained by Ms. Coffee Bean: https://youtu.be/FWFA4DGuzSc
What is probing? Ms. Coffee Bean explained: https://youtu.be/fL22NAtMNYo
Outline:
* 00:00 Solving biology with the Transformer
* 01:08 Language BERT recap
* 02:37 Biology Bootcamp
* 05:43 BERT on biology
📄 Paper: Jesse Vig, Ali Madani, Lav R. Varshney, Caiming Xiong, Richard Socher, and Nazneen Fatema Rajani. "BERTology Meets Biology: Interpreting Attention in Protein Language Models." arXiv preprint arXiv:2006.15222 (2020). https://arxiv.org/pdf/2006.15222.pdf
💻Code: https://github.com/salesforce/provis
More code providing state of the art pre-trained models for proteins: https://github.com/agemagician/ProtTrans
Scientific advisor for this video: Nils Trost
Hungry for more? 📚
Check out @Yannic Kilcher's video on this: https://youtu.be/q6Kyvy1zLwQ
🔗 Links:
YouTube: https://www.youtube.com/AICoffeeBreak
Twitter: https://twitter.com/AICoffeeBreak
Reddit: https://www.reddit.com/r/AICoffeeBreak/
#AICoffeeBreak #MsCoffeeBean #BERT #attention #Biology #MachineLearning #AI #research #Bioinformatics
00:00 Solving biology with the Transformer
01:08 Language BERT recap
02:37 Biology Bootcamp
05:43 BERT on biology
