Transformers are meanwhile famous in natural language processing. New research applies the BERT Transformer architecture on amino-acid sequences and discovers biological properties of proteins! 𧬠Because why not? π
"BERTology Meets Biology" paper explained!
ββββββββββββββββββββββββββ
π₯ Optionally, pay us a coffee to boost our Coffee Bean production! β
Patreon: https://www.patreon.com/AICoffeeBreak
Ko-fi: https://ko-fi.com/aicoffeebreak
ββββββββββββββββββββββββββ
Transformer explained by Ms. Coffee Bean: https://youtu.be/FWFA4DGuzSc
What is probing? Ms. Coffee Bean explained: https://youtu.be/fL22NAtMNYo
Outline:
* 00:00 Solving biology with the Transformer
* 01:08 Language BERT recap
* 02:37 Biology Bootcamp
* 05:43 BERT on biology
π Paper: Jesse Vig, Ali Madani, Lav R. Varshney, Caiming Xiong, Richard Socher, and Nazneen Fatema Rajani. "BERTology Meets Biology: Interpreting Attention in Protein Language Models." arXiv preprint arXiv:2006.15222 (2020). https://arxiv.org/pdf/2006.15222.pdf
π»Code: https://github.com/salesforce/provis
More code providing state of the art pre-trained models for proteins: https://github.com/agemagician/ProtTrans
Scientific advisor for this video: Nils Trost
Hungry for more? π
Check out @Yannic Kilcher's video on this: https://youtu.be/q6Kyvy1zLwQ
π Links:
YouTube: https://www.youtube.com/AICoffeeBreak
Twitter: https://twitter.com/AICoffeeBreak
Reddit: https://www.reddit.com/r/AICoffeeBreak/
#AICoffeeBreak #MsCoffeeBean #BERT #attention #Biology #MachineLearning #AI #research #Bioinformatics
00:00 Solving biology with the Transformer
01:08 Language BERT recap
02:37 Biology Bootcamp
05:43 BERT on biology