What is tokenization? Ms. Coffee Bean explains tokenization (highly relevant for transformers nowadays) and highlights the importance of flexible tokenization. Enjoy the animations!
π Our Veritasium Contest submission: https://youtu.be/a6sKYiGMuXc
Watch and like it to support us! only the top 100 most viewed videos qualify for the contest.
πΊ The Charformer: https://youtu.be/debgj24BAZE
Thanks to our Patrons who support us in Tier 2, 3, 4: π
donor, Dres. Trost GbR
ββββββββββββββββββββββββββ
π₯ Optionally, pay us a coffee to help with our Coffee Bean production! β
βΊ Patreon: https://www.patreon.com/AICoffeeBreak
βΊ Ko-fi: https://ko-fi.com/aicoffeebreak
ββββββββββββββββββββββββββ
Outline:
00:00 Warning
00:27 What is tokenization?
01:47 How does it work?
02:34 The problems
03:31 Β―\_(γ)_/Β―
-------------------------------------
π Links:
AICoffeeBreakQuiz: https://www.youtube.com/c/AICoffeeBreak/community
Twitter: https://twitter.com/AICoffeeBreak
Reddit: https://www.reddit.com/r/AICoffeeBreak/
YouTube: https://www.youtube.com/AICoffeeBreak
#AICoffeeBreak #MsCoffeeBean #MachineLearning #AI #researchβ
00:00 Warning
00:27 What is tokenization?
01:47 How does it work?
02:34 The problems
03:31 Β―\_(γ)_/Β―
