Sign in
Size does not matter | The efficiency misnomer | What does the number of parameters mean?
Dec 06, 2021
|
31 views
AI Coffee Break with Letitia
Follow
Details
πΊ Watch on YouTube π
How important is the number of parameters in deep learning models? But what about other measures like FLOPs or speed/throughput? βΊ Check out our sponsor Aleph Alpha π
https://www.aleph-alpha.de/
! Follow them on Twitter: Aleph__Alpha Paper π: Dehghani, Mostafa, Anurag Arnab, Lucas Beyer, Ashish Vaswani, and Yi Tay. "The Efficiency Misnomer." arXiv preprint arXiv:2110.12894 (2021).
https://arxiv.org/abs/2110.12894
π Megatron-Turing NLG 530B:
https://www.microsoft.com/en-us/research/blog/using-deepspeed-and-megatron-to-train-megatron-turing-nlg-530b-the-worlds-largest-and-most-powerful-generative-language-model/
Thanks to our Patrons who support us in Tier 2, 3, 4: π donor, Dres. Trost GbR, Yannik Schneider Outline: 00:00 Model efficiency comparison 02:51 FLOPs 03:55 Number of parameters: means what? 06:31 Speed / throughput 09:39 Aleph Alpha (Sponsor) ββββββββββββββββββββββββββ π₯ Optionally, pay us a coffee to help with our Coffee Bean production! β Patreon:
https://www.patreon.com/AICoffeeBreak
Ko-fi:
https://ko-fi.com/aicoffeebreak
ββββββββββββββββββββββββββ π Links: AICoffeeBreakQuiz:
https://www.youtube.com/c/AICoffeeBreak/community
Twitter:
https://twitter.com/AICoffeeBreak
Reddit:
https://www.reddit.com/r/AICoffeeBreak/
YouTube:
https://www.youtube.com/AICoffeeBreak
#AICoffeeBreak #MsCoffeeBean #MachineLearning #AI #researchβ
00:00
Model efficiency comparison
02:51
FLOPs
03:55
Number of parameters: means what?
06:31
Speed / throughput
09:39
Aleph Alpha (Sponsor)
Category: Research Paper
Comments
loading...
Reactions
(0)
| Note
π No reactions yet
Be the first one to share your thoughts!
Reactions
(0)
Note
loading...
Recommended
18:33
ICAPS 2014: Stefan Edelkamp on "Symbolic and Explicit Search Hybrid through Perfect Hash Functions"
ICAPS
| Aug 7, 2014
50:27
ICAPS 2012: Invited Talk by Anthony Cohn
ICAPS
| Aug 15, 2014
56:04
ICAPS 2012: Invited talk by George Pappas
ICAPS
| Aug 18, 2014
51:45
ICAPS 2012: Opening Remark & Invited Talk by Robert Ambrose
ICAPS
| Aug 18, 2014
23:37
ICAPS 2012: Session Ia on "Temporal Planning & Scheduling"
ICAPS
| Aug 18, 2014