OpenAI’s CLIP explained! | Examples, links to code and pretrained model

Nov 27, 2021

|

32 views

AI Coffee Break with Letitia

Details

Ms. Coffee Bean explains ❓ how OpenAI‘s CLIP works, ❔ what it can and cannot do ⁉️ and what people have been up to using CLIP in awesome applications! ▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀ 🔥 Optionally, pay us a coffee to boost our Coffee Bean production! ☕ Patreon: https://www.patreon.com/AICoffeeBreak Ko-fi: https://ko-fi.com/aicoffeebreak ▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀ 📺 DALL-E explained by Ms. Coffee Bean: https://youtu.be/mvG2FGF0TvM Outline: * 00:00 CLIP and DALL-E * 02:14 What can CLIP do? * 04:22 How does CLIP work? * 09:12 Limitations * 12:39 Applications 📄 CLIP paper: Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., ... & Sutskever, I. Learning Transferable Visual Models From Natural Language Supervision. Image, 2, T2. https://cdn.openai.com/papers/Learning_Transferable_Visual_Models_From_Natural_Language_Supervision.pdfd (last date accessed: 24.01.2021) 📚 CLIP blog: openai.com/blog/clip/ 💻 CLIP Colab link: https://colab.research.google.com/github/openai/clip/blob/master/Interacting_with_CLIP.ipynb Other links: 📄 Switch Transformers: Fedus, W., Zoph, B., & Shazeer, N. (2021). Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity. arXiv preprint arXiv:2101.03961. https://arxiv.org/pdf/2101.03961.pdf 📚 Check out the section “10 Novel Applications using Transformers”: https://paperswithcode.com/newsletter/3 🐦Links to the application showcase in Twitter: 1. https://twitter.com/l4rz/status/1352630033832140800 2. https://twitter.com/haltakov/status/1351271379103002632 3. https://twitter.com/nagaraj_arvind/status/1350030541467258881 4. https://twitter.com/sayantandas_/status/1351830997403254785 ------------------------------------------ 🔗 Links: YouTube: https://www.youtube.com/AICoffeeBreak Twitter: https://twitter.com/AICoffeeBreak Reddit: https://www.reddit.com/r/AICoffeeBreak/ #AICoffeeBreak #MsCoffeeBean #MachineLearning #AI #research Thumbnail contains the tick emoji designed by OpenMoji – the open-source emoji and icon project. License: CC BY-SA 4.0

00:00 CLIP and DALL-E 02:14 What can CLIP do? 04:22 How does CLIP work? 09:12 Limitations 12:39 Applications

Category: Research Paper

Comments

loading...

Reactions (0) | Note

📝 No reactions yet

Be the first one to share your thoughts!

Reactions(0)

Note

loading...

Recommended

ICAPS 2014: Truong-Huy Dinh Nguyen on "Bootstrapping Simulation-Based Algorithms..."

ICAPS | Aug 5, 2014

ICAPS 2014: Peter Stone on "TEXPLORE: Real-Time Sample-Efficient Reinforcement Learning for Robots"

ICAPS | Aug 6, 2014

ICAPS 2014: Ron Alford on "On the Feasibility of Planning Graph Style Heuristics for HTN Planning"

ICAPS | Aug 6, 2014

ICAPS 2014: Mike Barley on "Overcoming the Utility Problem in Heuristic Generation: ..."

ICAPS | Aug 7, 2014

ICAPS 2014: Stefan Edelkamp on "Symbolic and Explicit Search Hybrid through Perfect Hash Functions"

ICAPS | Aug 7, 2014