Channel: Siraj Raval

Category: Education

Tags: pythonsirajnatural language processinghashingtransformerlanguage modelingaigoogletext summarizationtext classificationmachine learningword2vecneural networkartificial intelligencetext generationdeep learningtext processingnlpsiraj ravalprogrammingreformer

Description: Google just released a new Neural Network architecture called "The Reformer" that helps make the state of the art in Natural Language Processing available to people with a smaller budget. They introduced two key ideas here, "reversible layers" and "locality sensitive hashing". Both of these concept build on about a decade of research in neural networks applied to NLP, so to truly understand the Reformer, we've got to understand the timeline of deep learning based NLP. In this episode, I'll take you on a journey through the advances that lead up to the Reformer, what the relevant applications of it are, and how you can get started building your own text application with it today in a few lines of Python code. All the code I demonstrate in this video is from various developers across GitHub, I do not claim any of it as my own, I am using them as examples to enable more people outside of Academia to understand modern NLP. Enjoy! TWITTER: bit.ly/2OHYLbB WEBSITE: bit.ly/2OoVPQF INSTAGRAM: bit.ly/312pLUb FACEBOOK: bit.ly/2OqOhx1 Please subscribe for more educational videos! It would mean so much to me. Google's Reformer Code (Text Generation): colab.research.google.com/github/google/trax/blob/master/trax/models/reformer/text_generation.ipynb Bintang's winning code (forked, yes, but thanks for bringing this to our attention, cool repo!) : github.com/baswam95/SUAVE Are you a total beginner to machine learning? Watch this: youtube.com/watch?v=Cr6VqTRO1v0 Learn Python: youtube.com/watch?v=T5pRlIbr6gg Free Natural Language Processing Curriculum: youtube.com/watch?v=GazFsfcijXQ Natural Language Processing: youtube.com/watch?v=bDxFvr1gpSU Sentdex has some great playlists on NLP: youtube.com/watch?v=FLZvOKSCkxY Google's Initial Reformer Blog Post ai.googleblog.com/2020/01/reformer-efficient-transformer.html Word2Vec Explained: youtube.com/watch?v=LSS_bos_TPI FastText Explained: youtube.com/watch?v=4l_At3oalzk Transformer Explained: youtube.com/watch?v=iDulhoQ2pro Jay Alammar is an ML animation beast: jalammar.github.io/illustrated-transformer BERT explained: youtube.com/watch?v=BhlOGGzC0Q0 Backpropagation explained: youtube.com/watch?v=Ilg3gGewQ5U Beat used for Hashing rap: youtube.com/watch?v=zkaT8-NhD5c Hit the Join button above to sign up to become a member of my channel for access to exclusive live streams! Signup for my newsletter for exciting updates in the field of AI: goo.gl/FZzJ5w And please support me on Patreon: patreon.com/user?u=3191693