A Comfy Guide to Start Deep Learning

If you don’t know what deep learning is,then go here or here and read carefully .

Studying deep learning is a challenging but at the same time super exciting because you get to experiment with a mix of sci-fi & art & science .

When I started learning the subject and I’m still learning I fell into many pitfalls and as a russian proverb says if you chase two rabbits you will not catch either one that’s why many self-taught people end up lost or find more obstacles the lack of organization and the huge ammount of hype you’ll end up spending more time filtering the noise,this little guide here is supposed to get you started ,up and running (I hope) and help guide you in your learning path . I wrote this for myself personally you may not like this guide and I don’t blame you ,but I personally prefer books and papers + videos I spend a lot of time focusing on the basics and understanding everything at a deep level before moving to the next thing .

Trends come and go but my foundations still eternal - Tupac

The time intervals is given as an approximation I’m in college so I usually get not much time during the week,this is like night classes or weekend classes .

I also assumed you’re comfortable with undergrad mathematics Linear Algebra,Calculus Statistics and Probabilites I put some ressources at the end to help with this as well .

Note that this guide isn’t a school curriculum but more like an organized step by step guide . I’m not an expert

The Challengers: one shot learning:

• fast.ai : This course is complete end-to-end I loved it and following it should get you started writing models and doing kaggle submissions on day 1 it’s perfect for people who prefer visual learning more than just reading .

• I wouldn’t recommend Deep Learning by Google on Udacity some will like it others won’t I’m part of the others and I didn’t take The deeplearning.ai course by Andrew Ng so I can’t say nothing about it .

• Deep Learning Book Keep a tab open for this one it’s the most important book on deep learning ,if deep learning is a religion this is the holy book .

• Deep Learning Tutorial I loved it and read it twice it has the best mixture of theory and practice altough using Theano , you won’t have an issue using PyTorch or Tensorflow but the theoretical parts are really worth reading .

---

Week 1 -> Week 2:

• Take the first week to learn how to manipulate data plain and simple data science take this first week to go trough this book one of the most common pitfalls is not to understand data manipulation if you can’t clean and organize your data visualize it and make hypothesis you’ll have a hard time . :

• Python Data Science Handbook

Week 2 -> Week 4:

I think getting started is the easy part especially if you just want to understand how things work for starters I recommend the following :

• Michael Nielsen Neural Networks and Programming :

  This is a gem period this is the best piece of literatture one should read to get started it’s clear,concise,practical (Python code) and explains the important concepts like backpropagation,regularization,hyperparameters and explains in detail why Neural network actually even work . Take your time to read it carefully and read it slowly and understand every part of it because trust me it will help you on the long run more than you can imagine

• CS231n : a Stanford course by the mighty Karpathy I suggest module 0 and 1 to get started learn to use numpy and matplotlib writing neural networks without libraries is a great exercice to understand things at a deeper level

---

By here you should be exited to dive deeper and it’s here where I stop you don’t google LSTM or RNN or CNNs and try to squash them because it’s gonna give you some issues later by here you should started getting your hands dirty with code,lots of code :) .

• Download 3 different datasets from Kaggle anything you like (Housing + Titanic + MNIST) is a good starting choice because you get Image,Regression,Classification but anything else should be good .

• Once you have the data use what you learned from the previous weeks to build models and evaluate them I suggest two ways :

  • If you’re like me you’ll write everything from scratch it’s long,complex,and will be buggy but trust me manipulating matrices using numpy is very very useful on the long term bonus points if you use PyTorch this repo should get you started .

  • Otherwise then install keras and start using it :

    • https://keras.io/

---

Misc:

This are the extras read these when you’re sipping you’re morning coffee or taking a pause or whatever :) :

• Chris Olah Backpropagation tutorial .
• Trask’s blog Read all of it from the first to the last post and send him a thank you message .
• General Guidelines A Chapter from THE deep learning book .
• Optimization Methods and Gradient Descent This should clarify anything about gradient descent .

---

Week 4 -> Week X :

Most will suggest to start with an architecture and go from there you can pick LSTM,RNN,CNN … but trust me if you got the basics right you’ll have no problem picking a specialty so from this point you can take the following in any order you would like and I mean any order the rest is mainly practice :

Now you’ll probably have enough understanding to get into some formal literature

I suggest the following papers to get started and this Roadmap should help you organize what you read next :

• Lecun,Bengio,Hinton Survey

• Hinton et al An Algorithm for fast Deep Belief training a corner stone .

• Hinton,Salakhutdinov Dimensionality Reduction using NN

---

CNN :

• Finish CS231n it’s the best Convolutional Network course I’ve ever read and it’s full of other ressources

• Chris Olah CNN’s

• Understanding Convolutions

• CNN can be used for NLP

• ConvNets in practice

• Kaggle Diabetic Retinopathy winner’s writeup

• Object Detection Using Neural Networks

---

RNN GRU LSTM

• Karpathy’s blog post You’re gonna love this architecture .

• Tensorflow’s Tutorial

• Understanding LSTM’s by C.Olah

• LSTM For Sent analysis

• Introduction to Memory Based Networks

• WildML Intro to RNN

Autoencoders

• Autoencoders in Keras

• A Description of Autencoders

Nature Language Processing

• CS224d This is the equivalent of CS231n but for NLP .

Books

• Grokking Deep Learning by Andrew Trask Bookshelf worthy amazing book

• Hands On Machine Learning … by Aurélien Gérond

• Fundamentals of Deep Learning by N.Buduma

Misc Ressources :

• Oxford Course uses TORCH (Lua Version) but the slides are amazing here

• If you ever want to see a tensorflow implementation this repo is gold .

• Datasets because how can you learn without getting your hands dirty

• Kaggle of course .

• r/MachineLearning A Great subreddit with amazing content .

General Advice

• Pick PyTorch for working because it’s dynamic that means you reiterate faster on your code you’re computational graph that constitutes you’re neural network doesn’t need to be compiled ( tf.sess.run() I see you ) this will speed the process for you learning and experimenting plus it’s less verbose and more hands-on also choose to use Keras , because in my opinion as a self-taught person I spent hours… days learning tensorflow and I just couldn’t understand why go trough so much hell to to write a 3 layer network prefer fast and terse of over-engineered and verbose and complex .

• Practice makes perfect and the master has failed more than the begineer ever tried this is the most important thing you should do and learn and live by .

Happy Learning and if this ever helped you say thanks on twitter :)