Theoretical and advanced machine learning with TensorFlow
Before starting on the learning materials below, be sure to:
Complete our curriculum Basics of machine learning with TensorFlow, or have equivalent knowledge
Have software development experience, particularly in Python
This curriculum is a starting point for people who would like to:
Improve their understanding of ML
Begin understanding and implementing papers with TensorFlow
You should already have background knowledge of how ML works or completed the learning materials in the beginner curriculum Basics of machine learning with TensorFlow before continuing. The below content is intended to guide learners to more theoretical and advanced machine learning content. You will see that many of the resources use TensorFlow, however, the knowledge is transferable to other ML frameworks.
To further your understanding of ML, you should have Python programming experience as well as a background in calculus, linear algebra, probability, and statistics. To help you deepen your ML knowledge, we have listed a number of recommended resources and courses from universities, as well as a couple of textbooks.
Step 1: Refresh your understanding of math concepts
ML is a math heavy discipline. If you plan to modify ML models, or build new ones from scratch, familiarity with the underlying math concepts is important. You don't have to learn all the math upfront, but instead you can look up concepts you are unfamiliar with as you come across them. If it's been a while since you've taken a math course, try watching the Essence of linear algebra and the Essence of calculus playlists from 3blue1brown for a refresher. We recommend that you continue by taking a class from a university, or watching open access lectures from MIT, such as Linear Algebra or Single Variable Calculus.
This introductory course from MIT covers matrix theory and linear algebra. Emphasis is given to topics that will be useful in other disciplines, including systems of equations, vector spaces, determinants, eigenvalues, similarity, and positive definite matrices.
Step 2: Deepen your understanding of deep learning with these courses and books
There is no single course that will teach you everything you need to know about deep learning. One approach that may be helpful is to take a few courses at the same time. Although there will be overlap in the material, having multiple instructors explain concepts in different ways can be helpful, especially for complex topics. Below are several courses we recommend to help get you started. You can explore each of them together, or just choose the ones that feel the most relevant to you.
Remember, the more you learn, and reinforce these concepts through practice, the more adept you will be at building and evaluating your own ML models.
Take these courses:
MIT course 6.S191: Introduction to Deep Learning is an introductory course for Deep Learning with TensorFlow from MIT and also a wonderful resource.
Andrew Ng's Deep Learning Specialization at Coursera also teaches the foundations of deep learning, including convolutional networks, RNNS, LSTMs, and more. This specialization is designed to help you apply deep learning in your work, and to build a career in AI.
DeepLearning.AIDeep Learning Specialization
In five courses, you will learn the foundations of Deep Learning, understand how to build neural networks, and learn how to lead successful machine learning projects and build a career in AI. You will master not only the theory, but also see how it is applied in industry.
⬆ And ⬇ Read these books:
To complement what you learn in the courses listed above, we recommend that you dive deeper by reading the books below. Each book is available online, and offers supplementary materials to help you practice.
You can start by reading Deep Learning: An MIT Press Book by Ian Goodfellow, Yoshua Bengio, and Aaron Courville. The Deep Learning textbook is an advanced resource intended to help students deepen their understanding. The book is accompanied by a website, which provides a variety of supplementary materials, including exercises, lecture slides, corrections of mistakes, and other resources to give you hands on practice with the concepts.
You can also explore Michael Nielsen's online book Neural Networks and Deep Learning. This book provides a theoretical background on neural networks. It does not use TensorFlow, but is a great reference for students interested in learning more.
Step 3: Read and implement papers with TensorFlow
At this point, we recommend reading papers and trying the advanced tutorials on our website, which contain implementations of a few well known publications. The best way to learn an advanced application, machine translation, or image captioning, is to read the paper linked from the tutorial. As you work through it, find the relevant sections of the code, and use them to help solidify your understanding.