Advanced Machine Learning
Faculty
Sergey Nikolenko
Chief Research Officer, Neuromation Head of AI Lab, PDMI RAS
Course length
Duration
Total hours
Credits
Language
Course type
Fee for single course
Fee for degree students
Skills you’ll learn
Overview
Ten years ago, machine learning went through a revolution. While neural networks had been one of the oldest tools in artificial intelligence, people had not really been able to train deep architectures efficiently until mid-2000s. After the breakthrough results of the groups of Geoffrey Hinton and Yoshua Bengio, however, deep neural architectures quickly outperformed state of the art in image processing, speech recognition, natural language processing, and by now they basically define the modern state of machine learning in many different domains, from face recognition and self-driving cars to playing Go. In the course, we will see what makes modern neural networks so powerful, learn to train them properly, go through the most important architectures, and, best of all, learn to implement all of these ideas in code through standard libraries such as TensorFlow and Keras.
Learning highlights
- Learn classical and modern architectures in neural networks
- Learn how to train various deep neural architectures.
- Understand a wide variety of neural architectures suited for different tasks
- Learn to implement these ideas in standard neural network libraries
Course outline
10 classes
Session 1
Neural network basics. The perceptron.
Neural networks: history and basic idea. Relationship between biology and mathematics. The perceptron: basic construction, training, activation functions. Practice: intro to TensorFlow and Keras.
Feedforward neural networks
Feedforward neural networks. Gradient descent basics. Computation graph and computing gradients on the computation graph (backpropagation). Why deep learning is hard. Practice: a feedforward neural network on the MNIST dataset.
Session 3
Optimization in neural networks.
Gradient descent and its problems. Nesterov’s momentum. Second order methods. Adaptive methods of gradient descent: Adagrad, Adadelta, Adam. Practice: comparing gradient descent variations.
Autoencoders
Autoencoders. Sparse autoencoders, regularization, denoising autoencoders. Deconvolution and convolutional autoencoders. Practice: feature extraction with autoencoders.
Convolutional neural networks
Convolutional architectures: idea and structure. Examples. Deconvolution. Modern architectures. Practice: CNNs for MNIST.
Autoencoders
Autoencoders. Sparse autoencoders, regularization, denoising autoencoders. Deconvolution and convolutional autoencoders. Practice: feature extraction with autoencoders.
Recurrent neural networks
Recurrent neural networks: idea, backprop in RNNs. Vanishing and exploding gradients. LSTM, GRU, and other architectures. Practice: sentiment analysis with RNNs.
Generative adversarial networks
Why do we need generative models? Idea of generative adversarial networks. Modern GAN architectures: DCGAN. Practice: learning a normal distribution.
Deep reinforcement learning
Reinforcement learning with neural networks. DQN. DQN for Atari games, AlphaGo. Practice: solving problems from OpenAI Gym.
Neurobayes
Neurobayesian methods. Variational autoencoder. A Bayesian look at dropout and dropout in RNNs. Practice: generating numbers with a variational autoencoder.
Prerequisites
This course is one of three in a wholistic series.
Students that have already taken MSL-111 and those with prior experience with HTML, CSS, and Javascript building simple web pages will be good candidates for this module.
Sergey Nikolenko is a computer scientist with vast experience in machine learning and data analysis, algorithms design and analysis, theoretical computer science, and algebra. He graduated from St. Petersburg State University in 2005, majoring in algebra (Chevalley groups), and earned his Ph.D at the Steklov Mathematical Institute at St. Petersburg in 2009 in theoretical computer science (circuit complexity and theoretical cryptography). Since then, Sergey has been interested in machine learning and probabilistic modeling, producing theoretical results and working on practical projects for the industry.
Sergey Nikolenko is currently serving as the Chief Research Officer at Neuromation, leading the Artificial Intelligence Lab at the Steklov Mathematical Institute at St. Petersburg, and teaching at the St. Petersburg State University and Higher School of Economics. Dr. Nikolenko has published more than 170 research papers on machine learning (ICML, CVPR, ACL, SIGIR, WSDM...), analysis of algorithms (SIGCOMM, INFOCOM, ICNP…), and other fields, several books, including a bestselling “Deep Learning” book (in Russian), lecture courses in ML, DL, other fields of computer science (St. Petersburg State University, NRU Higher School of Economics...) and much more. He has extensive experience in managing research and industrial AI/ML projects.
See full profileApply for this course
Advanced Machine Learning
by Sergey Nikolenko
Total hours
30 Hours
Dates
Apr 10 - Apr 21, 2017
Fee for single course
€1000
Fee for degree students
€500
How to secure your spot
Complete the form below to kickstart your application
Schedule your Harbour.Space interview
If successful, get ready to join us on campus
FAQ
Will I receive a certificate after completion?
Yes. Upon completion of the course, you will receive a certificate signed by the director of the program your course belonged to.
Do I need a visa?
This depends on your case. Please check with the Spanish or Thai consulate in your country of residence about visa requirements. We will do our part to provide you with the necessary documents, such as the Certificate of Enrollment.
Can I get a discount?
Yes. The easiest way to enroll in a course at a discounted price is to register for multiple courses. Registering for multiple courses will reduce the cost per individual course. Please ask the Admissions Office for more information about the other kinds of discounts we offer and what you can do to receive one.