Optimizing the Learning Rate of your Neural Networks

If I have learned anything from watching online lectures about gradient descent, the worse your drawings are, the more likely you are to be a math professor.
Source Fastai course chapter 5, the loss steadily decreases then rapidly rises

Discriminative Learning Rates





Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Face Recognition with Deep Learning (Part II)

What is AutoML? [Part 1]

Exploring AWS Lambda Deployment Limits

Easy Topic Classifier on iOS with Apple’s Natural Language Framework

What is Machine Learning?

My Journey Into Reinforcement Learning (Part 3) — Dynamic Programming

BiLSTM-CRF for Aspect Term Extraction

Accuracy vs Interpretability paradox

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Aidan Coco

Aidan Coco

More from Medium

Developing a simple CNN for MNIST

Introduction to artificial neural networks and activation functions

Applications of Deep Learning: Convolutional Neural Network Models In the Healthcare Industry: Part…

Introduction to Neural Networks Part-1