Optimizing the Learning Rate of your Neural Networks

If I have learned anything from watching online lectures about gradient descent, the worse your drawings are, the more likely you are to be a math professor.
Source Fastai course chapter 5, the loss steadily decreases then rapidly rises

Discriminative Learning Rates

Conclusions

--

--

--

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Face Recognition with Deep Learning (Part II)

What is AutoML? [Part 1]

Exploring AWS Lambda Deployment Limits

Easy Topic Classifier on iOS with Apple’s Natural Language Framework

What is Machine Learning?

My Journey Into Reinforcement Learning (Part 3) — Dynamic Programming

BiLSTM-CRF for Aspect Term Extraction

Accuracy vs Interpretability paradox

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Aidan Coco

Aidan Coco

More from Medium

Developing a simple CNN for MNIST

Introduction to artificial neural networks and activation functions

Applications of Deep Learning: Convolutional Neural Network Models In the Healthcare Industry: Part…

Introduction to Neural Networks Part-1