Plans for week 37, September 8-12_澳门葡京手机版app下载

mhjensen
2025-09-07T11:32:26.848Z
type:structured-message
www.uio.no

Dear all, welcome back to FYS-STK3155/4155 and a new exciting week! Our plans this week are

to discuss the family of gradient descent methods which we need to implement in the project (parts c-e, including Lasso regression).

1. Plain gradient descent (constant learning rate), reminder from last week with examples using OLS and Ridge

2. Improving gradient descent with momentum

3. Introducing stochastic gradient descent

4. More advanced updates of the learning rate: ADAgrad, RMSprop and ADAM

Readings and Videos:

1. Recommended: The textbook Goodfellow et al, Deep Learning, contains a good introduction to gradient descent, see sections 4.3-4.5 at https://www.deeplearningbook.org/contents/nume...