Fixed

learning rate固定不變

base_lr = 0.01 lr_policy = "fixed"

Step

learning rate在每迭代stepsize次后減少gamma倍。lr=lr×gamma

base_lr = 0.01 lr_policy = "step" gamma = 0.1 stepsize= 10000

Polynomial

learning rate呈多項(xiàng)式曲線下降。LR(t)=base_lr×(tT)power

base_lr = 0.01 lr_policy = "poly" power = 0.5

Inv

learning rate隨迭代次數(shù)增加而下降。LR(t)=base_lr×(1+gamma×iter)power

base_lr = 0.01 lr_policy = "Inv" gamma = 0.0001 power = 0.75

總結(jié)

以上是生活随笔為你收集整理的learning rate四种改变方式的全部?jī)?nèi)容，希望文章能夠幫你解決所遇到的問(wèn)題。

如果覺(jué)得生活随笔網(wǎng)站內(nèi)容還不錯(cuò)，歡迎將生活随笔推薦給好友。