监督学习中的损失函数及应用研究

邓建国,张素兰,张继福,荀亚玲,刘爱琴

Loss function and application research in supervised learning

Jianguo DENG,Sulan ZHANG,Jifu ZHANG,Yaling XUN,Aiqin LIU

表8 基于softmax损失的演化损失函数

名称 演化形式 softer softmax损失函数 $-\frac{1}{n}{\displaystyle \sum _{i=1}^n\frac{e^{\frac{f_{Y_i}}{T}}}{{\displaystyle \sum _{j=1}^c{e}^{\frac{f_j}{T}}}}}$ NSL损失函数 $-\frac{1}{n}{\displaystyle \sum _{i=1}^n\log \frac{e^{s\cos \left({\theta }_{Y_i},i\right)}}{{\displaystyle \sum _{j\ne Y_i}{e}^{s\cos \left({\theta }_j,i\right)}}}}$ LMCL损失函数 $-\frac{1}{n}{\displaystyle \sum _{i=1}^n\log \frac{e^{s\left(\cos \left({\theta }_{Y_i},i\right)-m\right)}}{e^{s\left(\cos \left({\theta }_{Y_i},i\right)-m\right)}+{\displaystyle \sum _{j=1,j\ne Y_i}{e}^{s\cos \left({\theta }_j,i\right)}}}}$ L-softmax损失函数 $-\frac{1}{n}{\displaystyle \sum _{i=1}^n\log \frac{e^{\Vert W_{Y_i}\Vert x_i\Vert \psi \left({\theta }_{Y_i}\right)}}{e^{\Vert W_{Y_i}\Vert x_i\Vert \psi \cos \left({\theta }_{Y_i}\right)}+{\displaystyle \sum _{j\ne Y_i}{e}^{\Vert W_j\Vert x_i\Vert \cos \left({\theta }_j\right)}}}}$ A-softmax损失函数 $-\frac{1}{n}{\displaystyle \sum _{i=1}^n\log \frac{e^{\Vert W_{Y_i}\Vert x_i\Vert \psi \left({\theta }_{Y_i}\right)}}{e^{\Vert x_i\Vert \psi \left({\theta }_{y_i},i\right)}+{\displaystyle \sum _{j=1,j\ne Y_i}{e}^{\Vert x_i\Vert \cos \left({\theta }_{j,i}\right)}}}}$ AM-softmax损失函数 $-\frac{1}{n}{\displaystyle \sum _{i=1}^n\log \frac{e^{s\left(W_{Y_i}^T{f}_i-m\right)}}{e^{s\left(W_{Y_i}^T{f}_i-m\right)}+{\displaystyle \sum _{j=1,j\ne Y_i}^c{e}^{s{W}_j^T{f}_i}}}}$ 正则化softmax损失函数 $-\frac{1}{n}{\displaystyle \sum _{i=1}^n\log \frac{e^{f_{Y_i}}}{{\displaystyle \sum _{j=1}^c{e}^{f_i}}}}+\lambda R\left(W\right)$