引用jph00 · 2017-05
“a good example would be Wasserstein GAN”
举它为例,说明理论能带来 GAN 训练的实际提升
WGAN:改用 Wasserstein 距离训练 GAN,更稳定且曲线可用于调试
论文提出 WGAN,作为传统 GAN 训练的替代:可以提高学习稳定性,消除模式崩溃等问题,并给出对调试和超参数搜索有意义的学习曲线。作者还证明相应的优化问题是合理的,并通过大量理论工作阐明它与其他分布间距离的深层联系。
评论者把它当作理论带来实际收益的例子,也说它对统计和概率基础要求较高,有人认为真正有用的是梯度惩罚技巧而不是新的损失函数。
“a good example would be Wasserstein GAN”
“The kind of math that would be necessary to understand papers like [1][2][3][4].”
“There is a paper that indicates Earth mover, aka Wasserstein-1 is better for convergence of GANs:”
“I would be very surprised if someone who did not know population variances and marginal probabilities would be able to follow the Wasserstein GAN paper. Just see for yourself: Wasserstein GAN”
“IMO the useful thing to take away from that paper is their 'gradient penalty' technique and not the new loss function.”