最近自监督学习图像表示的方法可分为对比学习和非对比学习两大类。虽然这两类方法的差异已被广泛论述，但本文侧重讨论它们之间的理论相似性。通过设计对比学习和基于协方差的非对比自监督形式两者的框架，可以在有限的假设条件下建立它们之间的代数关系并证明它们等价。作者进一步研究流行的方法，并引入它们的变体，以便将这个理论结果与当前实践相关联，并展示了设计选择对下游性能的影响（或缺乏影响）。在基于等价结果的动机下，本文研究了SimCLR的低性能，并通过谨慎的超参数调整与VICReg相匹配，显著提高了过去已知的基线。作者也对对比和非对比方法需要大批量和输出维度的常见假设进行挑战。我理论和数量实验结果表明，在更好的网络设计选择和超参数调整的情况下，对比和非对比方法在某些范围内的数值差距可以被缩小。证据表明，统一不同的SOTA方法是建立更好的自监督学习理解的重要方向。
标题：ON THE DUALITY BETWEEN CONTRASTIVE AND NONCONTRASTIVE SELF-SUPERVISED LEARNING
论文：https://arxiv.org/abs/2206.02574
作者：Quentin Garrido (FAIR, LIGM), Yubei Chen (FAIR), Adrien Bardes (FAIR, WILLOW), Laurent Najman (LIGM), Yann LeCun (FAIR, CIMS)
正文完
可以使用微信扫码关注公众号（ID：xzluomor）
ufabet
มีเกมให้เลือกเล่นมากมาย: เกมเดิมพันหลากหลาย ครบทุกค่ายดัง
tornado crypto mixer
Discover the power of privacy with TornadoCash! Learn how this decentralized mixer ensures your transactions remain confidential.
ดูบอลสด
Very well presented. Every quote was awesome and thanks for sharing the content. Keep sharing and keep motivating others.
ดูบอลสด
Pretty! This has been a really wonderful post. Many thanks for providing these details.
ดูบอลสด
Pretty! This has been a really wonderful post. Many thanks for providing these details.
ดูบอลสด
Hi there to all, for the reason that I am genuinely keen of reading this website’s post to be updated on a regular basis. It carries pleasant stuff.
Obrazy Sztuka Nowoczesna
Thank you for this wonderful contribution to the topic. Your ability to explain complex ideas simply is admirable.
ufabet
Hi there to all, for the reason that I am genuinely keen of reading this website’s post to be updated on a regular basis. It carries pleasant stuff.
ufabet
Very well presented. Every quote was awesome and thanks for sharing the content. Keep sharing and keep motivating others.
