[2026-1] 정유림 - A Simple Framework for Contrastive Learning of Visual Representations

• Title: A Simple Framework for Contrastive Learning of Visual Representations
• Citations: 33,428 (as of July 11, 2026)
• Link: https://arxiv.org/pdf/2002.05709
• ICML 2020

---

Previous approach to learn visual representation

1. Generative approaches
   • A generative method attempts to model or reconstruct the input data.
   • SimCLR paper says that pixel-generation can be computationally expensive and may not be necessary for representation learning.
2. Discriminative approaches
   • Discriminative approaches trains the network to distinguish among alternatives or predict a target.
   • SimCLR is called discriminative because it distinguishes the correct pair from incorrect pairs. It does not generate image pixels.

---

Main Results (ImageNet)

1. achieves 76.5% top-1 accuracy
   • 7% relative improvement over previous SOTA (the performance of a supervised ResNet-50)
2. achieve 85.8% top-5 accuracy
   • fine-tuned on only 1% of the labels.
   • outperforming AlexNet with 100× fewer labels

Figure 1 Results

1. SimCLR outperformed earlier self-supervised methods
2. Self-supervised learning benefits strongly from model scaling
3. Strong self-supervised representations can approach supervised performance

• Blue circles: previously published self-supervised methods
• Red stars: SimCLR models
  • The “2×” and “4×” refer mainly to network width, not directly to parameter count.
• Gray cross: standard supervised ResNet-50, trained using ImageNet class labels.

---

SimCLR : self-supervised contrastive learning (training target is constructed automatically from the data)

 

---

Importance of SimCLR

Data augmention Results

“composition of data augmentation plays a critical role in defining effective predictive tasks”

→ A good augmentation should preserve semantic identity while removing easy shortcuts.

Figure 5: Which augmentations produce useful representations?

The values are ImageNet linear-evaluation accuracies.

How to read it

• Diagonal cells: only one augmentation
• Off-diagonal cells: two augmentations applied sequentially
• Last column: average performance for that row

For example:

• Crop only = 33.1 (this can be too easy)
• Crop + Color = 56.3

→ Single transformations perform poorly

Figure 6: Evidence for the color shortcut

Figure 6 shows pixel-intensity histograms for different crops.

• (a) Without color distortion → color can match positive crops. without recognizing the actual semantic content
  • shapes are fairly similar.
    • why? still contain similar colors
• (b) with color distortion → it have to use more stable features. (such as: object shape, body structure,.. )
  • Even though all four crops still came from the same dog image, their histograms now look very different.
    • the model can no longer use the color histogram as a reliable fingerprint.

So, it can be said “Harder contrastive task can produce a better representation”

Table 1. Stronger data augmentation improves performance in SimCLR (not in Supervised)

 

How much information about the applied image transformation remains in h and in g(h)?

-h: output of the main encoder

-g(h): output of the projection head, used for the contrastive loss

• Random guess: accuracy expected without useful information. baseline
• Representation information: h>g(h)

→ SimCLR design: use hfor downstream tasks.

---

SimCLR Summary

• Goal: Learn useful image representations without class labels.
• Method: Create two augmented views of the same image, pull their embeddings together, and push embeddings of other images apart using NT-Xent loss.
• Key finding: Strong augmentation—especially random crop plus color distortion—and a nonlinear projection head are critical for good performance.
• Significance: SimCLR showed that a simple contrastive framework can learn representations competitive with supervised pretraining.