If you have played with modern AI recently, you know it has become remarkably good at prediction. LLMs such as Claude, Gemini, GPT, and Kimi can predict the next token well enough to write a fluent essay. Diffusion models can reconstruct pixels to generate photorealistic images.
It feels fresh—something we would not have imagined ten years ago. But underneath the hood is a fundamental problem: fluency is not the same as understanding.
Predicting surface-level details, such as the exact word or the exact colour of a pixel, does not require an AI system to grasp why things happen.
A ball rolls down a hill because of gravity, not because “pixel 42 changed to brown.” A model may see that change as a pixel transition; people see behaviour induced by physics.
If the next goal is to build AI that understands how the world works, plans for the future, and reasons like a human, predicting raw data is not enough.
We need something else.
Enter JEPA—Joint Embedding Predictive Architecture—Yann LeCun’s vision for the next generation of AI. JEPA bets that the future is not in predicting the next word or pixel, but in predicting abstract concepts.
The self-supervised learning trap
To understand why JEPA matters, we first need to look at how computer-vision models currently learn without human labels. The industry largely relies on two approaches, and both have important limitations.
1. The invariance approach (SimCLR and MoCo)
Imagine showing someone a picture of a cat, then showing the same image cropped, flipped upside down, and turned black and white. You would still conclude that it is the same cat.
Invariance-based models are trained to look at these distorted views and mathematically conclude: “Yes, these are the same thing.”
This teaches the model high-level semantics—it learns what a cat or car is. But there is a catch: engineers have to hand-craft the data augmentations. That introduces human bias, which can create problems when the model is used for completely different downstream tasks.
2. The generative approach (masked autoencoders)
The second approach hides much of an image and forces the AI to recreate the missing pixels precisely.
This removes the human bias, but it creates a massive efficiency problem. The model spends enormous compute trying to guess the exact texture of a blade of grass in the background. It is predicting in pixel space, which is too low-level to teach the model abstract concepts.
So how do we get the high-level semantic understanding of the first approach without its human bias—and without spending compute on pixel-perfect reconstruction?
JEPA’s core insight: predict representations, not pixels
This is the problem JEPA solves.
Its core idea is simple: from a single context block, predict the representation of various target blocks from the same image.
Instead of predicting raw pixels, JEPA predicts abstract features. It deliberately eliminates unnecessary pixel-level detail.
Think of looking out a window. If a car passes behind a tree, you do not need to predict the exact microscopic pixels of its bumper to know the car still exists and is moving forward. Your brain works with abstract representations. JEPA trains AI to do the same.
How the architecture works
The predictor is a narrow Vision Transformer that takes the context encoder’s output and, conditioned on positional tokens, predicts the representation of a target block at a specific location.
JEPA uses an internal setup designed to avoid the pitfalls of generative models:
- The context encoder looks at a visible chunk of an image and processes it into an abstract embedding—a mathematical summary.
- The target encoder processes hidden blocks from the same image into their own abstract embeddings.
- The predictor takes the context embedding and tries to predict the target embeddings.
Notice what is missing: at no point does the network try to draw the missing image. The loss function does not compare pixels; it compares abstract mathematical representations.
By predicting multiple target blocks from a single context block, JEPA is forced to learn spatial and semantic relationships in the data without relying on hand-crafted image transformations.
Why this matters for the future of AI
Why should developers care about this shift from pixel space to representation space? Because it unlocks capabilities that current token predictors struggle with.
When a model such as I-JEPA is trained for images, it learns rich semantic features that are highly adaptable. Scale the idea to V-JEPA for video and the model can look at a few frames, then predict abstract representations of future frames.
Because it is not bogged down rendering the exact lighting of future frames, it can learn the physics of the scene. That is a meaningful step toward robotics systems that can plan physical actions in the real world.
The bottom line
We are reaching the ceiling of what token prediction and pixel reconstruction can achieve. To move from sophisticated parrots to systems with common sense, AI needs world models.
JEPA shows that teaching an AI about the world is not about forcing it to memorise every detail. It is about helping it grasp the big picture.