Controls
- Drag - orbit the brain
- Scroll - zoom in and out
- Predicted - model-estimated activity, colored live while the stimulus plays
- Normal / Inflated - folded surface vs its inflated version
- Open / Close - split the hemispheres apart to peek between them
- True / Compare - need a real fMRI recording of this stimulus; enabled only when ground truth exists
Pick a clip below - the brain on the left reacts as it plays.
Predictions from an open brain-encoding model. Warm colors mean more activity, gray means less. No scanner involved.
Three stages, from pixels to cortical activity.
- Encode the stimulus. The video track is embedded by a self-supervised vision model and the audio by a speech model - what eyes and ears would pick up becomes a stream of vectors.
- Fuse in a transformer. A multimodal transformer integrates sight, sound, and language over time, learning the dynamics that drive brain activity.
- Map to the cortex. A decoder projects the fused representation onto 20,484 vertices of the fsaverage5 surface - one estimate per second.
Predictions are shifted 5 seconds back to compensate for the hemodynamic lag: blood oxygenation responds slowly, so the brain "shows" a stimulus a few seconds late.
Independent project, open model.
human brain is a personal visualization project. All predictions were generated with TRIBE v2, an open brain-encoding model by Meta AI Research (CC-BY-NC 4.0), running on a local GPU. Cortical meshes derive from the FreeSurfer fsaverage template. This site is not affiliated with Meta.