Scientists are starting to figure out why these GIFs are so damn loud.
Can you “hear” visual motion? If so, you’re hardly alone. There’s a huge Reddit community of 114,000-plus people who post and discuss silent videos that have the illusion of making noise.
And now a new study in the journal Cortex finds around 20 to 30 percent of people will experience an auditory sensation when they view moving, energetic — but silent — GIFs like this:
Psychologist Chris Fassnidge, the lead author of the Cortex study, calls this weirdly common phenomenon vEAR, or “visually evoked auditory response.” He and his co-authors believe it may be a new form of synesthesia, the rare neurological phenomenon wherein different sensory experiences are connected.
Even if it’s not precisely synesthesia, vEAR may be a window onto a better way to understand how all our senses are complexly connected. It’s possible that just like we combine taste and smell to get a complete experience of flavor, we may in part hear with our eyes.
“The more we learn about the brain,” says Fassnidge, who recently completed his PhD at City University of London, “the more we learn it’s a very multisensory organ and that the senses can influence one another.”
What is synesthesia?
In people with synesthesia, activation of one sense triggers perception of another. The classic example is people who see a different color for different letters of the alphabet or for different numbers. In others, sounds will take on color.
There’s a form of synesthesia where certain sounds produce physical sensations on the skin and in the body. It’s believed these crossovers happen because neurons from one area of the brain are highly connected to another, or because the connections between the brain areas are easily triggered. And it’s pretty rare: Around 3 percent of people have these forms of synesthesia.
Similarly, vEAR could be a crossover of the visual and auditory systems of the brain. And the experience of it “varies from person to person,” Fassnidge says. “Some people describe it as a buzzing sound in their head. For other people, it’s kind of like a white noise. And then other people say it varies depending on what it is they are looking at.”
There are only a handful of studies on vEAR; the first one was published in 2008. Though the work is early, the published papers on vEAR suggest it’s a common phenomenon, with around 20 to 30 percent of people reporting hearing silent images.
“A lot of people don’t realize they have this thing until you start testing for it in the laboratory,” Fassnidge says. In the study in Cortex, the scientists had participants view 24 GIFs and rate if they noticed an auditory sensation on a scale of 0 to 5. Those who rated more than half of the GIFs at a level of 3 or above were considered to have experienced vEAR.
We may not think about the sound visual motion makes because sound and motion co-occur all the time. “Maybe because they co-occur so frequently you either aren’t aware of the mental sound until you strip away everything else,” Fassnidge says.
In the GIF above depicting the transmission towers jumping rope, I noticed a slight thudding sound. At first, I thought I was just noticing my heartbeat. But then I counted: The power line is jumping at 52 beats per minute; my heart is going at 72. “It would be odd that your heartbeat would be syncing with what you are seeing so closely,” he says. For me, the experience is weirder than the normal experience of hearing. It’s somewhere at the hazy intersection between a real sensation and imagination. (I don’t hear anything with the other GIFs.)
The rising popularity of GIFs (which are really just silent slices of film) seems to have made many people aware of the sensation for the first time. The Reddit forum /r/noisygifs is filled with people posting and discussing the GIFs that fill their heads with sounds. It’s part of a growing trend: The internet is proving to be an intriguing space for people to gather and discuss the peculiarities of human perception. There are millions of people who watch YouTube ASMR videos to experience a mysterious tingling sensation. In the past, these people perhaps would have just kept their unusual experience to themselves. Today, they can share.
How do I know if I have vEAR?
You can test yourself for vEAR with this survey here. It’s similar to what Fassnidge and his PhD adviser Elliot Freeman used in their new study in Cortex. The survey shows you GIFs, and all you have to do is rate how much of an auditory sensation each one elicits.
With data from more than 4,100 participants (some volunteer, some paid; it’s not a representative sample), the pair found that the experience of vEAR is very common. But they also found certain moving images provoke the experience more than others.
GIFs that predicted sounds (like the power line towers jumping rope above) were more likely to produce vEAR. So were GIFs that had a higher intensity of motion, like this glittering disco ball (Freeman says in an email this GIF produced one of the strongest responses in the survey).
People who report getting pieces of music stuck in their heads often (a.k.a. earworms) are also more likely to experience vEAR. “It might be that their auditory cortex is more excitable,; it doesn’t stop working,” Fassnidge says. It’s also more common in people who have other forms of synesthesia, suggesting something similar is going on.
Why would people “hear” what they see?
As with any new field of scientific study, there are a few alternative hypotheses to explain vEAR.
Fassnidge admits that his survey may be “observing several different effects at once,” and that there are overlapping explanations for why a person might hear a silent GIF.
For some people, the experience might be a true synesthesia that they can’t turn off. Other people might just be highly suggestible and using their imaginations to hear the sounds (or people who sense something in the realm between noise and imagination). It’s also possible that some of the people in the survey were just being agreeable and responding with what they thought the researchers wanted to hear.
Another part of the effect can be the result of learning: We’ve learned to expect certain sounds when we see certain objects in motion. We expect a ball bouncing on a court to make a bouncy noise, and so when we see it happen, our imaginations fill in the gaps. But even when Fassnidge and Freeman strip GIFs of all real-world connotations, and just show participants simple blinking lights, around 20 percent of study participants will say they hear something.
The follow GIF mimics the vibration pattern in the jumping power cable GIF but has no real-world images. Do you still hear anything? I don’t.
In another study of people who self-reported experiencing vEAR, Fassnidge and Freeman played faint audio while the participants watched flashes of light. The flashes of light, they thought, would be the simplest visual cue to provoke the vEAR, and the least weighed down with connections to things that make noise in the real world. “The people who reported being able to hear the visual events were more inclined to miss the auditory sound if it coincided with the flash,” Fassnidge says. “The imagined sound masks the real sound.” That makes them suspect vEAR is indeed working on the auditory system in the brain.
Is this really synesthesia?
There are a few clues that vEAR may not be exactly the same thing as other types of synesthesia. Other types tend to have very specific pairings, like the letter A being linked with the color red. “It’s hard to get that level of specificity with the auditory sensations,” Fassnidge says. More work needs to be done to try to pin down the phenomenon and trace how it works in the brain.
But at the very least, the phenomenon does illustrate how deeply the areas of the brain are connected.
Consider how blind people can achieve a certain sense of vision restored with special electrodes that attach to their tongue. These sensory substitution devices, as they are called, translate the visual world into small zaps of electricity users can feel on a grid placed on their tongue. Studies suggest these devices actually stimulate the visual areas of the brain. “It’s not biological vision as we know it, but it’s visual information conveyed to the visual cortex,” Fassnidge says.
Similarly, if electrical stimulation from the eyes happens to stimulate the auditory center of our brains, “it becomes more a philosophical question: Is that hearing?” Fassnidge says. Maybe.
And the research could perhaps help us explain why art forms that work upon many of our senses, like dance, are so appealing. In dance, music is timed with motion. “Why is that so pleasing to us as a species? is it because it is ticking all these neurological boxes?” Fassnidge wonders.
There are so many little curiosities about how we perceive the world. I’ve sworn that my eyes see slightly different variations of color. There are people who claim to have no internal eye, no ability to imagine physical objects that aren’t in front of them. Other people have trouble memorizing faces. Perception is our interface with the world, and the peculiarities of it make for fascinating research.
“Everyone has a strange thing about how they perceive the world,” Fassnidge says. “It gets people engaged in a way that a lot of other neuroscience topics don’t. It fires up their imaginations.”