Neural Harmony: When Art Meets Neuroscience
A new, interactive art installation on display at The New York Academy of Sciences explores language and the hierarchical relationship between brainwaves.
Published October 3, 2024
By Lois He, Vio Zhu, Sofia Shen, and Xing Zhang
NYU Interactive Telecommunications Program
Neural Harmony, an interactive installation developed in collaboration with Buzsaki Lab, and the NYU Neuroscience Institute, explores the concept of Neural Syntax, a theory by György Buzsáki that describes how different brainwave rhythms interact in a hierarchical structure similar to language. The installation transforms this complex neuroscience concept into an engaging and playful experience.
Brainwaves and Language: A Hierarchical Metaphor
Through the metaphor of language, the piece demonstrates how brain rhythms interact. Neural Syntax highlights the parallels between language and brainwave patterns. Just as letters form words and sentences, slower neuronal rhythms modulate faster ones, structuring how the brain processes information. This hierarchy organizes and structures neural activity, allowing for effective information processing.
Users can manipulate brainwave rhythms through a simple interface. By turning a knob, they control the speed and order of the rhythms. Slower waves align with sentences, middle waves with words, and the fastest with letters. This interaction allows users to see how these changes reorganize the text, revealing how brainwaves shape cognitive processes.
When the user turns the knob on the installation, letters rearrange within specific words. This mimics the faster neural oscillations nested within slower rhythms. Simultaneously, the according brainwaves adjust their frequency to synchronize with both the upper and lower layers.
Transforming Neuroscience into Interactive Art
Neural Harmony transforms the abstract concept of Neural Syntax into a tactile and interactive experience.
The device, constructed from wood blocks and acrylic sheets, uses software such as Phidgets, VS Code, Unreal Engine, and TouchDesigner to power the interactive components, allowing users to engage with the relationship between brain waves and text in real time.
The user adjusts the layers of brainwaves, creating a playful experience that helps them visualize the connection between neural rhythms and language. This interaction makes the complexities of Neural Syntax not only comprehensible but also engaging.
Conclusion: Understanding Neural Syntax Through Interaction
Neural Harmony bridges neuroscience and art, offering a dynamic way to explore the concept of Neural Syntax. By directly manipulating brainwave rhythms and text, the audience can demonstrate the connections between neural processes and language structure. Through tactile and engaging interaction, the project opens up new ways of understanding how our brains work.
By combining art and science, Neural Harmony serves as a model for how interactive installations can make complex ideas more accessible.
About the Artists in Their Own Words
Lois He
I’m a multimedia storyteller, transitioning from filmmaking to interactive media and AI. With a background in storyboarding, camera work, and scriptwriting, I’m now focused on creating narratives that invite active audience participation.
I’ve worked across diverse formats, from physical spaces using sound and sensors for interaction to virtual narratives blending film and gaming through XR.
Collaboration is key to my process. I’ve partnered with fashion designers, dancers, psychologists, and neuroscientists, adding new perspectives to the narratives I create.
I’m currently a second-year master’s student In NYU’s Interactive Telecommunications Program.
Portfolio website: https://www.heqiuyao.com
Vio Zhu
I’m a software engineer turned video jockey (VJ) and multimedia artist. As a VJ, I am passionate about merging natural aesthetics with digital visual production, creating visually compelling audio-reactive works. My art has been curated by LVMH and Public Visuals, and I have performed with DJ Rebolledo’s Japan tour.
As a multimedia artist, I collaborate with a wide range of creatives, from traditional Japanese musicians and techno DJs to club owners to neuroscience professors. My goal is to integrate scientific knowledge, evoke novel emotions, and inspire self-reflection through immersive, aesthetically driven interactive experiences.
I’m currently a second-year master’s student In NYU’s Interactive Telecommunications Program.
Portfolio website: https://viozhu.cargo.site/
Sofia Shen
I approach life through scattered feelings that are too complex to be spoken. Art became my way of making sense of these elusive emotions. For me, art is not merely a form of expression but a dialogue—one that connects both the personal and communal aspects of existence.
I began my creative journey as a fashion designer, previously worked on styling, creative direction, and have collaborated with various brands and influencers. I am currently exploring the potential of incorporating technical tools and sensors to create interactive experiences. I aim to redefine art as a medium that bridges thoughts, emotions, and experiences, making each piece a point of connection between herself and the world.
I’m currently a second-year master’s student In NYU’s Interactive Telecommunications Program.
Portfolio website: https://buymecheesecake.com/
Xing Zhang
I am a filmmaker and designer, currently at the Yale School of Art.
Also read:
