crystal vibes

feat. ott

Now available on Steam VR!

 

 
 
 
 

Download now on SteamVR here!

Experience candy-colored psychedelic sound rippling through an endless crystal universe. Crystal Vibes utilizes the cutting edge of spatial 3D audio and sound visualization that maps sound and light based on the science of the human senses, to push the frontiers of technology-mediated sensory experience in virtual reality. With the project's predecessor described as “transcendent” and “like traveling through a psychedelic kaleidoscope” (Forbes 2016), this piece ups the ante with music from producer Ott.

Exhibited at Sundance Film Festival New Frontier in January 2017, at Kaleidoscope Showcase Vol. 2 in 11 cities across the world, at E3 (Electronic Entertainment Expo) at the Women In Games International E3 Party hosted by Two Bit Circus, and hand selected as a Vimeo Staff Pick!

Interested in the development, Synesthesia Suit haptic design and user feedback of Crystal Vibes?  Read this paper presented at the International Conference on Virtual Systems and Multimedia (VSMM).

 
 

4K Sterescopic 3D 360 degree version.

 

 
crystalvibes_3b.png
OttSundance01_2016-12-20_19-59-18-081.png

synesthesia suit

Crystal Vibes is a collaboration with the team behind the Synesthesia Suit, including Tetsuya Mizuguchi.  The suit was designed for the new release of the cult classic, Rez Infinite, available now on PlayStation 4 and PSVR, and features high-fidelity vibrators located over the entire body.  Crystal vibes is not just for the ears and eyes, but for the whole body through vibration.

with music from critically acclaimed psychedelic dub producer Ott's latest album

Ott's track Harwell Dekatron is my favourite on his latest album, "Fair Children".  By visualizing the drums, vocals and other stems separately, the virtual environment powerfully responds to the sound's different parts.

OttSundance01_2016-12-20_19-59-46-621.png
OttSundance01_2016-12-20_19-58-56-801.png

Infinite lattice

Using some clever rendering techniques, a virtually endless field of particles allows for a virtual 3D display for visualized sound to flow through.

Earthy colors and beautiful symmetry

The world is designed to reflect the complexity and intimacy of our audial and visual senses.

 
Crystal vibes is a multi-crossed-sensory universe. It essentially asks the question, if a sound spectrum was actually a light spectrum, what colour would that appear to the human eye? In order to answer that question, it has to take account of the fact that humans are sensitive to sound frequency on a logarithmic scale, whereas with light, we are sensitive on a more linear scale. So we map the log-frequency sound spectrum to an effective visible spectrum. Then, we take account of the sensitivity of the three types of light sensitive retinal cells in the human eye, which are empirically determined psycho-biological functions that were measured in the 1930s, and multiply them by our effective light spectra to determine what are called tristimulus XYZ values, which can be simply converted to RGB values for display on whatever textures or geometries we would like.

Crystal Vibes does not use any 3D modelling, 2D design, or hand animation. Instead, everything you see is procedurally generated by equations and code, such as sin-waves, bezier curves, Bravais lattice structures, geodesic forms and Fourier transforms. I plagiarise nature’s innate beauty in 3D fractaline crystal structures, and leverage the artistry of rhythm and form in the music for visual beauty.

Crystal Vibes also communicates with the Synesthesia Suit, originally designed to enhance the game Rez Infinite. By tracking the user’s head and hands, we can apply high-fidelity broadband vibrations to the correct spatial locations on the user’s body related to where they are being touched by the sound-driven geometries. The result is a hyper-connected audio-visual-vibrotactile universe.

The crystal lattice section itself appears to contain over 4 million spheres, which the sound visualisation ripples through. The computer itself would be unable to render this level of fractal detail at the 100 frames per second required for virtual reality, and so several unique tricks have been employed. In fact, only 64 spheres, and 22000 sphere-looking particles are rendered, with the remaining visible particles being pre-rendered and presented as a 4K 360 degree video in the distant background. An occlusion algorithm was developed such that particles and spheres most likely to be visible to the user from their perspective are rendered. The result is a seamless, infinite-looking field of spheres that the user can interact with. A couple of other tricks are used to prevent the background from mismatching with the foreground rendered particles as well.

Crystal Vibes is currently not available to download, but may be available on a variety of platforms in the not-to-distant future. I am currently preparing a 360 video of the experience.
 

These are direct screen shots from Crystal Vibes as presented at Sundance Film Festival, New Frontier 2017.

Soon released on Steam! https://store.steampowered.com/app/623040/Crystal_Vibes_feat_Ott/