Dive into the Vibe!!
Introduction:
This is a personal project that I decided to make after I found the usage of a Spectrum Analyzer in Unity 5. The idea is to have a full fledged visualizer surrounding the player tuned to the music that is being played. The user can experience the throbbing beat completely in 3D in the form of cylindrical bars combined with varying flashes of light, thus giving a club like feel to the music.
Design:
The project is accomplished via one script that employs an AudioListener that listens to an AudioClip being played on the gameobject's AudioSource. This listener is fed to the SpectrumAnalyzer to extract the frequency response of the current instance of the music. The various frequencies are collected in bins with each frequency's amplitude. The amplitude is then multiplied by a gain factor to amplify the collective value and is connected to the height of a cylindrical prefab. The prefab, therefore gets a varying height at runtime, at each frame update, thus resulting in a visualizer component.
The script also carries out a second task of initializing multiple instances of the cylindrical prefab along the circumference of a circle with a public radius value and having its center at (0,0,0), where the VR camera is placed. The frequency response at one instance is divided into the same number of bins as the number of cylindrical prefabs, and each bin value multiplied with the gain is connected to a corresponding cylinder's height.
For visual purposes, the number of bins are spread out to cover only a half of the circumference; and the other half is mirrored, thus giving a much more beautiful effect.
Additionally, post processing is applied via changing the cylinder's main color at runtime, via HSV colorspace conversion and cycling the hue value, thus giving the effect of having a multitude of colors.
The final step is to apply Image Effects to the rendered frame in the form of a Bloom effect, with Lens Flares, that give the cylindrical bars a very flashy glow when the beat is in play.
Here is the VR music visualizer in full effect:
The highlighted area shows the components of the Music Visualizer script. Each component's significance is explained below. Note that each of these components is public so that they can be modified at runtime to test the responses.
1. Eq_col_r and Eq_col_l: These contain the cylindrical prefabs or "columns" for the right and left side, for getting the mirroring effect.
2. Bands: This an int value corresponding to the number of frequency bands to extract, and simultaneously, the number of columns to instantiate for the corresponding bins. The whole audio frequency range is divided into 32 parts in this particular run, thereby producing 32 cylinders on each half of the circle.
3. Samples: This is the sampling frequency, or speaking script wise, the number of samples collected from the audio at one instance. These frequency samples are divided into the 32 bins. For example, 4096/32 = 128, thus one bin contains the collective amplitude of 128 frequency values.
4. FFTw: This is the FFT window, which is used for converting the time domain audio signal to the frequency domain. The available options for conversion are,
Rectangular,
Trianglular,
Hanning,
Hamming,
Blackman and
Blackman Harris
5. Amplifier: This is the gain value that is multiplied to each bin of amplitudes. the effect is that, since the bins get amplified by this multiplier, the cylindrical columns bounce higher or are more responsive to the music.
6. Minclamp and Maxclamp: This is an additioanl feature that allows how much of the frequency response should be reflected to the visualizer. For instance, if you need a more bass response, the values are set to cover only the lower frequency range, i.e. upto a 100Hz. In this screengrab, the frequency range is set from the beginning to 60Hz.
7: Limitcol and Offsetval: These values correspond to the color changing of the cylinder's material. These are meant to change the visual appearance of the glow and the color; these are not related to the audio processing. Since Unity processes the material color in RGB, and I needed a continuous color sweep, I used a script snippet obtained from http://www.easyrgb.com/index.php?X=MATH&H=21#text21 which essentially converts RGB to HSV and back. Thus, at every frame, the script converts the material's RGB value to HSV, sweeps the hue via a certain degree, and converts it back to RGB, subsequently being applied as the new material color.
Screenshots: