📶Audio Spectrum

Prepare an audio source for sampling

Processes audio data to provide sampling data to Audio Sample behaviors. This works with audio files and live audio input devices supported on the target device.

When adding a new instance of Audio Spectrum, select the game object with the Audio Source you wish to sample, or if nothing is selected then it is applied to the main Audio Track object.

To create a new Audio Spectrum, select from the main menu:

Tools > Timeflow > Add > Audio Spectrum

For efficiency, only 1 Audio Spectrum is needed for each audio source. Sampling is based on the built-in Unity methods GetSpectrumData for frequency analysis, and GetOutputData.

Audio Spectrum does not display channels in the Timeflow view unless Volume or Pitch are enabled.

Audio Spectrum Editor

Select the object to view its settings in the Inspector.

For common settings, see Menu Bar.

This behavior does not support Update Settings since it doesn't relate to audio.

Audio Input

Audio Source

Analyzes the output from an Audio Source component. This is the most common setup using an audio clip asset as the source.

Audio Listener

Analyze audio of the entire scene using the Audio Listener.

Use Audio Listener mode for general audio-reactive behaviors that responds to all audio no matter where it originates in the scene.

Device

Analyze live captured audio from an input device. Clicking on the button displays a list of all available devices, which may be a microphone, web camera, or any device supported for audio input by Unity.

The warning icon appears if the specified device cannot be found (i.e. it is not connected)

Preventing Audio Feedback

To prevent audio feedback (ie. the microphone picking up the output audio in a perpetual loop causing high pitched sounds) it is recommended to use an Audio Mixer to route the Audio Source to a muted channel. If the Audio Source is muted directly, it stops audio processing and will not produce the reactive effects. So by routing it instead to a muted mixer channel, the audio processing can still occur but is prevented from being output. This set up is not an issue if the user is using headphones, so may be an optional configuration in your game or app.

Fallback

If enabled, the 1st device found is used if the device by the specified name isn't found.

When deploying to mobile, the built-in microphone is the default input device.

Device Permissions

Audio Spectrum automatically requests permission through the built-in API: Application.RequestUserAuthorization(UserAuthorization.Microphone)

If building for a device, additional configurations such as a manifest may be required to register audio listening permissions.

This only applies when using Device for audio input and isn't needed for Audio Source or Audio Listener input.

Display Device Name

Use this field to optionally assign a Text Mesh Pro object to display the name of the input device onscreen. This may be helpful for end users to see the current device in use.

Device Timeout

How long to wait (in seconds) for the device to initialize before timing out.

When an audio input device is working properly it should start up immediately, however there could be some latency causing it to take a moment. If it takes longer than the Timeout, then it is assumed the device is inactive or unavailable and audio listening is aborted. When this occurs, a different input device would need to be selected, or adjust the Timeout settings if needed.

Channel

Select between Left, Right, or Stereo audio analysis. When Stereo is selected, the left and right channels are averaged together.

In most cases a single audio channel (Left) is sufficient for audio reactive effects, however both channels may be used either averaged or separately with multiple Audio Spectrum instances to create different behaviors for left and right audio.

Sampler Rate

Enter the sample rate to match the audio source. This can be found by selecting the audio file and viewing it in the Inspector preview panel.

The sample rate is 44100

If the sample rate does not match the source, it may not operate properly.

When using an audio device for input, the sample rate is automatically reduced to match the device specifications. Lower quality microphones may not support higher sample rates and will produce errors if the rate is unsupported.

If you don't know the sample rate of the target device or it varies, set a value of 48000 and allow it to automatically adjust to the highest sample rate supported by the device.

Spectrum Window

Sets the quality level for analyzing audio frequencies, listed in order from simple to complex.

The default option is BlackmanHarris which is the highest quality, though if performance is a concern, a lower setting may be selected to optimize performance.

For more information, see the Unity documentation for FFTWindow.

Resolution

Each snapshot of audio represents the full range of frequencies, approximately from 20Hz to 20000Hz. The Resolution determines how many samples across the frequency range are made. The larger the value, the higher quality the sample result, though at the cost of performance.

A power of 2 value is enforced on this field with the default value of 128. This also affects the display resolution of the graph, if enabled.

Volume

Enable this to expose the Volume channel in the Timeflow view, outputting the current detected volume in the audio sampled.

Raw

If enabled, the RMS values are output directly without any processing.

RMS values are logarithmic ranging from 0 to 1.

RMS Reference

Divides the RMS value to calculate decibels (dB). A value of 0.1 is 0dB

For volume levels in decibels (dB) disable Raw and enter a RMS reference value. Decibels are calculated using the following equation:

dB = RMS / Reference

RMS and dB are advanced audio topics not covered in detail here. However,

understanding these parameters is not required for most use cases and the default values may be used.

When using the Volume channel with Channel Link, be aware the Time Offset isn't used and volume levels are always at the current time based on audio detection.

Frequency

Enable to detect the dominant pitch and output the value in a separate channel in the Timeflow view.

Min Threshold

Sets the minimum level of RMS change to detect pitch. This essentially adjusts the sensitivity of pitch detection.

Pitch detection by nature can be unpredictable and inaccurate. While the human ear is able to hear pitch and filter out noise, an FFT computation on the other hand isn't as sophisticated and produces imperfect results.

One technique for smoothing out results is to use Channel Link.

Graph in Game View

Enable to display a preview of the frequency spectrum in the game view.

This is an editor-only feature and only draws over the Game View while audio is playing. Use this to visualize audio sampling and to help identify active ranges of frequency.

The frequency spectrum drawn in the Game View.

Scale

Multiplies the spectrum values to increase the scale vertically.

Frequency Range

Use the range slider to limit the frequency range displayed. This can be helpful when setting up an Audio Sample to find the right range for audio-reactive effects.

Threshold

Set this above 0 to only show data spikes above the threshold volume level (RMS).