Microphone input in Unity3DRecently I was asked if it was possible to spawn objects in Unity3D based on audio input from a bass guitar. My answer was along the lines of “yeah, why not?” and so I was tasked to do it. I already knew, that there is microphone class in Unity and there is even usage example in the scripting reference. Now, in order to get what I wanted, there was a need to monitor the mic input continuously. There’s several topics on Unity Community (e.g. here, here and here)about the microphone input, but I haven’t found a proper tutorial that explains how the microphone input can be utilized, I decided to write this post.

The Microphone Class

Unity3D has since version 3.5 provided a class for capturing sound from microphone to an audio clip.  The class itself provides basic methods for starting and stopping the capture and it can also list the connected input devices. For starting a capture on the default audio input device set on your computer, the most simple use case using C# as our scripting language would be like this:

using UnityEngine;
using System.Collections;

public class MicrophoneInput : MonoBehaviour {
    void Start() {
        audio.clip = Microphone.Start(null, true, 10, 44100);
    }
}

What this does, is start recording from default audio device (hence the null passed as a first parameter) for 10 seconds at sample rate of 44100Hz and keeps on recording by looping back to the start of the audio clip (the second parameter set to true controls the loop) after the recording time set to 10 seconds (the third parameter). But this you could already find out from the script reference. Now this is all nice, but how do you actually use it? The answer comes from the AudioSource class.

Getting the data to something more useful

Let’s extend the script from the first part. What we’ll do first is make sure that AudioSource gets included to the object that we assigning our script by adding line

[RequireComponent(typeof(AudioSource))]

on top of our class. Next, we’ll set up some attributes from the audio clip itself and wait for the recording to start before playing the recorded clip itself. We’ll add following lines to the Start -function:

audio.loop = true; // Set the AudioClip to loop
audio.mute = true; // Mute the sound, we don't want the player to hear it
while (!(Microphone.GetPosition(AudioInputDevice) > 0)){} // Wait until the recording has started
audio.Play(); // Play the audio source!

Now let’s create a new function called GetAveragedVolume that we use to get a meaningful value for the volume of the signal that is coming from the microphone. We need to take absolute values for the volume and I myself like to average the value from several samples in order to eliminate unnecessary pops and smoothen other sudden transients from the signal. What we’ll do first, is get the output data from the AudioSource to an array of arbitrary size, I chose 256 values this time, by calling GetOutputData -function.

float GetAveragedVolume()
{ 
    float[] data = new float[256];
    audio.GetOutputData(data,0);
}

Next step is to go through that array and calculate the average of absolute values contained in the aforementioned array by utilizing Mathf.Abs -function and some basic mathematics. You all know how to calculate an average value from a set of values, don’t you? Anyway, the function in it’s final form is as follows.

float GetAveragedVolume()
{ 
    float[] data = new float[256];
    float a = 0;
    audio.GetOutputData(data,0);
    foreach(float s in data)
    {
        a += Mathf.Abs(s);
    }
    return a/256;
}

Now that we’ve got that done, let’s add a couple of global variables to the class itself. Other will be used for tweaking the output value and the other is used to store the average volume for further use. I’ll make it global so that it can be accessed from other scripts too. Add the following lines before the Start -function.

public float sensitivity = 100;
public float loudness = 0;

We’re almost done! All we have left is the update loop, which we use to call our averaging function and set our loudness -variable. We’ll do that by calling the averaging function and multiplying it’s result by our sensitivity -variable. The update loop should look something like this:

void Update(){
    loudness = GetAveragedVolume() * sensitivity;
}

And there you have it. Now we get a somewhat meaningful value for the microphone input and can use it to spawn objects or change color or whatever comes to mind. Next step is to make an empty object in Unity and attach the script we’ve made to it – I’ve added the full script to the end of this post for a reference. Now, let’s see a quick example of spawning objects depending on our recently acquired loudness value.

Spawned by sound

As I said in the beginning, want to spawn an object to a spawn point when the loudness value goes over certain threshold. Let’s make a script for that called SpawnByLoudness. First, we’ll set up three global variables that we’re going to use for linking our microphone listening object to our spawn point, define the actual prefab object that we want to spawn and the threshold we must overcome in order the object to spawn. We’ll also use one private variable for storing our The script will start as follows:

using UnityEngine;
using System.Collections;

public class SpawnByLoudness : MonoBehaviour {
    public GameObject audioInputObject;
    public float threshold = 1.0f;
    public GameObject objectToSpawn;
    MicrophoneInput micIn;

    void Start() {

    }

    void Update() {
    }

}

In the Start -function, we will check that all necessary parameters are set and initialized properly. First, we’ll add an error message if the prefab is not set.

if (objectToSpawn == null)
    Debug.LogError("You need to set a prefab to Object To Spawn -parameter in the editor!");

Next, we’ll get access to our audio input object by checking if it is set and if not,  we’ll do a search for it (I’ve named my object as MicMonitor). Then we will grab hold of our previously created MicrophoneInput script component.

if (audioInputObject == null)
    audioInputObject = GameObject.Find("MicMonitor");
micIn = (MicrophoneInput) audioInputObject.GetComponent("MicrophoneInput");

And that completes our Start -function. In the Update -function, our goal is to monitor the loudness parameter and compare it to threshold. If our criteria is met, we’ll spawn an object and scale it by adding our loudness value to it’s scale on every axis. The update function that achieves this could be something like this:

void Update () {
    float l = micIn.loudness;
    if (l > threshold)
    {
        Vector3 scale = new Vector3(l,l,l);
        GameObject newObject = (GameObject)Instantiate(objectToSpawn, this.transform.position, Quaternion.identity);
        newObject.transform.localScale += scale;
    }
}

To put this all together and try it out, create an empty object in Unity and attach the script you just created to it. Then make a prefab out of the object you wish to spawn and drop it to the script’s objectToSpawn -parameter in the editor. You should now have the basic knowledge about how to utilize the microphone input class in Unity3D and you are ready to venture deeper into audio analysis. For the next post in this topic, I will deal with spectrum analysis and do a short tutorial about how to find out what frequencies the sound coming from the microphone has. I hope this post was useful to you and if you have any questions feel free to drop me a line.

Oh and I almost forgot, here are the complete scripts!

MicrophoneInput.cs

using UnityEngine;
using System.Collections;

[RequireComponent(typeof(AudioSource))]
public class MicrophoneInput : MonoBehaviour {
    public float sensitivity = 100;
    public float loudness = 0;

    void Start() {
        audio.clip = Microphone.Start(null, true, 10, 44100);
        audio.loop = true; // Set the AudioClip to loop
        audio.mute = true; // Mute the sound, we don't want the player to hear it
        while (!(Microphone.GetPosition(AudioInputDevice) > 0)){} // Wait until the recording has started
        audio.Play(); // Play the audio source!
    }

    void Update(){
        loudness = GetAveragedVolume() * sensitivity;
    }

float GetAveragedVolume()
    { 
        float[] data = new float[256];
        float a = 0;
        audio.GetOutputData(data,0);
        foreach(float s in data)
        {
            a += Mathf.Abs(s);
        }
        return a/256;
    }
}

SpawnByLoudness.cs

using UnityEngine;
using System.Collections;

public class SpawnByLoudness : MonoBehaviour {

    public GameObject audioInputObject;
    public float threshold = 1.0f;
    public GameObject objectToSpawn;
    MicrophoneInput micIn;
    void Start() {
        if (objectToSpawn == null)
            Debug.LogError("You need to set a prefab to Object To Spawn -parameter in the editor!");
        if (audioInputObject == null)
            audioInputObject = GameObject.Find("AudioInputObject");
        micIn = (MicrophoneInput) audioInputObject.GetComponent("MicrophoneInput");
    }

    void Update () {
        float l = micIn.loudness;
        if (l > threshold)
        {
            Vector3 scale = new Vector3(l,l,l);
            GameObject newObject = (GameObject)Instantiate(objectToSpawn, this.transform.position, Quaternion.identity);
            newObject.transform.localScale += scale;
        }
   }
}