Audio Spectrum Analysis – Output to JSON File – Flash Tool
I have been unsuccessful so far in syncing animation drawn in Canvas using Processing js & audio played using the browser Audio tag with audio spectrum data I create from a ByteArray in Flash. It works in Flash due to the fact that the spectrum analysis and animation are executed together in an onEnterFrame event. The drawback to not having real time audio spectrum analysis from within the browser is the need to supply all the data to Processing js at initialization. It can be rather large and I have yet to perfect syncing the reading of the array with the current time of the audio. Some sort of progressive data stream and excellent timing system???
It may be I just have to wait until this feature is implemented into browsers. In the meantime, I thought I would post the source for the the Spectrum Analysis tool for greater minds than myself, wanting a quick way to get some data from their favorite tune and perhaps figure out a way to use it in a optimized fashion.
it’s very simple and outputs JSON with Left and Right channels to floating point numbers between 0 and 1. You can use it to make stuff move to the groove. It has a simple execution of this in the ui for you to see. Download the flash and php and throw it on your local server.
The process of accessing audio data from within the browser is quite beyond me, I really just want to see what I can create visually in Processing js and Canvas. Until it’s and easy to use API, I just have to keep looking for a way to get spectrum data quickly, hence this contraption. Have fun.
Download Source
This demo does not actually write the data for fear of abuse, but this is a sample of the output.
P.S. Carving Powder mp3 is one of my own creations. 
Related Posts:
Canvas, Audio, Raphael & Twitter
Last week I worked up an audio visualizer in Flash for a friend that yielded some nice results. The AS 3.0 ByteArray and computeSpectrum are powerful and have many potential uses. Doing the same thing with Canvas and JS requires a more round about method, doable, less accurate but more gratifying from a coding perspective.
My journey down this rabbit hole led me to almost doing this. http://vocamus.net/dave/?p=914 and here is some other interesting reading, https://bugzilla.mozilla.org/show_bug.cgi?id=532492. I got myself a copy of Minfield http://www.mozilla.org/projects/minefield/ and read up on how to build my own audio patched version with http://weblog.bocoup.com/javascript-fft-audio-sampling-in-firefox. Ok right. Still working on that one.
Needing more of a quick fix, I came up with two solutions. My first attempt at a solution was to build a Flash audio sampler that piped out the -1 to 1 floating point numbers to a file and use them in a JS array to manipulate coordinates of sprites on a Canvas. It worked quite well but the synchronization was off. I have not fixed this in that version but in the interim I got side tract and found another way to get some data. Quite simple really. Load up an mp3 into Audacity and go Analyze -> Time -> Beat Finder. Set the tolerance to about 55 and this will generate beat labels. Export those to a text file and then clean up the numbers to get an array of milliseconds for your JS setInterval();
Granted this is not spectrum data, but simpler beat info. In a way it works better for animating sprites because it’s cleaner. Spectrum data is very granular and works nice in widgets designed to show this, e.g. Graphs and Histogram visuals.
This test shows twitter searches on HTML5 Canvas and moves them around in time to the beat of the music loaded in an Audio tag. It’s not very exciting at this point but it’s got me thinking of some fun stuff to do. Every experiment like this is a step closer to moving away from flash. I still can’t settle on using Processing or Raphael though?
