I have been unsuccessful so far in syncing animation drawn in Canvas using Processing js & audio played using the browser Audio tag with audio spectrum data I create from a ByteArray in Flash. It works in Flash due to the fact that the spectrum analysis and animation are executed together in an onEnterFrame event. The drawback to not having real time audio spectrum analysis from within the browser is the need to supply all the data to Processing js at initialization. It can be rather large and I have yet to perfect syncing the reading of the array with the current time of the audio. Some sort of progressive data stream and excellent timing system???
It may be I just have to wait until this feature is implemented into browsers. In the meantime, I thought I would post the source for the the Spectrum Analysis tool for greater minds than myself, wanting a quick way to get some data from their favorite tune and perhaps figure out a way to use it in a optimized fashion.
it’s very simple and outputs JSON with Left and Right channels to floating point numbers between 0 and 1. You can use it to make stuff move to the groove. It has a simple execution of this in the ui for you to see. Download the flash and php and throw it on your local server.
The process of accessing audio data from within the browser is quite beyond me, I really just want to see what I can create visually in Processing js and Canvas. Until it’s and easy to use API, I just have to keep looking for a way to get spectrum data quickly, hence this contraption. Have fun.
Download Source
This demo does not actually write the data for fear of abuse, but this is a sample of the output.
P.S. Carving Powder mp3 is one of my own creations. 
Related Posts:
I had some trouble getting fonts to work in Processing and then more trouble when I posted the files to my Blog. I finally got it all working so this might save you some time.
Stuff you need:
- A True Type font, e.g Arial.ttf
- Batik TTF to SVG Convertor
- Possibly need access to either your servers httpd.conf or mime.types for .svg mime-type
If you have all this, you are going to get it working.
First off, stick the ttf font you want to convert in he dir where you extracted Batik & pull up a terminal and cd to it.
Type: java -jar ttf2svg.jar myFont.ttf -o myFont.svg
Replacing the myFont with your font. The result should be myFont.svg, great!!!. Copy it to your working Processing dir.
In your Processing setup function add the lines:
font = loadFont
var myFont = loadFont("myFont.svg");
textFont(myFont, 12);
Test it and see if you like it, if not repeat the process with another font. Go ahead and upload to your server and test live.
Looks good, ? yes : no;
For a quick test to see if your server supports svg head over to http://web-sniffer.net/ and type the path to your svg font in the box and hit submit. On return, look down to the HTTP Response Header table and look @ the content-type. If it says anything other than image/svg+xml or at least something with svg in it, you need to do this next step.
This is the part that got me and where you need access to the server config files I listed above. If you do not, you need to contact your sys admin or service provider and tell them you need support for svg files. If you do, edit either the httpd.conf or mime.types file “depends on your server setup” and add the line:
AddType image/svg+xml .svg
or
Content-Type: image/svg+xml
again it depends on what flavor of Apache or whatever you have. You can Google your setup and find out.
Restart your server and you should be good to go. Happy Processing!
Related Posts:
I jumped over from Raphael back to Processing again today. I have said it before, us Flash guys might get frustrated trying to pull off some simple stuff in either Processing or Raphael. Just keep in mind that they are young languages and the future looks bright.
Anyway, another attempt to do something here with Processing. It took a me minute to find an elegant way to get Twitter json data into processing without being too sloppy. The visualizer works perfectly in all three browsers on my local server “see pic”, but remotely it works in Safari and Chrome but does not load the svg font so I am off to read up on that, and it’s completely hosed in Firefox with some Processing js uncought exception error, Blah, Blah Bah… All stuff to figure out. If you grab the source and run it local, it should work. If anyone knows why I might be having those two issues, please let me know. Demo Here
Processing JS Twitter Visualizer
Related Posts:
Last week I worked up an audio visualizer in Flash for a friend that yielded some nice results. The AS 3.0 ByteArray and computeSpectrum are powerful and have many potential uses. Doing the same thing with Canvas and JS requires a more round about method, doable, less accurate but more gratifying from a coding perspective.
My journey down this rabbit hole led me to almost doing this. http://vocamus.net/dave/?p=914 and here is some other interesting reading, https://bugzilla.mozilla.org/show_bug.cgi?id=532492. I got myself a copy of Minfield http://www.mozilla.org/projects/minefield/ and read up on how to build my own audio patched version with http://weblog.bocoup.com/javascript-fft-audio-sampling-in-firefox. Ok right. Still working on that one.
Needing more of a quick fix, I came up with two solutions. My first attempt at a solution was to build a Flash audio sampler that piped out the -1 to 1 floating point numbers to a file and use them in a JS array to manipulate coordinates of sprites on a Canvas. It worked quite well but the synchronization was off. I have not fixed this in that version but in the interim I got side tract and found another way to get some data. Quite simple really. Load up an mp3 into Audacity and go Analyze -> Time -> Beat Finder. Set the tolerance to about 55 and this will generate beat labels. Export those to a text file and then clean up the numbers to get an array of milliseconds for your JS setInterval();
Granted this is not spectrum data, but simpler beat info. In a way it works better for animating sprites because it’s cleaner. Spectrum data is very granular and works nice in widgets designed to show this, e.g. Graphs and Histogram visuals.
This test shows twitter searches on HTML5 Canvas and moves them around in time to the beat of the music loaded in an Audio tag. It’s not very exciting at this point but it’s got me thinking of some fun stuff to do. Every experiment like this is a step closer to moving away from flash. I still can’t settle on using Processing or Raphael though?
Related Posts:
Took a day to extend the tentacle class to work on 360° and secondary motion from a parent node, but here it is, the first critter for my ocean. Thanks to Keith Peters book ActionScript 3 Animation, useful formulas section for getting me through this. It kicked my butt and fried my brain, but it’s so clear now. Onwards!
Related Posts:
Spent last evening coding some circles & spirals in Flash. I love spirals!
A logarithmic spiral, equiangular spiral or growth spiral is a special kind of spiral curve which often appears in nature. The logarithmic spiral was first described by Descartes and later extensively investigated by Jakob Bernoulli, who called it Spira mirabilis, “the marvelous spiral”.
This kind of simple Math and lengthy tweaking of geometry animation is akin to whittling a piece of wood for me, whenever I have some free time and get the urge to write some code with no particular objective. It’s very relaxing.
It went through numerous iterations over the weekend and truly came to life when I started using Vectors and local3DToGlobal to convert the x,y,z coords to draw the 2D lines. Throwing in some random offsets from time to time helps keep it interesting. I used Math.sin and Math.cos a lot with random offsets to create smooth undulating motion that sometimes reminds me of jelly fish or some sort of under water plant waving around in the current. Other times it takes on a galaxy formation.
Watch it for 10 minutes or so and you will see some interesting spiral explosions. I can stare at it for 20-30 minutes.
Related Posts:
Discovered this Web 2.0 Suicide Machine on Twitter. One day I might use it.
http://www.suicidemachine.org/
Related Posts:
I have been experimenting with the new HTML 5 Canvas and trying to get up to speed on the syntax. As a seasoned Actionscript 3 developer, I find JS easy to understand but difficult to write. I find myself thinking flash and trying to work a flash approach. Anyhow, I have tried canvas with js and canvas with Raphael and now canvas Processing JS.
They all have advantages and drawbacks. Processing js is giving me the most fun for now and I can’t wait till they finish implementing the entire set of methods, or at least what’s left that can be implemented into canvas.
I have read some eager articles saying that this can replace Flash and will be the end of flash. This may be so at some point in the future, but I can tell ya right now, it’s not even close. With no dedicated VM it craps out if you try anything with a couple hundred worthy sprites on screen. Also, objects are painted onto the canvas and no longer exist as objects you can have a pointer to. You basically have to paint the the canvas each time with your updates, example: Processing uses pushMatrix() and popMatrix() to store and restore stage translations. This is pissing me off and I am still trying to get my head around it. I just want to move stuff around dang it. On the other hand, when you paint, you can have an infinite amount of detail without worrying about overhead, like when you pipe MC’s into Bitmap object in Flash.
It’s all good though and a step in the right direction as far as I am concerned. No plugins, no 3rd party expensive IDE’s etc. It’s all free range and organic, yay. Can’t wait to start working in some div’s, copy etc and growing the graphics and UI for html pages.
As with all things animation and drawing, I like to start off with basic math and growing plants, trees, etc. Processing has no real shading, 3d or drop shadowing built in right now and it’s kind of refreshing going back to the old way of thinking where you have to fake it all.
Here is a screen shot of some alien plant thing I am writing. It takes about 20 mins to draw the entire 360 view. The depth and dimension is all fake like I said, drawing all the highlights, shadows etc in separate passes with slight offsets. Demo of my first processing attempt here.
Rendering Plants with Processing JS
Related Posts:
Thankfully Liam is ok and back in business.
Strike
Posted from my iPhone 3G using QikBlogr.
Related Posts:
It had to happen to my kid right? Bounced right out and got a concusion.
Knockout
Posted from my iPhone 3G using QikBlogr.