(view source for code, and who knows what bugs there are)

I’m getting into unfamiliar territory these days with exploring sound in Flash.

Flash 10 gives you the ability to take a sound object, and extract the entire thing into the raw data….a byte array.  How do you interpret this data? How can you process it into something meaningful?  Well I don’t know.

What I do know is that to get a firmer grasp on how I can utilize this, I need to understand what the sound object is at a basic level.  Probably the worst way to understand it is to take an entire song (mp3) file and let my eyes glaze over at the stream of numbers coming from the sound.extract feature.

No - the best way, I thought, would be to compose my own notes and chords, and work up from there.  To do this, I needed to understand what a note is, and what a sound is at a basic level.

My first thought was to picture sound as the nifty looking sound visualization that comes with music players these days…you know, those dancing bars:

Turns out that this is the worst way to imagine sound for this purpose.  My first question was….”OK, how do I get the value of the low frequency?”, “how about something in the midrange?”, “what about the highend?”.

Yes you can imagine sound this way, but only after processing your sound data with a Fourier Transform.  If you know how to do this, stop reading this right now, cause you’re way smarter than I am at this point in my sound exploration.

The BEST way to imagine sound is to picture a sine wave:

If you look at how tall the wave is, that is how loud the sound is, or the amplitude.   How close the peaks and valleys are in this picture is the frequency.

Frequency, at this very basic level can’t be thought of as high pitched, low pitched, etc - it’s only how far apart these peaks and valleys are.  Now think of this as a side-scrolling wave that goes on forever.  If you were to scroll at a constant speed, each peak would hit that vertical center line at a constant rate.  This rate would be the frequency.  If it goes faster, the frequency is higher - slower, the frequency is lower.  And, of course, this directly relates to how you hear it.  The more peaks and valleys at a given time, the higher the tone you hear.

Now think about how this relates to our Flash sound object’s byte array.  We could actually draw a sine wave with numbers.  This is nothing new to programmers.  Folks use trigonometry all the time.  But, I personally, never thought to use it for sound.

Try this AS3 code:

for (var i:int=0; i < 100; i++ ) {
var sample:Number;
sample = Math.sin(i * 2 * Math.PI) * 50;
}

This will basically plot a picture like the one above - a sinewave.

If we made this into a sound, it almost, but not quite be a tone.  However, if we pop over to google, we can actually look up the frequency, of say…..a middle C note, or a middle A.

http://www.phy.mtu.edu/~suits/notefreqs.html

There’s one more piece to this puzzle though, and that’s sampling rate.  A digital audio file could have all the sinewaves, peaks and valleys in the world, but if your audio playback is super slow, it’s gonna sound like garbage.  That’s why we tell our audio software to read our sound at a certain speed.

If you’ve heard the term 44.1 kbps when folks talk about sample rate - you can look back at our little for loop that drew the sine wave, and realize that you need more than the 100 points of data as we drew, you need 44.1k, or 44,100 PER SECOND.

So let’s rewrite that code:

for (var i:int=0; i < durationinseconds * 44100; i++ ) {
var sample:Number;
sample = Math.sin((i) * 2*Math.PI/44100 * 440) * volume;
}

OK!  So now, we’re creating a one second middle A (4th octave).  Our sample rate in Flash is always going to be 44100 if coming from a flash sound object (though that’s not true for any audio file).  We learned from our handy/note frequency chart when looking on google that 440hz is a middle A.

As a side note, lets say we want a different octave.  To go lower, half the frequency to get 220.  To go lower, half that.  Higher?  Double it.

Let’s think about the voice now.  The nice sine wave will give a nice, round tone.  For a dirtier tone that sounds like it has sharp edges….well our sine wave needs to have sharp edges - which is actually a square wave.  Our nice round peaks and valleys would be just straight corners.

To change the tone in the code, try this:

sample = Math.sin((i) * 44100 * 2 * Math.PI * frequency) > 0 ? volume : -volume;

As for more voices, well, I haven’t tried it, but a real world instrument would have a hard strike and then some falloff.  So our nice sine wave would be less round at the start of each peak, but comes back down way slower.

Here’s my final code (and keep in mind that I’m writing the sample 2 times, one for the left channel and one for the right):

returnBytes:ByteArray = new ByteArray();
for (var i:int=0; i < _duration * 44100; i++ ) {
var sample:Number;
if (_voice==VOICE_SQUARE) {
sample = Math.sin((i) * 44100 * 2 * Math.PI * this.frequency) > 0 ? _amplitude : -_amplitude;
} else {
sample = Math.sin((i) * 44100 * 2 * Math.PI * this.frequency) * _amplitude;
}
returnBytes.writeFloat(sample);
returnBytes.writeFloat(sample);
}

Now, what do we do with that byte array?  Well, as of Flash 10, our sound object, has a sample data event.  When this event is called, when it needs new bytes, it’ll call out to your custom sample data method.

What I do - and what you may choose to do, or not choose to do, is make my whole byte array first, and then read sequential amounts of data into the byte array each time it’s called:

soundBytes.position = 0;
dynamicSound = new Sound();
dynamicSound.addEventListener(SampleDataEvent.SAMPLE_DATA, addSoundBytesToSound, false, 0, true);
soundchannel = dynamicSound.play();

private function addSoundBytesToSound(event:SampleDataEvent):void
{
var bytes:ByteArray = new ByteArray();
soundBytes.readBytes(bytes, 0, Math.min(soundBytes.bytesAvailable, 8 * 8192));
event.data.writeBytes(bytes, 0, bytes.length);
}

To explain, about the 8 * 8192….

8192 is the maximum amount of samples you can use for each sample data event.  However….each sample is a left and a right 4 byte float.  So that’s 8…..time 8192.

There’s tons of cool stuff you can do with this.  If you don’t believe me, look up Andre Michelle - he’s THE MAN when it comes to this stuff.

Now, I wanted to take it a step further and visualize the sound before it’s even played.  Of course doing this with computeSpectrum is impossible because the sound has to be played first.

Luckily, in Flash Player 10, Adobe introduced sound.extract();  This can take the sound object and turn it into a byteArray.  I came across an example by Thibault Imbert and of course the example worked very well to visualize a sound spectrum from a byte array - though I still wasn’t sure what each byte represented in the Flash Sound object.  And even though, the spectrum looked OK,  a comment from Schell explained that this is a false waveform and to look at his blog post.

The blog post provides some great working code - but there was quite a bit I didn’t understand yet.

So, I attacked the problem from another angle today.  I thought - OK, I don’t really understand the bytes that are coming OUT of the sound object, but maybe if I play with shoving bytes INTO a sound object to create dynamic sounds, I can get a better handle on tings.

The real struggle came because I used Christopher Martin-Sperry’s code to create a sound object from the raw byte array provided by an MP3 file.  I used this before really looking into how all this stuff works.  The code works fantastically - but put my brain on the wrong track.

I was on the wrong track because MP3’s and Flash’s sound Object work a wee bit differently from each other.  In an MP3 file, there is a header to give some info about the file.  Also, the MP3 file is broken into “frames”.  Each frame has a header to give information about the frame (bitrate and other things).

This doesn’t map so well to Flash’s sound object.  First of all, the bitrate of a Flash sound object looks like it doesn’t vary.  It seems to be 44.1 kilobytes per second all the time.  And it’s always in stereo.  It looks like the source of the audio will always get upconverted or downconverted to this 44.1KBps stereo format.

Not only that, but there’s no concept of frames.  I thought I’d need to know how to read a frame header, or at least know how long each frame is to know what type of data I’m getting.

But that doesn’t appear to be the case.  The byteArray extracted from a sound object appears to be an alternating left/right channel stream.

How does the stream relate to time? I felt a little dumb when I figured this one out - because it’s a little obvious.  It generated a few tones before having this little gem of an epiphany.

Well, if the Flash sound object is 44.1KBps, that’s 44,100 bytes per second.  So when reading the stream, we read 44,100 bytes before a second is up. Right?

Well, not so right it would seem.  You have to consider both channels.  You’d read each channel 44,100 times to make up a second, making one second 88,200….something.

What is that something?  Well my computer sciece teaching has failed me, but I can speculate.  Adobe provides a morse code generation sample on their developer network.

In it, there is a comment when they are writing the samples to the sound stream that they are writing 8192 samples to the stream.  However, the length of the byte array they are actually writing is 8192 x 8, explaining that each sample is two 4-byte floating point numbers.

So perhaps when reading floats with ByteArray.readFloat, you’re actually reading both left and right floats from the same byte.  If I’m right, then you’re still using 44,100 bytes per second, but each byte represents two floating point numbers.

However, it works out, I’ve noticed that when creating six 1 second tones, by writing two floats at a time, 44,100 times,  it clocks in at 6 seconds.  So infer whatever you wish from that - I know I have.

My next step is to figure out how to pull frequency data at a particular point in time.

http://www.theWB.com

I wondered how we can increase replayability of music and get people to appreciate a song or collection of songs more and thought we can use games to achieve this.  After all 30 year olds like myself grew up on video games with soundtracks that repeat every time you play a level.  Why not use casual games to increase your band’s audience - to get people to listen to your music over and over again and get the tune stuck in their heads?

Of course, its easy to slap a soundtrack on an online game - but how can we make the game and soundtrack work together for a great experience?  How can we create musical escapism in games, just like MTV or musical theater?  What is the gaming equivalent of breaking out into dance, or fast camera cuts timed to the rhythm?

As you can tell, I’ve been thinking a lot about this lately, and I’ve recently read This is Your Brain on Music by Dr. Daniel J Levitin to better understand how our brains perceive music and how the various flavors of music appeal to us in different ways.

The most obvious way to tie a game to music is tempo or rhythm.  There are many rhythm based games on the market today - most noteably “Rock Band”.  Music has many more attributes, obviously, like the tone, contour, timbre, loudness, meter, theme, key,  harmony and melody….not to mention that rhythm and tempo are two different things.

Relating any of these aspects to games can be challenging given where we are with mainstream realtime audio-processing technology.  But even putting technology aside, how can we relate these to a piece of music in a systematic way?

Let’s start with tempo.  Tempo is the overall speed of the music.  It’s fairly constant for long periods of time - but can be an excellent baseline metric for establishing a musical pulse and to possibly tie it up with a visual pulse in your game.  When people tap their toes, dance, or bop their heads - its generally to this tempo, or the beat of the song.  So tying visual elements to this beat is probably the most effective tool I can think of.

And this is good, because a beat can be measured in pop music by picking the range of frequencies that the bass drum is on and listening to the volume of this frequency over time.  So tying visual elements to a beat is a very real tool that can be done automatically in music game creation.

Tempo only goes so far though.  It’s a very predictable musical trait.  Levitin argues that our musical tastes can somewhat be centered around complexity and predictability.  Children, for example, enjoy very simple and predictable music.  As we get older, and the more we listen to music, the more boring this simple and predictable music becomes.  So we listen to more complex music - but constantly strive for the right amount of predictability and the right amount of complexity.  As we listen to more and the level of complexity of our favorite music will probably go up to.  This is one of the reasons that the music of other cultures can be distasteful, or classical music, or jazz can be distasteful for people with pop only listening habits.  The more unfamiliar we are with music, the less predictable it is.  And if its not predictable at all - as another culture’s music can be to us - then it can really be unpleasant.

So, going beyond the beat to capture other nuances of the musical piece should be very important in manipulating visuals or gameplay elements.  ONLY syncing a predictable visual beat with a complex soundtrack would be such a shame because you lose all that which makes the music meaningful - and you can lose the connection between the two.  Alternately - having lots of movement that has nothing to do with the flow of the piece but loosely tie to frequency at a particular time, can create a similar disconnect because you’re picking up unpredictable parts of the piece to tie them together visually with the music.

So what types of musical nuances can we pick up and use?

Rhythm is another form of timing like tempo but it’s how notes are grouped together into phrases.  Guitar Hero and Rock Band use rhythm effectively.  I believe Harmonix actually does this by transcribing the notes themselves for each instrument, and grouping them together to have you play out a phrase at a time.   They’re not picking out random notes from the guitarists score and having you play them - no, they’re taking  the most meaningful notes  that make sense to put together the phrase with the limited amount of notes you can play.

Tone and frequency for example are aspects that may be quite hard to integrate effectively.  Considering an overall musical piece, many instruments are playing at different pitches and frequencies all at once.  The bass guitar has a very low frequency, while a flute can have a very high frequency - even though both can be playing the same note or different notes.  Taken out of context, different frequencies/tones in music don’t make us tap our toes.

Pitch, harmony, contour, key, and melody - taken together can be an entirely seperate but equal way to tie visuals to your music.  Unfortunately - it can be very difficult (if not impossible) with mainstream game technologies like Flash to take this into account in an automatic way.  I hope I’m wrong - and I hope somebody PROVES I’m wrong and lets me in on the secret, but consider this….

Using Fast Fourier Transform methods (FFT), I can grab the volume of any frequency at any time.  Can this provide me with what note is playing?  Well, maybe….but only if a single instrument is playing.  Unfortunately many instruments are playing at many different frequencies, and many can bleed into (if not use) another’s frequency.  I know some smart computer scientists have developed pitch detection - but it hasn’t made it’s way into any code libraries I know of yet (though I should look beyond Flash to see what I can find).

Also, on a technical level, using MIDI files, if the music is transcribed correctly (or if it originated on a sequencer or a computer in the first place), we would have access to all the notes seperated out into the different instrumental tracks.

So what if all this technical stuff presented no barrier?  How could we use it?

One of the most interesting parts Dr. Levitins books for me was Appendix B: Chords and Harmony.  Music, if you look at it in a very dull light, is all about patterns and manipulating people’s ears by breaking in and out of  the predictiability of those patterns.

Talking about pop music, there is a verse, a chorus, and a bridge.  A verse is normally a melody and tempo played over a few times.  At the end of the verse, we get into a similar thing with a chorus, which is generally shorter.  Going from chorus to verse breaks one pattern, but picks up another musical pattern which uses slightly different chords with a slightly different tempo.  So predictability is broken, but not very much.

Another way patterns are established is with long standing musical tradition.  The blues is somewhat defined by going from I Major to IV Major to I Major and then IV or V major, and then back to I Major - this happens to also be the basis for rock music as well.  Now, this is another established pattern - and when we break from this pattern, we’re introducing unpredictability.

The most intersting pattern Levitin mentions is when a chord is either resolved or unresolved.  In Western music, our ears have been trained to consider a tritone or an augmented 4th, the most unresolved interval we could possibly hear.  In fact, as Levitin recounts, this interval was banned in the Catholic church and named “Diabolus in musica” citing this interval as the work of the devil.  On the other hand, a simple major chord is considered resolved.  So when if we play a chord containing a tritone, we expect, and almost demand that it be resolved by something like a major chord.

Why do I bring these up?  Well, this type of musical behavior can start to paint a more visual or motion based picture.  When music breaks expectations either by changing tempo or with different chords, or even moving the notes up or down an octave - likewise our gameplay elements should match this level of broken predictability.

Likewise, when a chord is left unresolved - so too should our visuals on screen.  You know something is going to happen, but you don’t know what.

To spell it out more clearly, we can establish a visual and animated baseline in our games by listening to the beat of the music, and timing elements to this.  Assuming a pop song, our baseline is a song verse.

Deviating in minor ways while the verse is played, maybe be changing the tempo, going up an octave, or different instrumentation, will produce a minor change in the visuals or animations.

Going from verse to chorus however is a less subtle change, and often implies changing the tempo, chords, or otherwise quite drastically.  In this, the gameplay needs to change in the same fashion.

Hitting a chord that needs to be resolve creates musical suspense - and so too should it create visual or gameplay suspense.  Something on screen about to fall, or teetering from side to side.

Going back to technical implementation, however, assuming we had access to the entire score of notes in a musical piece, can we feel out these suspensful moments or feel out when a song is predictable and when it becomes unpredictable?

I believe the answer is yes, but it would take quite a bit of work to look for augmented fourths, or other unresolved intervals, or to run pattern matching algorithms on our melodies.

A more pragmatic approach might run a FFT analysis to find beats, and possiby even do some light pattern matching to seek out changes in tempo.  Meanwhile, do some manual markup of subtle changes in verse, and then manually mark where verse goes to chorus, chorus to verse, and where the bridge is.  We can also manually mark where we’re creating suspense, and then where we resolve that suspense.

In this fashion, we could create a sort of music markup language to map a timeline of all these events and use this to create a musical gameplay experience.

I guess my work is cut out for me!

Netflix On Demand is damn cool - I was watching HBO series and new movies through a nice Windows Media Center interface called MyNetflix.

Not only that - but the digital content from online looked a lot better than my cable signal did (I still have a normal non-HDTV by the way).  I think that they compress the content digitally before it comes through the cable wires - and then of course Windows Media Center compresses it again in a different way….and thats no good when you compress the same digital video twice under two different compression standards.

So - thus begins my experiment:  Bringing HDTV to my computer even if my TV doesn’t really support it.

Saturday morning, I installed the card.  I was sorta hoping that it would just magically pick up HDTV from the air without plugging in any antennas or anything.  Well that was a big fat no.

Then I went to Best Buy - I got an multidirectional amplified indoor antenna.  Basically, a big flat sqaure, probably with some wires inside, and you plug it into the wall.  So - I hooked that up….in my finished basement…where we watch TV.  OK, a big fat no.

“This is discouraging” I thought - but thought I’d keep trying, since I was in the basement after all.  I grabbed a roll of coax wire and ran the indoor antenna outside to my raised porch.  I tried again.  This time it was more promising.  I scanned through the channels, it said there were a few digital channels available - so I started flipping through.

It looked good but not great.  It was a little snowy, some bad pictures, definitely not looking like what I expected out of HD.   Oh but what’s this………there’s a WRAL-DT?  Umm….hey a whole bunch of DT channels in the low 1000’s.  So apparently, I wasn’t looking at HD, Windows Media Center just threw in the Analog channels into my guide for good measure at the beginning of my guide.  OK, lets flip to the HD stuff…….daaaaaaaaaamn.  It looked awesome.  Very crisp and clean on my computer monitor, I hadn’t even hooked it back into my TV yet.

So, I set out to see what channels I could get…….

ABC, check….NBC….check, CW….check, PBS….check, FOX….check, CBS…….where’s CBS?

I went to my local CBS affiliates website, and found that I’m about 15 miles outside of their range using an indoor antenna.

Sunday morning, I set out to the hardware store to see what else I could buy in antennas.  I sorta wanted an outdoor one, but the thought of me going up on the roof with my non-existent ladder and installing something - somewhere, somehow on my house seemed hazardous to my health.

I got to Lowes, saw they had rabbit ears and outdoor antennas.  Thats it.  But I picked up a 4 foot long outdoor antenna box and looked at the packaging - “Can be used in your attic”.  Hey!  Awesome, I hadn’t thought of that!  I can put this huge freaking thing in my attic and run a wire down 2 stories to my basement….somehow.

So I bought it at $60 - and a 100 foot roll of coax cabling.

I got it home, brought it into my attic, started unfurling it - and I won’t lie - its very unweildly.  Picture me, in an attic with no floor, standing on beams trying to shift around a 5 foot wide, 4 foot long antenna.  I dropped it in place eventually - didn’t bother bracketing it to anything, and connected the coax cable, and searched for a way to get it out of the attic.

And look….a vent, with a screen over it thats slightly unhinged at the bottom.  Perfect place to drop cable out.  So the cable was unreeled, and unreeled, and unreeled.  I brought it in around the window entrance for the cats into the basement and over to the TV.

At long last, I have all the channels I need!  CBS included, so now it’s just a matter of returning the indoor antenna to Best Buy and setting things back up on Windows Media Center.   Media Center tells me I have 21 channels - most don’t come in, but as I look up the call letters on the internet, I find that those are pretty far away like in Wilmington, Winston Salem, Greensboro, etc.

I settle on 8 channels - WUNC (PBS), WRAL (CBS), WTVD (ABC), WNCN (NBC), WRDC (CW), WUVC (Univision), and WRAZ (Fox).

Now the tricky part….for the short time I’m actually paying for cable still, I want to make use of it, and Windows Media Center (as far as I know) won’t let you have both over the air and cable channels together set up at the same time.  So, I’m installing Beyond TV as I write this (a different DVR software solution), and I plan to use that to record cable, and Media Center to record over the air.  But that’s just till we cancel cable.  I mean - I’ll record Battlestar as long as I can until I have to go to SciFi.com (or someone elses house) to watch it.

But at last count - the Daily Show and the Colbert Report are free at Comedy Central.com.  Monk and Psych are free at Hulu.com, my wife watches Ghost Hunters and both of us Battlestar which is free at Scifi.com.  I just can’t think of any other cable shows we watch really.  And if miss something, we’ll wait and get it on Netflix.  And hopefully I’ll get an invite for Boxee TV on Windows (please please please) since I filled out the request form on their website last week.

So now, I’m just waiting for Beyond TV to install, and then go hit a superbowl party even though I hate football.

If I have this problem - others probably do to, and may even be perfectly happy to be oblivious to this.

This is also a problem for musicians.  If people give music only superficial listens, and are just as happy to move onto the next song by a different artist, why would a musician have fans that bother coming to shows, buy t-shirts, or buy follow-up works by that musician?  Isn’t it easier just to turn on an internet stream and listen on shuffle than to seek out and buy an album or see who’s playing in your town on Friday night?  It’s as though being a fan is becoming a lot less fanatical.

My concern over this is that music will be written to appreciate on a superficial level - will only make you nod your head to a beat.  Of course it’s already been happening in pop music for years.  Musicians will only get their “radio friendly” hit song played wide-spread.  But many times, this radio-friendly song will be the gateway drug that leads you into the album, and get you to repeatedly listen to all songs by the artist.  Other songs on the album could provide deeper enjoyment.

Whether this problem is new and exasperated by our digital devices, or an old problem but just becoming apparent to me as I change my listening habits, I have to wonder what we can do to change these habits and produce ways to make people want to listen over and over again and come to a deeper appreciation for the music and the artist.  Listening to music over and over again can even make things more memorable.  How many times have listening to a song for the first time in years bring you back to old times when you were listening to that song constantly.  It can be very nostalgic.

I’ve been thinking a lot lately about music and video games (as you can infer from my blog posts).  I think that games like Dance Dance Revolution, Guitar Hero, Rock Band, etc do go quite a way in invoking repeated listening habits in music.  Pressing buttons in time to a beat, however, is just one direction to take this - I would call this a music creation game (even though you aren’t actually creating the music, there’s the illusion that you are).  I’m quite interested in exploring  a game where music is creating the game play environment that you’re in.

Another way to put this is I’d like to explore what games can be to music as what music videos are to music.  The best music videos, in my mind, produce another world where people break into singing and dance.  A mailbox will spring to life on the street and dance with Bjork.  Christopher Walken will dance his way down an escalator and fly through the air in grand choreographed moves.  Weezer will give a show to screaming members of the Happy Days cast.  You get my escapist point (and my Spike Jonz fixation).  I’m not a musical theater fan, though Gilbert and Sullivan have a lot of experience with this too, and they go a little farther back than MTV.  From what I understand, even the ancient Greeks had musical theater.

Bringing it back to games - a lot of great work has been put into cinematic gaming soundtracks, but to my knowledge, they are just soundtracks and don’t really change what happens in the game.  A change in tempo doesn’t make more bad guys come out to hurt you, a minor chord struck doesn’t usually signify coming doom.  I shouldn’t say never.  I’m not an avid gamer - I just hadn’t ever heard of this attention to music detail.  In fact - it’s quite the opposite - more bad guys coming out will change the tempo of the music - and coming doom, will change  the music to a minor key.  Game composers will typically write tiny segments that can be switched depending on what happens in the gameplay.

So, how can we produce a truly musical game?  One that gives a level of escapism straight from Broadway and MTV, gives music a forum to be listened to again and again, and is fun?  One where the music isn’t changed for it’s gameplay, but where gameplay revolves around the music?

(Yah, I’ll answer that question next, or at least try to)

I was a late comer to both Battlestar Galactica and 24.   I didn’t start watching either one until a couple seasons in.  I recorded a bunch of 24’s with my DVR, and bought a few seasons of Battlestar Galactica on DVD.  Either way, I wound up watching entire seasons of a show in a week or two in contrast to the six or nine months it usually takes to watch a season of a show.

Watching habits were changed from when the network demanded it to whenever I wanted to see it.  In this, you could almost argue that the story itself behind the show has changed.    There’s no more downtime between episodes to think and ponder, and be on the edge of your seat about what happens next.

In fact, my DVR, Netflix, and DVD collection have changed my attention span on shows when I do have to wait to access them.  When I was watching my 24 marathon, I was really into it.  Then, when I had to wait a week to see what happened, I was less into it.  And now, I’m not even sure how much I care about 24 now that its been a whole year.

This is stark contrast to how we used to watch TV.  Either we were chained to our sets when our favorite shows were on, or our favorite shows just happened to be those that we had the specific nights free for.

The point is that having constant, on demand access to TV and film can ultimately change the story as we percieve it.  Watching TV can become more like reading a book.  You can rewatch/reread, read/watch when you want, and you don’t have to read/watch the whole chapter/episode in a sitting, or you can finish the entire thing in one sitting.  This is especially true as we can watch video on our mobile devices now.

Music can be a little more complex.  In addition to being short form, where consumption of a song can take only 3 or 5 minutes, there’s not usually a cliff-hanger or a compelling reason to listen to the next song on the same album for most people.

This became apparent when your single disc CD player became a 5 disc CD changer back in the 90’s.  I found it was easier to put my 5 discs on shuffle to get a little variety in each music listening session.

My car was a major place that I’d listen to music, and it was especially great to have my music on long rides.  My car, however, had just a single disc CD player.  That means when I put a CD in, it would stay in.  And because I was driving it, was a bit complicated to remove and change it unless there was a break in traffic.  I also only had a limited selection to choose from, as I’d usually only take the last 5 or 6 CD’s I bought with me on a normal car trip.

This means when I bought new music, I’d get very familiar with it.  I’d listen to it over and over again.

I suspect that most people’s music habits had the same limiting factors.  Either you listened to what the technology at the time allowed you to hear, or what the radio allowed you to hear - and each of these things led to the same thing, you’d listen to a song over and over again.

Of course with digital audio players with lots of storage this has all changed.  I have a Microsoft Zune with 120 Gigabytes of storage capacity.  This means that I can put my wife’s music collection and my music collection (probably 500-600 CD’s) on my Zune in its entirety 3 or 4 times over.  Many music players have at least 1 Gigabyte of memory allowing 10-15 albums at a time.

Suddenly, it becomes more convenient just to put every song you own on shuffle than to find the last albums you bought to listen over and over again - and given people’s tendency towards variety, a big shuffled list sounds more appealing when it comes time to listen to music.

So, listening habits have changed - and so has how we percieve music as a result.

Have you ever listened to an album or song, and didn’t really like it at first, but maybe after listening to it for the 5th time or the 10th time it’s one of your favorites?  It used to happen to me all the time.  But it really doesn’t happen to me much anymore.  Music listening has become sort of superficial to me.  I’d buy some new music, listen to it once, and then it appears on shuffle sometimes as I’m buying newer music.

It’s somewhat of a personal goal to give my new music a better listen from now one, despite the fact that my entire collection fits in the palm of my hand.

So as things are getting long, I’m going to continue things in a second post where I think about how people can listen to music on a less superficial level despite the technology we have.

I’ve been busy early this fall with writing “reactive music games”.  If you read my blog, which you probably don’t, you’d know that I couldn’t shut up about music games all summer.  I even did a little research into the different types.  The reactive game, is where music affects the things that happens in your game environment - but you don’t have to play along to (like Guitar Hero or Rockband).

There’s actually no popular music games I can think of that ARE reactive.  But I like to think of reactive games really…. like an interactive music video.  I even watched a bunch of Spike Jonz videos before I started sketching.  I have 10-15 games to do in the short term, and most are designed to be pretty easy to develop and generic - ESPECIALLY since this summer I’ve been working on min-framework in Flash AS3 that listens to beats in music.

The first game, I’ve done is basically an alpha, and if the music doesn’t load, just hit refresh (sometimes my security sandbox seems to go a little wacky).  I’ve codenamed it “Popfly” in my sketches, cause the balls kinda pop and fly out with the beats.

I’ve tried to pay special attention to things that I’ve noticed I didn’t like in other games.  Specifically, trying to tie visual events to music better.  Balls pop out, and when they reach their peak, its not very tied to music because it’s a split second after the beat.  This is why I put a subtle glow burst at the bottom when the balls do pop out.  I also gave the side gutters some ambient glow - just to have something additional on-screen that’s tied to the music in the game.  I also made some bigger balls pop up when the volume gets high - and it nicely adds a little crazy mosh effect when the music goes crazy.

I can’t wait to get through some more games, and see where this takes me, and hopefully way down the line, do a nice in-depth game with some characters.

But for now - here’s the PopFly game

Anyway, where was I?  So BIZKIDZ.COM is a site for kids to go and set up a storefront, and sell actual product from real online retailers.  Lots of retailers in fact, at least like 400 of em last I checked.  The idea was to create a virtual lemonade stand of sorts.  As a child with an account, you’d populate your store with products and decorations, and send links to family and friends.  If they buy anything through the store, the account holder gets commission!

It’s pretty cool - I ended up doing most of the front end.  We got some help from a great contractor - Clint Little, and J2 Interactive did our server side architecture.  What was great was that I got to touch every part of the project - I did 99% of the Flex (sorry Clint, but those initial designs we had you work on, I had to do them over when the client changed their mind, oh well….great job though!), and maybe 5% of the server side stuff, and most of the HTML.  And of course our talented art team at 360KID did some wonderful designs and illustrations.

I got a lot of Flex training on this, how to make a Flex application NOT look like a Flex application - all in all I think it was a job well done - and if I know BIZKIDZ, they’ll be continuing developing their site with 360KID to make it even better than we imagined at the beginning.