The Light of Day

Homegrown spectral analyzer

I whipped this spectral analyzer up this weekend. Unfortunately, like everything else coded in Handyland, it doesn’t run in realtime; it has to be rendered as a movie first and then re-synced to the music. I think it’s fun to sit and watch when it’s done, though. Sometimes, if you look hard enough (or sniff enough glue), you can see which peaks correspond to which sounds.

The featured instrumental here is Kid in a Candy Store, from Leave of Absence vol. 1, currently close to being ready for reissue. It was created by slapping a backwards orchestra track onto a drum track, relishing in the serendipity, and then coming up with bass, guitar, and piano bits that would glue it together a little more. This is the music I want playing when the aliens come to pick me up.

Maybe a graphic as simple as this, in combination with lyrics and/or factoids, would lend itself to my earlier idea of using YouTube as an audio player. My only gripe is that I would have to use a workaround if I want the music to be in stereo — at least until YouTube realizes it’s not 1950 anymore.

Sound atoms

So, according to Wunderground, March will in fact be going out like a lamb. It’s just waiting until the last minute to put its lamb suit on. And if I’m waiting until the last minute to have my car inspected (we have to do this once a year in New York state), then it would be hypocritical for me to demand anything else.

I’m working out an idea for a new coding project. It’s not in code language yet. It’s not even in pseudo-code. It’s in English, but not “blogosphere friendly English”, so I’m not going to alienate all y’all by posting said brainstorm here. Essentially it’s a kind of analysis and resynthesis for creatively simulating and manipulating voice-like sounds, which is something I’ve been wanting to do for quite a while now. I’m basing it around something I call “sound atoms”.

Why on earth we as humans feel the need to simulate voices, when we actually have voices, I couldn’t tell ya. But it should be interesting.

Look ma, no bubbles

Now that I’ve taken out the trash…

Strangest thing about my old auto-morphing code that I’ve had a couple of demos up for (also on youtube): it’s had an incredibly stupid bug in it all this time, that I would never have noticed if I wasn’t just staring at the code mindlessly. It was transitioning smoothly from picture to picture, but there seemed to be an excessive amount of white or light gray circular areas that would unnecessarily billow up before resolving to the next keyframe. Everything was working correctly except for there was this line of code that I copied and pasted and didn’t finish changing it the way I’d intended to, so instead of getting the data for a, b, c, and d, it was getting “b” twice and leaving “a” at zero.

The reason it didn’t totally cause it to malfunction outright? To interpolate, it takes the data from not two, but four pictures. The keyframe just before the current frame is called “b”, the one just before that is called “a”, the one right after the current frame is called “c”, and the one right after that is called “d”. So you’ve got a, b, c, and d, and the thing you’re filling in is somewhere between b and c.

The reason it uses info going that far back and forward is so that I can make the data curve as it goes along. It’s called a spline curve, when the computer sort of “improvises” a curve to fit an arbitrary set of points, and in order to know how the curve is shaped, you want to know where at least a couple extra points are beyond the ones you’re between. Well, since “a” was treated as though it were all zeros, it would come out of “b” at the right place but at the wrong angle. So the data would always have sort of an “upward lurch” at the beginning of the transitions, thus making all those white and light gray bubbles.

Here is a demo video of a transition sequence that works the way I meant it to.

(Is this the same Keith Handy guy that does the music, or am I at the wrong website?)

Our cushions never clash…

I made my vocal pitch graphing a little easier to read visually by de-saturating the quiet parts, i.e. graying the spaces between notes where the line on the pitch graph doesn’t mean anything and is just connecting the dots. So the bright colored columns are where the syllables happen. The idea will be that you can then open it in a graphic editor, mark it up where you want it tuned, re-save it, and have another program “read” your squiggles and go from there. The meta-idea being that I’ll have an alternative to Autotune to rein pitch into the general vicinity without flatlining it (killing the nuance and vibrato, and destorying emotive pitch scoops and fall-offs). I’m already doing this anyway, but this should help make it less tedious. Of course, you can do this with Autotune if you’re not lazy. But I’m not just lazy — I’m also poor.

Ethics? Learn to sing? Practice more? Do more takes? Bah.

I did have a fairly productive week last week, as I promised, and now I’m extending that promise to have a productive week this week too, even though it’s half over. Well, that’s okay (it being half over), because the pitch thing should be useful. In my remixing projects I keep coming to these points where my ears are fatiguing too quickly to be confident of the vocal pitch. I never have a problem getting it to sound good on big, loud speakers, when I have all that bass and stuff to support me; it’s the little, quiet speakers that taunt and heckle me.

But I digress! Last week, did a halfway decent rhythm guitar part to my ode to selling out, Curtis’ Classic Collection of Comforts — which is meant to evoke a train wreck without actually being one — and video’d it for posterity. I may also revive a similar but much older rhythm guitar take, from an earlier attempt at the song, back when I was a Stratocasterist, to combine with the newer take (dueling me’s!). It was a different version altogether, so if I do that, I’ll no doubt have to Frankentempo it (Feel that vocabulary s-t-r-e-t-c-h!), as in cut it up and slide bits to and fro.

Tip for the intermittently depressed: the thing that will make you feel better may not be a completely new idea or epiphany, 180 degrees away from whatever you’re whining about focused on — it’s more likely something about 20 to 30 degrees off the edge of your peripheral vision, something you’re aware of but haven’t been consciously thinking about. Beyond that, there’s always the next weather change to look forward to. (Thunderstorms are cool.) In the meantime, drink water and eat something healthy.

Chronically depressed: you’re on your own. Get pills.

If you build a better vocal pitch grapher…

Workin’ on it… middle of the yellow row is C, middle of the green row is D, and so on…

The unknowable black art of mastering part 2

Blah blah blah linear frequency blah blah logarithmic amplitude blah blah bright pretty colors.

There’s no particular reason why they have to be in the colors of a rainbow, mind you, that’s just my own personal eye-candy; in fact it’s probably detrimental, because I wind up sitting here going “ooh pretty” instead of moving along to the next step. (No, everyone, I really am working! It’s mental work!)

Anyway, the kindergarten version of this … if I can translate it into kindergartenese … “mastering”, as we all understand (and as none of us are the least bit prone to confuse with “mixing”) is the process of taking an otherwise complete, already mixed-down recording, and running it through some final nit-picky EQ and other processing to make it “play well with others” … particularly YOUR EARS.

The top half is an analysis of an old, crappy-sounding song, and the bottom half is a clean, full, big-sounding modern song. Left to right is low pitch to high pitch, but since the pitch scale here is linear, it means it’s not spread out the way we hear it. Most of the important stuff to our ears is smooshed into the left part of the picture.

The stuff that drops way off at the far right is in a range that only small children can hear. It’s interesting to consider that since music fans tend to be far younger than mastering engineers, they can hear stuff that the mastering engineers can’t.

The useful thing about all this is that any recording can be matched to any other recording’s profile. So what the top song lacks in highs (notice how much faster it slopes downward), it can re-gain by modeling it after the bottom song’s profile. And it can do this without creating a shrill, ear-hurting “spike” because it’s aware of the peak levels across the spectrum for both songs. It’s not just a dumb treble boost — it’s a precise auto-tailoring to a model.

The intriguing thing about these graphs, for me, is how straight of a line they both seem to form. I’m not sure what to make of that. This is the first time I’ve ever converted the values to decibels before graphing them.

The unknowable black art of mastering

I don’t think mastering is really as subjective as people say it is.

I think it’s about providing a balanced spectrum to the ear, so that you can take a lot in without being fatigued. Having a poorly-mastered recording is akin to having a lot of potholes in the parking lot at an art gallery; the paintings are still intact, but you will enjoy them less because you’re worried about your car.

The ear adjusts to imbalances, or at least the brain does, so if you have a recording that’s bottom heavy, it can figure out that it needs to shift its attention to a weaker part of the spectrum. But this is work for the brain to do, and you’re putting more strain on yourself for no aesthetic advantage. If a listener is going to make any adjustments at all, consciously or otherwise, wouldn’t you rather they go from more strained to less strained as their mp3 player’s random shuffle leads them from someone else’s song into yours?

The visual equivalent of mastering is adjusting the brightness and contrast of a picture. I think there is rarely any advantage to not using the full black-to-white range — if you want it to be a “dark” picture, then yes, have larger dark areas overall, but at least one tiny area should hit full-on white (and vice versa for bright pictures). Gamma and similar adjustments are great for all the stuff in the middle, but there need to be points of reference at both of the extremes.

This brings my analogy back to mastering, sort of. What is the equivalent of “gamma” in mastering? Well, where in pictures we have colors (red, green, and blue being the primaries), in sound we have frequencies from low to high. (Think of frequency not as a simple tone, but as a thin slice of music that “rings” at that pitch.) For any given frequency, we really want it at some point to be loud and at some point to be quiet. To have a continuous full-volume tone at some pitch for the entire duration of a song would be torture. But to never have much energy in that part of the spectrum would leave the sound “hollowed out”, like there was a hole in it. So at any frequency you need to have things happening; you need to have a range of louds and quiets. “Audio gamma” would be how much or how little of the medium-volume sound is closer to full volume and how much is closer to silence. The maximum volume (for that frequency) isn’t really an aesthetic choice, it can be “loud enough to get your attention without hurting you”. The aesthetic choice is how much of the time to spend closer to that maximum.

As I go along, I’ll be using existing music as a model, to automatically adjust the overall sound of my own. Stay tuned!

Pathetic, that’s what it is

Well, I’m back from the hospital (pictured above is an abstract representation of my mental anguish over the last week). Sorry I didn’t write a post to tell you I was going to the emergency room! I don’t know where my priorities were.

My right foot — yes, the same one that was broken a few months back — was pwned by a severe staph infection, probably as a joint result of scratching maniacally and soaking in stale/dirty water. My goal now is to not contract anything new for the rest of December, until my health insurance kicks in. (That’s right, boys and girls, I’m still vulnerable!)

Okay, I did lie about the picture. That’s a still from a follow-up experiement with the auto-morphing code. Although you may get a neat looking frame here and there, by and large cross-breeding between two simultaneous video sources is a failure. For one thing, it takes absolutely forever to run for even a short amount of footage, because it has to analyze two images every time it generates one frame (I left it running for several weeks; it was still going the entire time I was hospitalized!). For another thing, because of the way individual frames are interpreted, very small differences in the input can yield very large differences in the output, so the end result is extremely jumpy.

So the final verdict is that it’s really only ideal for filling in missing frames in a single video sequence. Analyze few, generate many.

I think I’ve improved on your methods

Above: a still from an improved version of my auto-morphing picture-bleeding algorithmic video whatchamacallit thingy. At this point, increasing the picture quality is just a matter of increasing the number of “passes”, so I’m going to concentrate more mental energy on making it more useful.

From the user’s standpoint, the way it works now is linear. You start with a single stream (thread, channel, whatever you want to call it) of video, with some frames missing, and it fills them in. You could start with a very small number of key frames (like in my early tests below), so that it generates the majority of images for a morphing effect; or you could have most of the frames already (as in pre-existing video) with only a few missing frames, and use it as more of a transition effect. In either case, though, as it stands now you start with one list of numbered frames, and it fills in the gaps.

What I want to do next is give it the ability/option to cross-mutate between two or more simultaneous video sources, sort of like an A-B roll but with the middle zone being more of a cross-breed than a crossfade…

Bleed test 2

No attempt made to synchronize anything:

So I suppose if you were sedated enough, this could pass for a music video. My initial criticisms:

1. What I already said above: no attempt was made to synchronize anything.
2. It doesn’t stay on the actual pictures long enough. Too much time spent in abstract-land.

Both of which could easily be dealt with.

(Holga pics by Christy)

“Bleeding” transitions

As I become wiser in the ways of the YouTube, I may later be able to upload something with better image quality. For now, though …

This is my first test of something I coded a while back, for cross-mutating between images. It should also be useable in video to fill in missing frames, or as an unusual transition between one scene and the next, though I haven’t tried it for either. It’s sort of like morphing, but not quite. It’s not a straight crossfade either. The best I can describe it is that the dark and light areas “bleed”, expanding or contracting as necessary until they become the next picture. It can happen over as many or as few frames as you wish (the key frames in this example are all spaced apart differently).

Who’s down with .CPP

As an amateur programmer with the organizational skills of your typical guitar-beating neo-hippie, eventually one accumulates so much code that one starts to forget where to find such-and-such neat trick that actually worked once upon a rainy day. So I figured I’d better go through my files now and write up a little reference guide for myself, while I still have any mental faculties at all. (Sorry if the indentation makes you seasick.)

It’s good to see that most of what I found in my old brainstorms has actually been done since then. What’s a little unnerving is finding some older stuff, literally hundreds of lines long, and not having the faintest idea what it is or what it’s supposed to do. COMMENTS ARE FOR PUSSIES. Uh huh. Keep telling yourself that.

Amidst much old and embarrassing garbage, buried deep in a sub-folder named simply OLD, one lonely and neglected project called tunnel sparkled off my eye like the edge of a favorite old toy in an attic. It felt bittersweet to look it over and recall how much love I had invested in it. It was an ambitious way to distort perspective and create the illusion that you were going “through” an arbitrary series of still images. Each aspect of the project was a project unto itself. It’s not that I couldn’t revive it, but it seems unimportant now, like that idea you had for the ultimate band in seventh grade.

Bleepies!

Here are some BLEEPIES (click to listen) for your chin-stroking, “that’s innn-teresting”-type listening pleasure.

And here is a screenshot of the Commodore emulator I generated them on:

The reason it matters is because I actually did these same bleepies on a real C64 twenty years ago, as a way of “modernizing” the rhythm on Mana. I just never recorded it. Or saved my program. But what you see above is pretty much functionally identical, and it gave me great pleasure to resurrect it.

I used that Commodore 64 a lot when I was a kid. I did a couple of other sequenced songs, entirely in BASIC, using DATA statements to list all the pitches it would play, and empty FOR-NEXT loops to create pauses. Anything you wanted to do on the SID chip, be it pitch, volume, filters, envelopes, or waveforms, you had to use POKE statements, which made the code entirely unreadable by the human eye. You could even program it to behave like a real-time synthesizer, playable via the computer keyboard, as long as you were willing to play every note a little early to compensate for the lag. (I once tried bringing it along to band rehearsal, but it wasn’t terribly practical.)

So anyway … I’m not ruling out using this again, or possibly some friendlier software that emulates only the sound chip without forcing me to stare into the above headache-inducer. Stay tuned!

This is your brain on drums … any questions?

Lennon was dead. Bonham was dead. Floyd were primed to split.

But worst of all, The Drum Machine had just signed its multi-million dollar contract, and one way or another you were going to have to confront it.

No musician in his right mind could have anything but a love-hate relationship with the thing. It was death knell and panacea rolled into one, in a portable black box. It forced you to re-examine whether or not there was such a thing as ethics when it came to making music, and the already fuzzy line between right and wrong became exponentially more elusive.

In a way, things are better — I daresay cooler — now, because you no longer have to defend and justify technology. It just is. And it exists in harmony with instruments and talents that took centuries to evolve, secure enough in its own power that it no longer needs to bare its teeth at the old geezers. You no longer choose between black (drum machine) and white (real drummer) — your pallet is an infinite gradient. Electronic sounds can be mapped to organic performances, and vice versa. Quantization can be done in degrees. Sampling can be done at the individual hit level, or at the pattern level. Things can be combined and edited in infinite ways. And ironically, proficient drummers stand to benefit the most from the tools.

The diagram above is just an organized brainstorm of the various ideas I’d like to connect together regarding drum tracks. A lot of the components have already been successfully coded. In particular, a way of mapping live human performance to a smooth list of tempos — see the upper left — has come a long way, baby. This is good for when I want to start the recording with an “unplugged” performance, and layer it up from there. This gives me my GRID.

I’ve also had some success with generative pattern mutation. Rescuing the kick and snare from dead repetitition, yet establishing ground rules to keep them out of the bullshit zone. (Example: three fast kicks in a row is bullshit. It screams “I have a drum machine and I don’t know what I’m doing.”)

Immaturity is just an idea at this point. It’s about deliberately inducing timing inaccuracies — not on a random per-note basis (that would just be “sloppiness”), but rather mimicking the subtle stretch-and-squash that happens when humans switch from one note duration to another. All the while, of course, paying reasonable mind to how far thou shalt stray from THE GRID.

This chart will undoubtedly grow more tentacles as I go along.

As for musical uploads, I’m still waiting for my old computer to talk to my new computer. And then we will have fun.