Guitar Hero Logo
Kai Huang
RedOctane co-founder, President and CEO, Kai Huang. Image courtesy of RedOctane.
Alan Flores
Guitar Hero producer for Neversoft, Alan Flores. Image courtesy of Neversoft.

Sure, having the arcade game you helped pioneer in the U.S. parodied on Malcolm in the Middle is cool (Malcolm's dad battles a young fanatic for "Jump, Jump Dance Party" supremacy—"Dance, Dance Revolution's" doppelganger). But RedOctane co-founder, President and CEO Kai Huang knew he'd really arrived when South Park riffed on Guitar Hero, the pop cultural phenomenon he and his brother, Charles, helped co-create.

In the South Park episode, "Guitar Queer-o," Mr. Kincaid (evidently, his Partridge Family royalties have run out), signs a smokin' Stan to a contract (saying "You could earn over a million points!"), but, in an effort to ease the pressures of impending rock stardom, Stan spirals down into a "Heroin Hero" addiction—a sly nod to the addictive nature of Guitar Hero itself.

The latest installment of the game, Guitar Hero World Tour, features expanded peripherals—including drums and a microphone—as well as a potential game-changer: the GH Music Studio, which enables players to compose original music (using the GH peripherals) and upload to GHTunes, a user-generated content hub, for fellow enthusiasts to download and play. According to Kai, he and Charles helped usher in the "rhythm game" revolution because they wanted everyone to be able to rock in front of adoring throngs, irrespective of innate ability. Both Kai and lead Guitar Hero producer for Neversoft, Alan Flores, are hopeful that the GH Music Studio will unleash every player's latent songwriting ability.

Neversoft, as you may recall, was betrothed to RedOctane following its split from Harmonix and acquisition by Activision in 2006. Harmonix, in turn, ran off with MTV Games and spawned Rock Band in 2007. And, while Rock Band was first out of the gate with an expanded peripheral set (guitar, drums, and vocals), GH intends to catapult over its rival with the GH Music Studio feature. "You compose through the guitar peripheral," explains Flores, which now boasts a bigger body, detachable neck, longer whammy bar, and slider.

Peripheral Vision

"We tested every conceivable combination," says Kai of the GH guitar. "Three buttons, four, six, and seven." Five seemed to offer the perfect level of complexity (and simplicity) for newbies and enthusiasts alike. With five buttons, so goes the theory, your hand has to slide up and down the fret board, like a real guitarist. They also tested other interfaces, but the rock 'n roll highway UI won out in the end.

"For World Tour, we worked on our peripheral kit, including drums and a microphone, for over two years," says Flores, who relied on pros like Travis Barker (Blink-182) and Stewart Copeland (The Police) to enhance the level of verisimilitude. "Our drum kit has a nice bounce," Flores contends, "and they're velocity-sensitive, so if you hit harder, you get extra points." In addition to the drum kit, the World Tour bundle boasts a professional-looking USB microphone.

Travis Barker
Blink-182 drummer Travis Barker helped developed World Tour's drum kit peripheral. Image courtesy of RedOctane and Neversoft.
Brian Bright
Guitar Hero project lead, Brian Bright. Image courtesy of RedOctane and Neversoft.

Project Lead Brian Bright says Neversoft developed a proprietary game engine for Guitar Hero III: Legends of Rock in anticipation of the Xbox 360 and Play Station 3 platforms in 2002. "Our team took six months off to develop it and, at the time, it was next generation," Bright says. But, technical advances come so fast and furious in the world of game development, it's now current—and probably on its way to defunct. The core functionality of a game engine typically includes rendering for 2D or 3D graphics, a physics engine (or collision detection) and collision response, sound, scripting, animation, artificial intelligence, networking, streaming, memory management, threading, and a scene graph.

Most game makers re-use or license existing game engines, then rely on middleware to augment the engine ("middleware" are software suites with more robust specific capabilities than the core engine). "For instance, we used Havok middleware in Guitar Hero for hair and what we call 'accessory bones,' or physics-based objects that sway with the characters," says Bright. Much of Guitar Hero (and Neversoft's other monster franchise, Tony Hawk) is frequently implemented first in script "to put powerful tools in the hands of designers like Flores," notes Bright, "but then it's coded properly so it runs fast and is optimized for the game." Despite quantum leaps in processing power on the part of manufacturers like Microsoft and Sony, they're always up against the limitations of the console's CPU.

Take Note

Of course, it all starts with a notation process that is never easy, even for a simple, three-chord scorcher like "Anarchy in the U.K." by the Sex Pistols. "First, we get the master tapes," Flores explains. "Often, the artist just hands over the Pro-Tools sessions and we get the stems" (the individual components of a master mix). The heartbreaker is when the master tapes are literally that—analog tapes that have decomposed and degraded, or been lost altogether. "We hear horror stories," Flores laments, "of band members who sold master tapes in exchange for a thousand bucks, or are rotting in someone's basement."

"But once we've got the stems," Flores continues, "we tempo map the song, and start sequencing the stems into harmonic progressions that map to the guitar peripheral's buttons." There was an attempt at automating this notation process, and various scripts were written, but something always got lost in translation. "We dubbed the notation program Murder Face, because it just murdered the notation," Flores laughs. "There is no software that can match the human ear for polyphonic chord detection," Bright agrees. "We're waiting for some university research department to crack the code."

For the first time, vocal tracks were added to Guitar Hero, and Bright says they're easier to notate than guitar. "With our guitar, we have only five buttons to map myriad chords and notes," he explains, "and, once we have the 'expert' notation, we have to interpolate for 'hard,' 'medium,' and 'easy' settings." While notation may be easier for voice, the team was in virgin territory with vocal recognition. "We had to develop pitch recognition software, and that came with a whole host of issues," Bright reports. (Music recognition is mathematical analysis of an audio signal and its conversion into musical notation—usually in MIDI format—a very difficult artificial intelligence task.)

"And, we had to make sure the vocal recognition software operated quickly enough, didn't hinder the CPU, and could distinguish two vocalists singing at the same time." The company developed specialized algorithms to interpret the incredible range of the human voice. "When we're happy with the notation," says Flores, "we note-track the button presses, and add cues for lighting and pyrotechnics."

Capturing the Avatar

Of course, the coolest thing about Guitar Hero is that you're not just hitting notes by following a bouncing ball; you're playing keg parties and dive clubs, wending your way up the rock and roll food chain till you're selling out massive arenas. Well, not you, but your avatar. "With the development of Guitar Hero III for Xbox 360 and PS3, we really ramped up our avatar development with motion capture," says Bright. "In fact, we built a mo-cap studio and hired a motion capture team with a cinematic director."

Motion capture, or performance capture, is the process of recording the actions of human actors and using that data to animate digital character models in 3D animation environments like the game engine or Maya. The performers are outfitted with skin tight "flak suits" (they're used to it, what with all the Spandex®), covered with little light balls that capture their actions—singing, strumming, strutting, smashing guitars—on infra-red cameras. "We have thirty digital cameras capturing the action from every angle," says Bright.

Highslide JS
Neversoft built a motion capture studio to record rock stars such as Sting, Smashing Pumpkins' front man, Billy Corgan, and Aerosmith's Steven Tyler. Clip courtesy of RedOctane and Neversoft. (Click to play.)

The camera movements themselves can also be motion-captured so that a virtual camera in the scene will pan, tilt, or dolly around the stage driven by a camera operator, which means the computer -generated characters, images, and sets will have the same perspective as the video images from the camera. Developers use "match move" software such a boujou, to composit, or digitally "blend," this live action and virtual footage.

"To capture more intricate facial mannerisms and expressions—snarls and sneers—we put dots, or markers, all over the face, and position ten to fifteen cameras around the performer at close range," Bright explains. An object with markers attached at known positions is used to calibrate the cameras and obtain their positions. As long as two calibrated cameras see a marker, a three-dimensional fix can be obtained, "and the face gets mapped to the skeleton in the game," says Bright. During these video-based motion capture sessions, a real-time, "stick figure" version of the performer—a model for the 3D object—is projected to ensure the motion is being captured correctly.

Billy Corgan
Smashing Pumpkin's front man, Billy Corgan is immortalized in World Tour. Image courtesy of RedOctane and Neversoft.

The Guitar Hero avatars not modeled on real rockers such as Slash or Steven Tyler are modeled on one versatile performer. "He comes in, he can snarl like Billy Idol, or just give us a range of 'rock star' moves, and our avatars are so highly stylized, it works," claims Bright, whose team recently had the privilege of creating a Jimi Hendrix avatar for World Tour. "We hired the foremost Jimi Hendrix impersonator in the world for the mo-cap," says Bright, "and then used lots of reference material, period photographs and books, to get the specific texture (everything on the surface of the 3D object, such as hair and skin tone, eye color, etc.) exactly right." Adding to the pressure of accurately rendering an icon was the desire of the Hendrix estate for the guitar god to be depicted in a very specific way. "Early Hendrix, but not Woodstock Hendrix," clarifies Flores.

Three new Hendrix master tracks will be available for download in mid-November: the blues track "If 6 was 9," "Little Wing," and a live version of "Fire," along with tunes by the Raconteurs. The Raconteurs make their debut appearance in a videogame with three master tracks off their second album Consolers of the Lonely: "Salute Your Solution," "Hold Up" and "Consoler of the Lonely." And Guitar Hero is experimenting with what are known as "day and date releases"—master recordings released in Guitar Hero at the same time as the CD and digital downloads are available—from artists like Metallica and Coldplay.

Master Class

At first, artists were reluctant to release their master recordings to the game (a combination of the underlying publishing rights, as well as performance), and early Guitar Hero recordings were high-end covers. But, rock royalty relented once they witnessed the promotional power of GH, best exemplified by Dragonforce—the most popular mega-metal band you never heard of before their blistering "Through the Fire and Flames" became the song to beat in Guitar Hero III. (Just don't ask the band whether they can beat their own songs; the answer is "yes," but they are so over that question.)

For World Tour, the entire set list is original master recordings, including live numbers by Lynyrd Skynyrd and Jimi Hendrix. And, of course, Guitar Hero took the master recording concept one step further by creating a game based on the career of a single band, Aerosmith, for Guitar Hero: Aerosmith. "It was guitarist Joe Perry's son who convinced him to jump on the Guitar Hero tour bus," says Flores.

More single-artist focused versions of the game are in the works, but don't hold your breath for Jimmy Page or Robert Plant avatars any time soon. Page is notoriously protective of Led Zeppelin's master recordings, which have rarely been licensed for film or commercial use (One notable exceptions is "That's the Way" on the Almost Famous soundtrack—well, Cameron Crowe's quasi-memoir was a thinly-veiled roman a clef about his experience covering the band for Rolling Stone; other exceptions are "Immigrant Song" in School of Rock and that dreadful Cadillac commercial starring Cindy Crawford.). While acknowledging that Led Zeppelin would be a stunning "get," says Kai: "We're ready for players to create their own 'master tracks' in the GH Music Studio."

"Four players can jam together, making original music, then appoint an engineer," says Flores of the revolutionary, new the GH Music Studio module. Working with the guitar controller, the engineer can compose, arrange, and mix on a virtual mixing board complete with faders and volume and panning control. "Creating the interface was kind of tricky," Bright acknowledges, "since we were limited to the functionality of the guitar controller." Users can mini-navigate with the strum bar, and use the directional pad to control "snap" values as composers move through the musical timeline. "When you lay down your track, you can define your scale and tempo, and even rip the recorded song to MP3 or, of course, upload to GHTunes," says Bright, who is hoping that budding songwriters get their feet wet with the GH Music Studio, then jump into the Garage Band and Pro-Tools pools.

GH Music Studio
The GH Music Studio virtual mixing board. Image courtesy of RedOctane and Neversoft.

According to Bright and Flores, some 10,000 original songs have already been uploaded to GHTunes—YouTube sensation "WTF" is currently the most popular—for which Neversoft built a huge back-end data storage system. Asked if the company is concerned about players "composing" and uploading songs they're yearning to play in Guitar Hero—like the Led Zeppelin cannon—Flores insists the content will be closely monitored for copyright infringement. "No, we're not spidering, like Google with YouTube," he admits, "because the audio imprint won't work with users' interpretations of copyrighted material." Instead, users will be asked to flag questionable content, and monitors will troll the site for that illicit version of "Stairway to Heaven."

"More than anything, we're looking to make Guitar Hero a conduit of social connection," says Kai. "Friends and families can jam together, and make music together, then share it online." Like South Park's Stan, who eventually kicks his "Heroin Hero" addiction to reunite with Kyle and beat the game. Then it's Cartman and Butters' turn to get hooked.