The application of skeleton animation in 3D games is becoming more and more common, and its standard mode is to use forward kinematics. In general, animators use animation software to make a series of animations required by the project in advance, and then when the game is running, the program should decide which animation to display according to logic and calculate the specific display according to time. This way of pre-made animation clips brings about a big problem: all animations are immutable. To make the game as realistic as possible, the animations that our game may need are rich and changeable, making all these animations may be unacceptable in terms of cost and time. To overcome this problem, the game industry introduced the technology of motion capture.
Regarding mocap, Jonathan gives a very comprehensive analysis in the eleventh chapter of his book. The following is the content of the book I intercepted
GAME ANIM: VIDEO GAME ANIMATION EXPLAINED – by Jonathan Cooper
Chapter 11
Arguably, the single largest innovation in game animation in the last few decades has been the widespread adoption of motion-capture (mocap for short)—the process of capturing motion from live actors.
Much was said about mocap in the early days along the lines of “It’s cheating,” “It’s not real animation,” and “It’ll replace animators and we’ll lose our jobs,” but one must only look a decade earlier to see the same fears vocalized by animators regarding the shift from 2D traditional animation to 3D computer animation. The idea then that a computer character could have the same life as a series of artfully hand-drawn images was incomprehensible to many at the time, but later proved possible when handled by talented animators, and the same is true now of mocap.The simple fact is that as video games have matured and their subject matter moves from simple cartoonlike characters and actions to more realistic renderings of human characters and worlds, the old approach of keyframing humans simply wasn’t cutting it visually, not to mention the sheer volume of animation required for a fluidly moving character with all its cycles, blends, and transitions would simply be impossible to create any other way.
That’s not to say that mocap can’t be a crutch, and when wielded incorrectly, the results are far from satisfying. There are some in production still who incorrectly believe that we shoot mocap then implement in the game and the job is done. A search for a “silver-bullet,” one-stop solution to capturing the subtleties of acting is ongoing, though this technology is merely yet another tool for animators to wield in their quest to bring characters to life. Mocap is but another method to get you where you want more quickly, and the real magic comes when a talented animator reworks and improves the movement afterward.
Do You Even Need Mocap?
For our hypothetical project, long before getting into the nitty-gritty of motion-capture production, a very important question to be asked should be whether to even use motion-capture or stick with a traditional keyframing approach. Here are some considerations to help answer this question.
1.What is the visual style of the game? A more realistic style benefits greatly from mocap, whereas mocap on highly stylized characters can look incorrect. Cartoony and exaggerated motion will help sell characters that are otherwise lacking in detail and visual fidelity, including those seen from afar. Mocap makes it easier to achieve realistic character motion.
2.Are our characters even humanoid? While some games have been known to mocap animals, the approach is typically used only for humans. If our main characters are nonhuman creatures or nonanthropomorphic objects, then mocap often isn’t even an option.
3.What kinds of motions will feature most in the game? If the characters are performing semirealistic motions such as running, jumping, climbing, and so on, then mocap will suit, whereas if every move is expected to be outlandish or something that no human could perform, then keyframing might suit better. The balance of these actions should determine the project’s adoption of mocap.
4.What is the scope of the game? Mocap gives the best value for the money when used on large projects with lots of motion, at which point the production cost of setting up a mocap shoot and the required pipeline is offset against the speed at which large quantities of character motion can be created. That said, cheaper yet lower-quality solutions are also becoming more readily accessible for smaller projects.
5.Would the budget even cover it? While affording an unparalleled level of realism, motion-capture shoots can be expensive. When compared to the cost of hiring additional animators to achieve the same quantity of motions via keyframe (depending on volume), the cost sometimes becomes more comparable, however.
6.What is the experience of the team? An animation team built over the years to create stylized cartoony games may take issue with having to relearn their craft, and attempts to adopt mocap may meet resistance. That said, motion capture does become a great way to maintain a consistent style and standard across animators.
How Mocap Works？
While not absolutely necessary for great results, an understanding of the mocap process will only aid the game animator in finding ways to speed up the pipeline and get motion capture in the game faster and to a higher quality in as little time as possible.
Different Mocap Methods
Optical Marker–Based
While there are several alternative motion-capture methods, the traditional and most commonly used is via the triangulation of optical markers on a performer’s suit, captured by arrays of cameras arranged around a stage so that they create a “volume” within which the performance can be recorded. This provides the highest quality of motion-capture but is also the most expensive.
These arrays of cameras can number anywhere between 4 to upward of 36, and highly reflective markers are tracked at higher frame rates than required by the game project (typically 120 frames per second). As long as no fewer than three cameras can simultaneously follow a marker, the software simulation model will not lose or confuse the markers for one another. When this does happen, the cleanup team (usually provided by the stage) will manually sort them again.
Accelerometer Suits
The performer dons a suit with accelerometers attached, which, when combined with a simulated model of human physics and behavior, provide data without the need for a volume of cameras. However, unless the animation team is prepared to work longer with the data, the results are far from the professional quality provided by marker capture. Accelerometer mocap is therefore useful for lower-budget projects or previz before real captures for larger ones. Depth Cameras A third and experimental approach is to use depth-sensing cameras with no markers, applying body motion only to a physical model. This provides the cheapest option of all, and has created interesting results for art installations that deal with more abstract representations of the body.
Depth cameras
may provide some decent reference-gathering and previz but are ultimately less than ideal for an actual videogame project due to the amount of work still required to make it visually appealing post-shoot. That said, the quality of all options is increasing at an encouraging rate.
Performance Capture
Perhaps the biggest breakthrough in increasing acting quality in recent years has been the introduction of performance capture. While motion capture refers to the recording of only the body, performance capture records the body, face, and voice all at once by using head-mounted cameras (HMCs) and microphones. Doing so adds a level of continuity in subtle facial acting that was simply impossible in the previous method of recording everything separately and recombining in a DCC.
While this method has become ubiquitous with cinematic cutscene shoots and head-cams are becoming more affordable, care must be taken to ensure all three tracks (body, face, and audio) remain in sync. As such, the mocap stage will generally provide time codes for each take, which must be maintained during the assembly and editing phase.While it used to be a requirement for real and virtual actors’ faces to match as much as possible in order to best retarget the motion to the facial muscle structure, current methods employ retargeting to digital doubles first, then translate to the desired face of your chosen video game protagonist, though the extra step naturally makes this process more costly at the benefit of freeing you up to cast the best actors regardless of how they look.
Due to the extra overhead of camera setup and calibration, cinematic shoots typically go much slower than the often rapid-fire process of in-game shoots, not to mention the reliance on scripts and rehearsals and multiple takes to get the action just right. While rehearsals can certainly benefit in-game shoots (most notably when requiring choreography such as during combat), they are less of an absolute necessity as is the case with cinematic cutscenes. The secret to successful cinematics isn’t a technology question, but ensuring the performance (and writing) are as good as possible. There’s only so much an animator can polish a flatly delivered line of awkward dialogue from a wrongly cast actor.
For the remainder of this chapter, we’ll be focusing on the first approach, (optical marker-based), as this is the most commonly used method at video game studios and therefore the most likely one an animator will consistently encounter in his or her career.
The Typical Mocap Pipeline
The typical workflow for optical marker–based mocap involves:
1.The actor arrives and, once suited up, is calibrated into the system matching his or her height and size (and therefore unique marker positions) with a character in the capture software.
2.The actor is directed on a stage to capture the desired motion. Either then, or later via a viewing software, the director decides upon which takes (and sometimes frame ranges) he or she wishes to purchase from the stage. See the “Directing Actors” section later in this chapter for some best practices when directing actors.
3.The stage crew then clean up the motion by fixing lost markers and smoothing out extraneous jerky motion due to marker loss or interference. This process can take anywhere from a few hours to a few weeks. The cleaned-up motion is delivered to the game studio as “takes.”
4.A technical animator at the game studio retargets the delivered data onto their in-game character, checking the quality of the delivered mocap and requesting redeliveries if the quality is off.
5.The animators then begin working on the mocap that is now driving their game characters. This usually consists of a mocap rig and control rig both driving the export skeleton, allowing the animator to trace motion back and forth and work in a nondestructive manner, adding exaggeration and appeal without destroying the underlying motion that has been bought and paid for. For details on this stage, the most involving step for the animator, see the section “Working with Mocap” at the end of this chapter.
Mocap Retargeting
Because actors rarely match the dimensions of the video game characters they are portraying, the studio must incorporate retargeting of the motion from the delivered actor-sized motion to the game character. There are a variety of settings to get the best possible translation between characters without issues that are difficult to fix at a later stage.
This process is generally performed in MotionBuilder, and the single biggest potential issue to be wary of is the use of “reach” to match limbs’ captured positions regardless of the difference between the source actor and game character.
Used generally for the feet to ensure they match the ground and prevent foot-sliding, reach can also be used for hands when it is essential they match the source position, such as when interacting with the environment like grabbing onto a ladder. However, leaving reach on for arms in general can be disastrous, as the hands will always match, causing the arms to bend or hyperextend unnaturally to maintain the source position.
At this stage the person tasked with retargeting should also keep an eye out for individual bad retargets or jerky motion where lost mocap markers weren’t correctly cleanup up, systemic issues that plague every delivered motion such as bent clavicles or spines, or loss of fine detail due to smoothing applied as default to all motions by the mocap studio.
Mocap Shoot Planning
The absolute worst thing an animator can do is arrive at the stage on the day unprepared. Here are several essential practices that will ensure as smooth and productive a shoot as possible.
Shot-List
A shot list is invaluable both on the day of shooting and in the run-up to the shoot, as it’s the single best way to evaluate everything you’ll need to capture, so it helps plan out the shoot. While the mocap stage will often provide their own formatting as they require a copy before the shoot day for their own preparation purposes, you can make a start yourself within Excel or Google Docs. Any shot list should contain these following columns:
•Number: Helps to count the number of shots and therefore make a time estimate.•Name: Shots should be named as per your file naming convention.
•Description: A brief explanation of the desired action lest you forget.
•Props: Which props will be required by the actors—even if not captured.
•Character: Essential for multicharacter shots for retargeting purposes.
•Set builds: Similar to props but rather walls, doors, and so on actors will be interacting with.
•Notes: Added on the day as required, describing directing notes such as preferred takes.
Ordering/Grouping Your Shots
There are a variety of factors that will determine the ordering of mocap shots, not least being the priority in which they’re needed for the game schedule to ensure work isn’t delayed should you fail to capture everything, which happens often. In addition, grouping multiple actions requiring the same set builds and props (especially if the props are captured) will ensure a good flow onstage.
Perhaps the largest time sink on any shoot day is the building of sets, so great consideration must be taken to try to capture everything required on one set build before it’s dismantled. It is wise to avoid capturing high-energy actions at the end of the day (or even right after lunch), as the actors will naturally be more tired then. Conversely, start the day with something fun like fast and easy rapid-fire actions that will build momentum and set you up for a great day of shooting.
Rehearsals
The most proven way of having an efficient mocap shoot while obtaining the highest quality of acting is rehearsing beforehand. Not only is it a great way to build a relationship with the actors, it also allows them to go deeper into any performance. While not needed for many in-game actions that can be quickly done on the day, rehearsing is essential for cinematic or story shoots that require deeper characterization and will likely have closer cameras that can highlight detail, especially for full facial performance capture. Something as subtle as a thoughtful pause or a tilt of the head can make the world of difference to a performance when an actor is fully invested. Giving them time to know and understand the characters they’re portraying is impossible without rehearsing.
Mocap Previz
Another excellent way to avoid costly wasted time on set (or afterward when you find the actions just won’t fit the scene) is to previsualize the actions before even going to the shoot. This way, not only can progress be made on the various gameplay and story scenarios without having to wait for shoot day to roll around, you’ll already have figured out many of the technical issues from within simple cost-effective scenes, such as how much of the set you’ll actually need to build based on touch points.Importantly, understand that the actors will likely improvise and suggest much better actions than your previz on the day of shooting, so it should be used as a guide and initial inspiration only, giving the director a better evaluation of what will and will not cause technical issues when new suggestions arise.
Working with Actors
While it can be physically exhausting to shoot mocap, sometimes over a series of days, working directly with actors can be one of the most rewarding aspects of video game animation as you work together to bring your characters and scenes to life. The improvisational collaboration, sometimes to an intimate level as you work together to create the perfect performance, can be a refreshing break from working at a computer screen.
CASTING
Props & Sets Props
are an essential part of any game mocap shoot, usually taking the form of weapons or other gameplay-related items. Importantly, props give actors some tangible “business” (something to play with in their hands or touch in the environment) to work with in cinematic story scenes and make the scene more natural.