Mike Sherwood describes how to get the best out of iClone’s powerful new facial animation tools
This tutorial will take you through the many tools and techniques available for facial animation in iClone 7 – including lipsync, keyframing, puppetry, and mocap – as well as timeline editing. I’ll also describe ways to mix the different techniques to improve workflow, as well as how to fix and refine animation at any point in the timeline.
Working purely in iClone 7 with a pre-built character creator model which has been additionally polished in Photoshop and Sculptris, the tutorial will apply to any iClone/ CC characters or any non-standard human characters which have been brought into iClone and correctly set up for facial expressions via 3DXchange.
‘Out of the box’, iClone 7 provides a uniquely comprehensive toolset for facial animation which can be used at many levels – from rapid ‘blocking in’ of expression keyframes through to subtle face puppetry and motion capture.
Probably the most important thing to understand is that these techniques can be used separately, or together – depending on the results you’re aiming for. You can produce decent animation extremely quickly just by using one of the tools – or you can spend time combining different tools and refining the animation, right down to the nervous twitch of a character’s lip or eyelid.
For more information about iClone and other Reallusion products, visit http://www.reallusion.com
01 UNDERSTANDING THE TIMELINE
All facial expression animation is handled on the character’s expression timeline. This has four tracks – a clip track at the top, then muscle, eye and head keyframe tracks respectively. Select and drag clips or keys to move them, or select and right-click to edit. Check ‘Loop’ or ‘Speed’ on the timeline toolbar to loop, speed up or slow down a clip by dragging its right-hand boundary.
02 EDITING EXPRESSION CLIPS
Adding keys or recording animation to an empty timeline creates a new expression clip. Right-clicking on the clip gives the usual edit functions, including ‘Break’, along with ‘Flatten’ and ‘Sample’ clip commands. ‘Flatten’ embeds all visible keys into the current clip, so you can add more keys cleanly, whilst ‘Sample’ generates keys on every frame. ‘Break’ auto-samples, so for clarity, I recommend flattening immediately before or after breaking.
03 PERFECTING AUTO LIPSYNC
Use Create Script on the Modify > Animation panel for auto lipsync methods, which load audio and viseme keys into the viseme track. Keys can be moved, also edited via the right-click menu. Left click on a key or a track to open the Lips Editor – to replace or create a key, and to adjust the strength. Select a range on the Lip Options track and right click for global adjustments.
04 FAST FACE KEY EXPRESSIONS
The Face Key Expression Panel is a library of expressions – use the top right drop-down list for different sections. Build facial animations in seconds simply by scrubbing along the timeline and setting keys by double-clicking on the desired expressions, To better establish expressions for more naturalistic animation, set a particular expression key, then set it again a little further along the timeline before moving on to the next expression.
05 USING THE EXPRESSIVENESS SLIDER
Use the expressiveness slider to make expressions more subtle if required. You can also set expressiveness keys by adjusting the slider between existing keys. This control is present on all Face Key Panels and has a global effect, so if you’ve used expressiveness and want to continue setting keys in the same timeline area without the current value affecting the new keys, flatten the expression clip before adding new keys.
06 REFINING WITH MUSCLE KEY
You can build up complete facial expressions and animations using the Muscle Panel alone, but it’s especially powerful when adjusting and refining existing expressions, and between keys. Single or multiple parts of the face, as well as the head, eye, and jaw rotation, can all be selected and manipulated simply by left clicking, holding and dragging the mouse in different directions from the black background area of the feature selection panel.
07 DEFAULT KEY AND ZERO KEYS
The Muscle Panel Default Key button sets muscle, eye and head keys on the current frame to the default clip state. This will be neutral if the clip has contained no flattened or recorded animation. You can also set keys in the muscle, eye and head track simply by double-clicking in empty frames – use these to establish the current timeline state, eg before/ after applying expression or rotation keys.
08 ACCESS ALL AREAS WITH MODIFY
Modify Panel sliders give you access to all underlying morphs, as well as jaw rotations used by the Expression and Muscle Panels. You can see the sliders in operation as you play or scrub through a timeline with keys in place. Use the sliders for precision fine-tuning of existing keys, as well as between keys. Extremely useful for error correction, modify sliders also accept negative numeric values if needed.
09 FACE KEY WORKFLOW
You could build up an animation solely using modify sliders – but this would take ages! It’s far quicker and easier to start with blocking in expressions, then refine using Muscle keys and finally Modify sliders. If you find that the clip is becoming cluttered with keys, or expressiveness values are limiting what you want to do – simply flatten the clip and continue adding and editing keys as before.
10 FACE PUPPET – MOCAP YOUR MOUSE
Face Puppet lets you drive facial animation using your mouse: it translates mouse motion into full or part-face expressions or combinations of these depending on what you have selected. It’s important to preview before recording to get a feel of how far you need to go, and in which directions – to get the animation you want. Use the strength slider to increase or decrease the effect.
11 FACE PUPPET RECORDING
Press enter to record at half speed, but combine this with ‘By Frame’ (click the ‘Realtime’ button to toggle) on the play bar to go slower still for even more control. You can blend animation by recording over an existing clip with ‘Blend data on next recording’ checked – with this unchecked, it will overwrite the previous clip. You can blend or overwrite full or part face expression recordings.
12 FAST FULL FACE PROFILES
Think of Full Face Profiles as the Face Puppet equivalent of Expression Keys – only dynamic. You can create animations very quickly by simply moving the face into and out of the expressions you want at appropriate points on the timeline. There are many options available, from character specific profiles to generic, universal profiles for emotions, with lips closed (ideal for use during lipsync) and lips open options.
13 HEAD MOTION PUPPETRY
Character profiles have head motion active by default – you can toggle head turns and tilts on and off using the buttons on the top left of the Solo Feature Selection window. Use associated head motion for quick animation eg previz, but keep it off for more precise face puppetry and add head motion in a separate pass either via Face Puppet, Face Key and/ or using Edit Motion Layer.
14 THE EYES HAVE IT
iClone’s auto-blink function is ideal for previz, but for control turn this off on the character’s main Modify Panel, and create blinks yourself. Subtle tweaks of the eyelids, when associated with focused eye motion – can have dramatic effects even without using additional parts of the face. For eye motion without mocap – use Face Puppet or set a key to indicate a particular focal point, then set this again a little further along the timeline for establishment before changing eye rotation.
15 REFINING WITH SOLO FEATURE
Use the eraser button to deselect the current profile, then select a solo feature and you’ll see a number of thumbnail options below the selection head. These are profiles which drive different expression variations for the selected part. Use them separately to drive an individual feature to refine an existing animation – or combine them by selecting multiple features and profiles – also mix them with Full Face Profiles if you wish.
16 FACE PUPPET WORKFLOW
I generally think of Full Face Profiles as a broad brush, whilst solo features are for more delicate work, but there are no hard and fast rules. Whilst you can block in whole sections of Full Face Profiles before refining with Solo Features, you can also work much more subtly from the start – for example, by doing a pass of eyebrow motion, then mouth, then eyes etc.
17 WORKING WITH PUPPET CLIPS
Recording a pass of Face Puppet generates a clip on the expression track in the same way that expression keys create a clip. All clips can be moved, edited, looped or speeded up/ slowed down. When blending (blend checkbox selected) Face Puppet passes, you can start at any point, and the new clip which is created will mix seamlessly with any existing clip already present in the same timeline area.
18 FACIAL MOTION CAPTURE
The new iClone 7 Facial Mocap Plugin opens the door to true facial motion capture from established third-party developers. Starting with Faceware camera-based technology, facial mocap gives instant results based on tracking the user’s own facial expressions, making facial animation easier and more straightforward than ever. Mocap recording creates a clip on the expression track as usual and can be blended with existing keys and/ or puppet clips.
19 SETUP AND CALIBRATE
Even lighting and good camera setup are important for mocap – the user’s head needs to be framed centrally and straight on as if taking a passport photo. Calibration matches your neutral expression with that of your character. You can use calibration to tweak animation results – eg make your eyebrows higher during calibration to make low brows character animation more effective, make your mouth smaller to increase wide mouth effects etc.
20 MASKING AND SETTINGS
Mask off parts using the Mask Select head for multi-pass animation: you can record just eyes, mouth or head – or whatever combination, and use the ‘Record Blending’ checkbox to choose whether to blend or overwrite the previous pass. Use strength sliders to exaggerate or reduce the relative degrees of animation, as well as smooth to reduce noise/ jitters. For lip sync, record audio simultaneously using the ‘Audio recording’ checkbox.
21 COMBINING THE TOOLS
You can mix iClone’s facial animation approaches in any way, and in any order – but it makes sense to consider the best workflow, especially lipsync – since adding lipsync later to an animation which already includes the lips or jaw opening can be tough to resolve. So if the animation includes speech – lipsync first, either by auto lipsync or during mocap – before moving on to use other tools.
22 EXAMPLE WORKFLOWS
Again, no rules – but possible workflows could be:
A) auto lipsync (including manual viseme refinement) > face key expressions and/ or face puppet full face blocking > face puppet solo feature and/or face key muscle/ modify refinement
B) facial mocap > if needed, blend in lip options filtered auto lipsync to audio recorded during mocap (particularly useful for adding tongue motion) > face puppet solo feature and/ or face key muscle/ modify refinement.
23 POLISH AND FIX
Once your animation is blocked-in, it may already be time to stop if it’s for previz. But if you’re producing finished animation for production – it’s time to start polishing and fixing. This is all about iteration – repeatedly playing and scrubbing through the timeline to check for areas which need improvement. Use Face Key Muscle and Modify for fine tuning, combined with timeline editing, for final animation fixing and polishing.
24 CLIP PROBLEM SOLVING
To reset/ zero animation for all or part of a clip, use Face Puppet with ‘Blend data…’ unchecked and record without mouse movement – to zero individual features, eyes, head or all animation. Mocap with masking/ zero strength recording and no blending does the same. To clear issues without reset, break and delete the problem clip area and either rework it or simply use clip transitions to blend between the separated clips.
25 MODELLING FOR ANIMATION
A character’s face may look great at the start when static but could appear blocky with vertices bunching unnaturally during the animation, particularly at the mouth, eyelids, and brows. This is due to modeling: for best animation results try to keep the facial mesh as balanced as possible, not simply modeling the 3D shape for appearance – but maintaining good vertex positions which relate well to the default morphing mesh for animation.
26 LESS IS MORE
Finally, with so many tools and possibilities, it’s all-too-easy to overdo animation. So try to keep your focus on what the character is communicating, and what’s needed to clearly get that message across. Good animation can speak volumes with the simple raising or lowering of an eyebrow, and extreme expressions are generally reserved for comedy and extreme situations. Please do bear these points in mind when creating animations.
Tip 1: EASY TIMELINE NAVIGATION
Use Alt key with the mouse wheel, or plus/ minus keys to zoom in/ out of the timeline.When refining lipsync, use the audio waveform for reference whilst precision editing
Tip 2: CONTROL YOUR WORK AREA
Drag the timeline start and finish markers to bracket work areas into manageable chunks, as well as to limit clip creation ranges for more precise animation recording
Tip 3: STOP AND CRITICS ARE GOOD
It’s easy to lose objectivity when animating – so take a break, get some feedback from others, and stay open to criticism: it’s all about improving results for an audience!
Mike Sherwood (aka 3Dtest)
has worked in CGI for over 20 years, with clients including broadcast, games and software companies. Specialising in character modeling and animation, he is currently assisting Reallusion with iClone development.
Visit his youtube channel