Outline a mask

Wondering how to accomplish a certain animation task? Ask here.

Moderators: Víctor Paredes, Belgarath, slowtiger

User avatar
mkelley
Posts: 1647
Joined: Fri Nov 02, 2007 5:29 pm
Location: Sunny Florida
Contact:

Outline a mask

Post by mkelley »

I'm beginning to believe this isn't possible, but perhaps you gurus can tell me something I'm overlooking.

I'm playing around with masking, particularly as it applies to eyes and mouths. Take, for example, a situation where you have a pupil on one layer. It would be nice to use the basic eye shape to control the mask on the pupil, so that as the eye narrows, for example, all you need do is pull down the eye shape and it will hide that portion of the pupil

This does indeed work, except that you get no outline around the mask, which kind of defeats the whole point of this. Oh, you can get an outline at the edges where the mask doesn't touch any masked features (say around the bulk of the eye) but at the pupil it actually masks it won't continue the line. To me it should either give you a line or not, not depend on whether it is actually masking an object.

Yet this seems like such an insanely useful feature it's too bad it doesn't work this way (it would make doing mouth shapes easy -- just define the back of the mouth, with tongue and teeth, and control only one shape as the mask for the lips).

Am I correct in thinking you can't get there from here? A good item for the wish list?
Genete
Posts: 3483
Joined: Tue Oct 17, 2006 3:27 pm
Location: España / Spain

Post by Genete »

You can do it just :
  1. Being sure that the mask layer have any outer outline.
  2. After finish the animation duplicate the mask layer and place it over all the layers of the group.
  3. Set the mask options to "Don't mask this layer"
  4. Delete all the shapes and create new ones but only for the outlines.
It should look like this:

Image
User avatar
mkelley
Posts: 1647
Joined: Fri Nov 02, 2007 5:29 pm
Location: Sunny Florida
Contact:

Post by mkelley »

Well -- that really won't work for what I have in mind.

I did indeed think about all that, but the idea is just to create mouth shapes by moving the mask points, and having to then duplicate the layer, create new shapes, etc. etc. -- might as well not do it in the first place (and you certainly can't animate this the way you can simply by moving the points on the mask).

There really ought to be this capability, though. I don't hold out much hope for a 6.0 AS anytime in the next year, but I'll at least add it to the wish section.
rplate
Posts: 257
Joined: Fri Oct 13, 2006 3:58 pm
Location: Minnesota USA

Post by rplate »

mkelley wrote:Well..., but the idea is just to create mouth shapes by moving the mask points, and having to then duplicate the layer, create new shapes, etc. etc. -- might as well not do it in the first place (and you certainly can't animate this the way you can simply by moving the points on the mask).
.
Agent Rose Tutorials here...

Have you watched the "Agent Rose tutorials yet? I just watched them again for the third or fourth time
and each time I am more and more amazed at how easily, what you're asking about can be done by moving points.
What you're talking about starts somewhere around Video Clip Part 6
I almost missed them. You have to scroll down a bit to see them.

Bob P
.
.
User avatar
mkelley
Posts: 1647
Joined: Fri Nov 02, 2007 5:29 pm
Location: Sunny Florida
Contact:

Post by mkelley »

Hmmmm, I'll have to check that out -- thanks for the heads up.

In the meantime I did find a reasonable workaround -- you can turn on shadows for the group layer and just turn off blur and offset (and, of course, set the shadow color to a solid black). This actually works fine, but it doesn't allow for the kind of line width fine tuning you can do with an actual shape (and, of course, only shows when you render).

But I've been meaning to check out all the Rose Tutorials (I have them all converted to WMV and can watch them in Premiere on a frame by frame basis -- just too lazy to spend much time with the process so far).
User avatar
mkelley
Posts: 1647
Joined: Fri Nov 02, 2007 5:29 pm
Location: Sunny Florida
Contact:

Post by mkelley »

Um, spoke too soon -- my workaround results in too thick a line no matter how small I try and make the shadows. Sigh.

Oh well, I guess I'll take a look at the tutorials you say -- but I don't think it's going to solve my problems because I think they approach the mouth a completely different way than I have in mind.
User avatar
heyvern
Posts: 7042
Joined: Fri Sep 02, 2005 4:49 am

Post by heyvern »

I agree, being able to "choose" if a stroke is part of the mask should be an option. I have yet to explore the scripting options for masking. I keep meaning to take a look at it and see if there is a way to get around it.

One of the reasons I like using bones as much as possible is for this purpose. A bone can move "identical" vectors. No need to "duplicate" the motion or limit your options. I can have multiple layers to "recreate" strokes that would be eliminated by masks and they move exactly the same as the mask with the bones.

If you prefer point motion over bones there is a layer script that will duplicate the vector animation from one layer to another. So you could animate the points of a mask layer and have a "linked" stroke layer duplicate the movement of those points. You never have to duplicate the motion... it just... happens. This is almost like having the "stroke" option for a mask except it isn't on the same layer.

-vern
User avatar
funksmaname
Posts: 3174
Joined: Tue May 29, 2007 11:31 am
Location: New Zealand

Post by funksmaname »

whats the script called vern? i cant remember...

Also - the bone tip is a good one, coz you could always put small bones over pretty much every point, and then do point animation using the bones (allowing them to control multiple copies underneith)
User avatar
mkelley
Posts: 1647
Joined: Fri Nov 02, 2007 5:29 pm
Location: Sunny Florida
Contact:

Post by mkelley »

For my purpose it isn't so much animating the points as it is using the vectors in a switch layer.

Let's see if I can explain (and maybe some of you bright AS folk can tell me another/better way of doing what I want). I am using interpolated switch layers for my lip sync. It looks terrific and is easy to manage *except* that I want to easily be able to set up the mouths to be "generic".

In order to do this I create a "cutout" in a larger shape -- that shape then is assigned a skin style which will blend into the skin of whatever character I want. Behind this is the back of the mouth, teeth and tongue. I create the mouth shapes by moving the points of the cutout (and sometimes tongue).

So far so good -- but the larger skin shape isn't as universal as I'd like. For one thing, beards are a big issue. For another, I don't want the skin to be so big it can't fit on a certain head, but it needs to definitely be big enough not to get in the way of the mouth opening.

Using a mask avoids both of these issues -- I don't even need to have a skin layer, just a mouth mask, and it will fit on anything, not even a "human" (like a can, or a door, or whatever). But since I can't easily outline it I can't do it.

It still seems to me that *something* ought to work in this regard -- something like a shape that has a larger outline and thus only the interior is cut out and the outline remains... I dunno, I could do this easily in Photoshop or Adobe Illustrator. I tend to feel that masks in AS were kind of an afterthought, and not well implemented in any case, so perhaps I shouldn't be trying to use them in this manner.
User avatar
slowtiger
Posts: 6246
Joined: Thu Feb 16, 2006 6:53 pm
Location: Berlin, Germany
Contact:

Post by slowtiger »

An example of the graphic style of your characters would help a lot.

The first character with a beard I can think of is Wickie's father Halvar (see here: Image).

His mouth is a simple shape which just is placed atop his beard.
User avatar
heyvern
Posts: 7042
Joined: Fri Sep 02, 2005 4:49 am

Post by heyvern »

I'm playing around with this for a different solution but in the meantime...

I'm curious, if you have everything in the mouth layer (teeth, tongue) why do you need to mask the beard or head at all? You can't put the mouth on top? Is it to hide the teeth and tongue?

If so here's what I've done before with 2 switch "duplicate" switch layers:

One switch is the "mouth" the other is a duplicate of the mouth and is the "mouth mask". When you duplicate the mouth switch for the mouth mask, delete everything but the mouth from each switch layer

Everything is in a bone or group set to "Hide all" in the masking. The mouth switch is on top and contains the mouth, tongue teeth etc. The mouth mask switch is above the head and is set to "Add to mask" (either "keep invisible" or not. It doesn't matter.) and the mouth mask switch ONLY has the mouth shape. Every layer but the mouth and mouth mask should be set to "Do not mask" in the mask settings.

(Here is where you run into trouble if the tongue comes OUTSIDE of the mouth. It will get masked. But you would have had that trouble anyway).

Remember the mouth mask is cutting a hole for the mouth to show through. Only the shape of the mouth is in the mask but a stroke is included with masking. This allows the stroke of mouth shape to show through.

If you need to you can vary the line weight of the outside of the mouth by changing the stroke width on both the mouth and mouth mask. You don't need to mask the head or face, just the mouth since they are both above the head. If you want the mouth to be behind the beard you would need to make sure there is a "hole" in the beard when you create the shapes. No mask would be needed in that case.

If you have other "lines" for the mouth like on the side or whatever just leave those in the mask and they will show through on the mouth switch. If the mouth shape "outline" includes lips or whatever make sure those are still in the mask switch layers. Basically the mask switch should contain the outermost outline shapes you want to see.

For animating both switches the same, since both switches are identical you can do one of two things depending on how you are lip syncing.

If you use papagayo you can just load the dat into both switches and you are done. If you aren't using papagayo you can duplicate the mask switch after the lip sync is done so you have those keys in place already...

OR you can use the Switch/slave script that links two switches. Set one switch with keys and the other one is a slave and you never have to bother with it. It always does what the main switch layer does.

Here is the link for the original script by 7feet:
viewtopic.php?t=2506

I am almost positive I made a new version that can use the same script for multiple slaves but I'm not sure if I posted it on the forum. I was unable to locate it. I think the script above requires multiple versions of the script with different names if you have more than one set of switches. It's been ages since I've used it.

this all sounds kind of complicated when it isn't... I should do a quick sample file.

-vern
User avatar
mkelley
Posts: 1647
Joined: Fri Nov 02, 2007 5:29 pm
Location: Sunny Florida
Contact:

Post by mkelley »

Vern,

That sounds like it would work but the dual mouth layers is a PITA.

The real PITA is that making changes is a bit complicated. Right now I can go in and quickly adjust each mouth position in a few seconds -- let's say I want a more smiling type speech. With this duplicate layer approach I'd probably be better off deleting the whole second mask switch layer and starting over again copying each mouth shape into it. That kind of defeats the whole point of the generic process.

And actually the reason I need to mask, rather than simply laying everything on top, isn't so much to hide the teeth and tongue (yes, those too) but the entire back of the mouth. The teeth and the tongue lay on the back, but the face mask has to extend around the back or the face will show through.

Here's a sample file to show what I mean -- basic thing to note is that all you need do to change the expression is to go into each mouth shape and modify the mouth (in actual practice you rarely need to modify the rounded mouth positions, like "O", which means you only need to change three or four -- very quick). And how the shape works well on generic faces but not on beards unless the whole face is covered with a beard (not like my own :>)

The sample file has the whole face covered, but in reality I'd like to have more "skin" showing around the mouth... but that means I have to go in and modify the generic mouth to accomodate this. A masking approach would mean I could use it in any situation (I think :>)
User avatar
mkelley
Posts: 1647
Joined: Fri Nov 02, 2007 5:29 pm
Location: Sunny Florida
Contact:

Post by mkelley »

Oh, and then I forget to post the file.

Here it is:

http://www.kelleytown.com/shared%20Files/mouthtest.zip
User avatar
heyvern
Posts: 7042
Joined: Fri Sep 02, 2005 4:49 am

Post by heyvern »

I usually make the "back of the mouth" a shape that is "connected" to the lips. The back of the mouth is not a separate vector shape. If the lips open the mouth shape follows along. The teeth or tongue shapes can be lowered or raised to be between the lips and the mouth back.

Yes, making changes to one switch means changing the other one as well. this can be a pain. It is difficult if you are constantly making changes to switches but it only takes a few minutes to delete all the shapes after duplicating the mask switch.

It all comes down to how much do you need the final effect? Is it worth the extra trouble? If there is no other solution you don't have much choice except wait for an upgrade.

I stopped using switches for mouths. I do everything with bones and actions. I can even import papagayo dat files that use actions. This allows for all kinds of complex masking and multiple duplicate layers without worrying too much. They all are controlled by the bones so any changes are much easier.

-vern
mgummelt
Posts: 2
Joined: Thu Feb 21, 2008 8:19 am

hear, hear

Post by mgummelt »

mkelley wrote:Oh, and then I forget to post the file.

Here it is:

http://www.kelleytown.com/shared%20Files/mouthtest.zip
I'm totally with you on this. I'm trying to do the same exact thing - have teeth inside my mouth that are masked so they show up in certain mouth shapes and in lip sync. But, of course, the teeth draw through the outline of the mouth shapes.

While there is the duplicate layer work-around and that works fine, it's absurd that you'd have to even do all that extra work when it seems like adding a simple option of "+Add fill only to mask" would solve all the problems in one fell swoop.

Anyway, until they add this really obvious feature, I'm resigned to either not having any detail inside my character's mouths or not having outlines on the mouths (which means no mouth drawn at all when it's closed). And only black irises/pupils for my characters' eyes (since my outline color is black).

All of which really kind of sucks.
Post Reply