audio sync drift over time

Blue · Post by **Blue** » Sun Aug 16, 2009 8:51 pm

Akgarner, keep in mind that with animation lip sync is not a "mechanical" process. Sometimes you need to have mouth shapes start before or after certain word sounds to make it "look right"--despite even what you might see in the waveform image. So there are definitely times that you would need to compensate by changing the animation (timing) to match the audio.

However, you probably are already are aware of the above, which would lead me to the next question:

Is there a lot of objects moving around or other intense processing happening at :40 and :60?

Next week I'll start doing some lip sync in AS so I'm very interested in this topic.

Vagabond · Post by **Vagabond** » Tue Aug 25, 2009 4:12 pm

Blue wrote:" if you are trying to playback HD files EXPECT audio/video losing sync and other playback anomalies.

Try rending uncompressed small proxy files and see what happens.

Really? I did a minute long scene that was 640x480 and the lip sync was fine when I scrubbed and when I rendered it in QT. Then when I uploaded to YouTube it was horrible. I deleted it then reloaded it and it was a little better. Is there a way around this?
http://www.youtube.com/watch?v=V2CJUjXxwfo

Akgamer: I haven't had an animation be in sync then out of sync but I have had actions mess up my lip sync. You sound more advanced than me so that's probably not it. Just a thought.

basshole · Post by **basshole** » Wed Aug 26, 2009 6:50 am

On a possibly related but maybe not note:

Now I'm noticing that a lot of times, the audio files play a few frames before they actually "occur" on the timeline. What I mean is, watching the sequencer, I can see where an audio file starts, and , when I hit play, I can watch the playhead speed through the timeline, and when it hits about 4-6 frames before the the first frame of the audio file, it starts playing it (i.e. if I have an audio file that starts on frame 30 I will hear it start when the play head hits around 24). It's pretty consistent about this. . makes it really hard to time dialogue, etc.

Is this just from having too much in the project/not enough horsepower to handle it all?

akgarner · Post by **akgarner** » Wed Aug 26, 2009 5:47 pm

All:

Thanks for your input. As it turns out I was doing everything perfectly but part of AS is faulty.

The source audio happened to be at 96KHz/24Bit. The downsampling component of AS6 didn't playback in sync with the keyframing NOR did the render component render in sync with the keyframes.

I converted the soundfile to a 44.1K/16bit sample rate and it works fine.

AS shouldn't allow a hi-fi sound file import if it doesn't work. Definitely needs fixin.

Best.

Adam

basshole · Post by **basshole** » Wed Aug 26, 2009 6:27 pm

i wonder if thats my issue? I WAS originally working with 96/24 and then exported 48k versions for use in asp but maybe it wants 44k?

akgarner · Post by **akgarner** » Wed Aug 26, 2009 10:00 pm

basshole wrote:i wonder if thats my issue? I WAS originally working with 96/24 and then exported 48k versions for use in asp but maybe it wants 44k?

Likely is the issue. Or the bitdepth. 44.1/16 worked fine for me. Didn't try anything in between!

basshole · Post by **basshole** » Thu Aug 27, 2009 12:03 am

I figure 48 and 44 are almost the same. 48 is the standard for video. . .44's more for music.

Darramouss · Post by **Darramouss** » Thu Aug 27, 2009 4:45 am

Being a stickler for quality I always prefer to use the highest quality sound I can. I found out long ago, however, that ASP has trouble with most things over 44.1/16. Problems I've encountered are the audio not synching with the animation, the audio tempo altering, the audio not even playing, and in my mind, the most annoying, the audio periodically dropping by a semi-tone and then correcting itself 10 or so frames later.

The best workaround I've found is to create my audio tracks, create a copy and 'dumb' the audio down to 44.1/16. I use the dumbed down in ASP so I can match the audio correctly and then render without the audio. I add audio later as an external process. Yes, I understand that it's more steps in my workflow, but if it allows me to have top quality sound then I'm happy to do it.

basshole · Post by **basshole** » Thu Aug 27, 2009 5:02 am

Yes, I exported the originally recorded 96/24 files from final cut pro as AIFFS at 48khz, thinking that would be easier for ASP to handle. Maybe I will try re exporting them at 44. That really is odd, though, as that is a music (CD -quality to be precise) standard and not a video standard.

I have also encountered the pitch shifts. . .except with mine it'll all of a sudden escalate to chipmunk pitch.

basshole · Post by **basshole** » Thu Aug 27, 2009 7:45 am

Alright, well, I tried making a couple of the audio files 44.1 instead of 48. It may be a LITTLE better, but I can still hear the audio files start playing at least 3 frames before the playhead reaches them. Haven't tried to export a movie with the 44.1 files yet so no idea if that works.

Darramouss · Post by **Darramouss** » Thu Aug 27, 2009 10:01 pm

I have found with 44.1/16 the sound can go off slightly when you're playing the video through ASP but renders OK. I think that's because there's so much going on that it can't play everything correctly, kind of like when you've got a squillion vector points and the animation playback get's jerky. You really need to export to see if there actually is a problem.

basshole · Post by **basshole** » Fri Aug 28, 2009 2:54 am

That's what I figured. But it makes it hard to see/hear the dialogue timing accurately in the program.

SpaceBoy64 · Post by **SpaceBoy64** » Wed Sep 30, 2009 7:08 am

I had a similar problem with audio sync. For files where I rendered out the audio with the animation, all was fine, but for some scenes I didn't get any audio for some reason. When I tried to match it up with the audio in Final Cut, the length of the audio was slightly off. It didn't match up. I thought it might be the 44.1 vs. 48k thing that causing a problem, but I checked that and that wasn't the problem.

I use animation codec usually, because it's lossless compression. Final Cut has to convert it and render it, but it's always worked well for me.

I thought maybe it's the frame rate issue. Audio editors don't usually specify a frame rate do they? Does Anime Studio use fractional frame rates, like 29.97? or does it just round up to 30 fps?

basshole · Post by **basshole** » Wed Sep 30, 2009 7:51 am

I tried the last scene using all files that were 44.1 . It was real close, but still a few frames off.

that may be an issue with how ASP divides 24 frames into a second and how FCP does it. You think 24 fps is 24fps, and frame 2 in your ASP project would = frame 2 of the rendered movie in FCP, but maybe it isn't necessarily so? I found myself having to slide the audio clips around in FCP to make it match up with the mouth movements from my ASP movie by a couple fraames.

SpaceBoy64 · Post by **SpaceBoy64** » Wed Sep 30, 2009 10:25 am

basshole wrote: that may be an issue with how ASP divides 24 frames into a second and how FCP does it.

Make sure FCP isn't using 23.976 fps, which is the video standard for PAL.
I'm assuming you're editing in PAL format?

If you're in the U.S., or other country that uses NTSC, you should be using 29.976 (or just 29.97, I can never remember, but it will be listed in FCP)
I'm just not sure Anime Studio is capable of this. It may be only doing 24 and 30 fps.

audio sync drift over time

Lazy Lip Sync