AAC 5.1 audio not always downgraded properly to stereo by AppleTV

Question

I have some HDTV recordings with 5.1 audio.
Using a version of ffmpeg (build date 2009 - so fairly recent) and encoding video at 1280x720 3Mbps (~24fps) and audio at AAC 5.1 448Kbps (6 channels), the resulting mp4 files play fine with QuickTime and iTunes under Windows, however, the AppleTV gets the audio channels mixed up on most of the encodings and the centre channel seems to be shifted to the left channel, another channel goes to the right channel and the other 4 channels are lost. There are some HDTV videos which do play fine. It almost seems as if the AppleTV only sees two channels rather than 6 and so isn't downgrading to stereo properly.
If I use Nero 7 to encode the audio to AAC 5.1 and then mux this in using ffmpeg, the audio is always fine, but this requires two steps, the first encoding the audio and the second encoding the video and muxing in the audio. I would much prefer to just use ffmpeg if there is an option I can set to fix the problem.
Is this a problem others have noticed?
I am guessing that it is a problem with the audio decoder on the AppleTV, as QuickTime and iTunes under Windows doesn't have this problem. If the audio is downgraded to stereo using ffmpeg, the audio is fine on the AppleTV.
Because I use my mp4 files with more than just the AppleTV, I don't want to have multiple copies of videos, ones with AAC2.0 and ones with AAC5.1 audio.
Only advice on fixing the problem appreciated. I do not need to be told what the audio specs of the AppleTV are.

Answer 1

Thanks for the response.
I'm using the analogue out from the AppleTV into my amp, so relying on the AppleTV to downgrade the AAC5.1 to 2.0.
I'm confused because some videos convert ok whereas others don't with ffmpeg. Although Nero recognises and converts the 5.1 correctly for all videos for the AppleTV, it struggles to detect the correct fps (always seems to be double actual fps). This is why I'm using Nero to do audio and ffpmeg to do video and mux in audio.
I was wondering whether the 6 channels in AAC5.1 are labelled somehow and so can be in any order but that the codec in the AppleTV doesn't look at these labels and assumes a specific order. It could be that Nero detects the labels (if they exist) but always outputs channels in a specific order, whereas ffmpeg always outputs the channels in the same order as the source.