## TL;DR We'll probe a real HLS stream to find out what audio it actually delivers, build a two-rung audio rendition set with correct EXT-X-MEDIA and CODECS declarations, and work out whether xHE-AAC is reachable for your stack. Spoiler on that last one: decoding is universal, encoding needs a license.
Your encoding profile has been re-tuned four times on the video side. The audio side is one 128 kbps AAC-LC track that nobody has touched since setup. Let's find out what that's costing and what the options are. You'll need ffmpeg 7.x or 8.x and, optionally, bento4. 🔎 Find out what you're actually shipping
Start with the master playlist, not your encoding config. These disagree more often than you'd think.
Two things to notice. First, mp4a.40.2 on every line: that's AAC-LC, the same audio in all three variants. Second, there's no EXT-X-MEDIA block, so audio is muxed into each video rendition and there is no separate audio rendition to switch.
Here's the number that should bother you. On the 360p rendition, total bandwidth is 628 kbps and audio is 128 of it. Roughly a fifth of the bytes going to your worst-connected viewers are audio, encoded at exactly the same quality as the audio going to the viewer on fibre. That is the whole problem in one line.
💡 Tip: run this against a live playback URL rather than a mezzanine file. What your pipeline intends to produce and what your packager emits are separate facts. Split audio out so it can vary
Muxed audio means audio is welded to the video rendition. Demuxed audio, declared with EXT-X-MEDIA, means the player picks an audio rendition independently. That's the prerequisite for everything else.
Now the 360p variant pairs with 64 kbps audio, and its total bandwidth drops from 628 to 464 kbps. That's a real saving on the rung where bandwidth is scarcest, and it required no new codec.
⚠️ Common mistake: pointing every EXT-X-STREAM-INF at the same AUDIO group and then wondering why audio never changes. The group ID is the switch. One group per audio quality tier. Verify the declaration matches the bytes
The CODECS attribute is a promise to the player. If it's wrong, some clients refuse to play before they ever fetch a segment.
| Codec | CODECS string | Notes | |---|---|---| | AAC-LC | mp4a.40.2 | The default everywhere | | HE-AAC v1 | mp4a.40.5 | SBR | | HE-AAC v2 | mp4a.40.29 | SBR + PS | | xHE-AAC | mp4a.40.42 | fMP4 only, common encryption only |
Apple's HLS tools (mediastreamvalidator) will catch mismatches too, and as of the 2026 release they run on macOS, RHEL 9.5, Ubuntu 24.04.3 LTS and Debian 13.2 with full feature parity, so this can live in CI on Linux now. 🎧 So where does xHE-AAC come in
xHE-AAC is MPEG-D USAC (the Extended HE-AAC profile) plus MPEG-D loudness and dynamic range control. Two properties make it interesting for the low rung of your ladder: Range. Fraunhofer specifies stereo operation from 12 kbps to over 320 kbps in one codec. Apple's HLS authoring guidance puts the practical stereo band at 24 kbps up to a recommended 160. Mandatory loudness and DRC metadata in every bitstream. The playback device adapts level to its own output. That's a normalisation stage you stop owning.
Playback support is not the blocker. Native decode has shipped since Android 9 Pie (2018) and iOS 13 / macOS 10.15 / tvOS 13, Safari decodes it through AVFoundation, and Fraunhofer puts the Android installed base above seven billion devices. Android 17 makes xHE-AAC encoding a standard feature this year, which is new.
Constraints to know before you plan around it: fMP4 only. No MPEG-2 TS carriage. Common encryption only. If you're on full-segment AES-128, that's a packaging change. AVPlayer does mono and stereo only. No multichannel, so a 5.1 ladder stays as-is. The encoding catch
Here's the part that decides this for most teams. Stock FFmpeg cannot encode xHE-AAC.
No USAC, no Extended HE-AAC. libfdkaac is the open-source Fraunhofer decoder/encoder kit and it stops at HE-AAC v2. Encoding xHE-AAC requires a licensed implementation: Fraunhofer's own, or the MainConcept xHE-AAC encoder plugin for FFmpeg. Packaging is fine either way, since Bento4 handles xHE-AAC in DASH and HLS without special options.
| | xHE-AAC | Cost | |---|---|---| | Decode | Universal, already deployed | Free | | Encode | Licensed implementations only | Purchase order | | Package | Bento4, no special flags | Free |