← Documentation ・ ← SmartCut ・ 日本語
Audio has no GOP structure, so it cannot be cut the way video is. This page covers how SmartCut cuts it instead: the three modes, what each one costs, how MPEG-2 AAC framing is preserved, how a 5.1 track is folded to stereo, and how a bilingual broadcast's two tracks are handled.
| Mode | Boundary error | Left over at a seam | Frames re-encoded |
|---|---|---|---|
copy |
±half a frame (10.7 ms for AAC at 48 kHz), no accumulation | up to 21.3 ms of the audio that was cut away | none |
smart (default) |
as above | nothing (it is silence) | at most 2 per boundary, none when the seam is in silence |
reencode |
−0.02 ms — one sample is 0.021 ms, so effectively zero | nothing | all of them |
The default is smart. On an ordinary cut — a commercial break taken out in the
silence around it — its output is byte-identical to copy, so there is nothing to
lose by making it the default. Where they differ, the cut is in the middle of sound
and the frame is worth touching. copy remains for work that must be exact to the
byte. The ±10.7 ms boundary error is the same in both, and far below the perceptual
threshold for lip sync.
Run tests/run_audio_tests.sh with SMARTCUT_AUDIO=reencode and all 5 cases land at
−0.02 ms. Video stays lossless in every mode (1798/1798, 99.1% on real material).
Audio is handled per kept interval rather than per video segment. Each interval's audio is anchored to the output time where that interval's video starts, so drift across intervals cannot happen by construction — or so it seemed.
Where the video starts is its first picture, not the bound asked for. A
range given in seconds on the command line need not start on a picture:
--keep 3.05-9 of a 5 fps recording re-encodes from 3.05, and its first
picture is the one at 3.2. Until 2026-11-10 the sound was laid from 3.05, and
the whole range, copied stretch and all, ran 150 ms behind its pictures — up to
a frame on any range that opened between two pictures. A range's sound now
starts on its first picture and ends on the first picture at or after its end,
where the last picture stops being shown (cut::sound_span). Only a
re-encoded end can sit between pictures, since a copy starts and stops on entry
points; a bound on a picture, which is what the editor sends, comes back as it
went in, and a reel written afresh on the master's frames keeps the range's own
times. The captions, the subtitles a disc draws and the data broadcast are laid
from the same picture; up to 0.8.44 they were laid from the bound asked for,
and went in up to a frame later than their pictures.
There was a trap. An MP4 audio track expresses time through sample durations
(stts); it does not keep a timestamp per packet. All it keeps is the track's start
offset, and the samples are laid out back to back after that. So dropping a single
audio frame at an interval boundary shifts every following interval permanently, and
the error accumulates interval by interval. Measured on material with a click every
0.5 s, the second interval came out a uniform 11 ms early.
The fix is to track the actual end position of the audio written so far and carry that error into the choice of the next interval's first frame. Since the interval opens on the frame nearest the boundary, the error is bounded by half a frame, about 10.7 ms for AAC at 48 kHz, and does not accumulate. A test with three intervals and every boundary deliberately off the frame grid showed a worst case of +7.67 ms.
Two routes to trimming in the container were tried, and both are dead ends:
AV_PKT_DATA_SKIP_SAMPLES, the side data used for gapless playback — the MP4 muxer ignores it. The samples marked for skipping stayed in and the audio ran late by exactly the amount requested.initial_paddingon the output stream — the muxer does not write it to the file. Probe the output andinitial_padding=0. Trying it made the single-interval error worse, from 0.00 ms to +9.33 ms.
That leaves re-encoding, which is what --audio-mode reencode does: each interval's
audio is cut sample-exactly and fed continuously into a single encoder.
Continuously, but not blindly end to end. Until 0.8.3 a packet's place was the
count of samples fed before it, so a stretch the track did not have — a
dropout in a broadcast, sound that starts after the pictures or stops before
them, a joined recording without the track — moved everything after it earlier
by as much. Now a hole of more than half a frame inside a range is filled with
silence (until 2026-10-08, more than a whole frame: the one frame a broadcast
loses where it is damaged was read past, and moved the rest of the range 21 ms
ahead of its pictures), and at the start of each range silence is added until the
sound reaches the output time the range's pictures start at. Where the sound
laid down already runs past that time, the range's own sound opens that much
later (Reencoder::align; more than a fiftieth of a second either way). The
sound already written is not taken back; what is given up is the same length
off the head of the range, which is where the copied sound of a smart cut
gives it up too (it opens on a later frame — pick_from in cut.rs). Until
2026-10-09 only the short side was handled. A range ending at a recorder
BD-RE's seam writes a picture fewer than its two times say, and every such
seam put the rest of the re-encoded sound that much behind its pictures: a
title cut across three seams was 26 to 73 ms late after each, where the same
cut's copied sound was on time. Ranges whose pictures fill them come out byte
for byte as before. The frame before each
range's first is decoded as well and thrown away: AAC, AC-3 and MP2 decode a frame
against the one before it, and without it the first frame of every range was mixed
with the end of the range before — a click at each join.
Two more things the whole-track encoder does, both missing until 0.8.4. An encoder whose delay is
not a whole frame is fed a lead of silence first — 1280 samples for AC-3, 671 for
MP2, the same lead_in the spliced frames
use — so that a packet begins on the track's first sample. Without it the first packet
held the priming and the first 1280 samples of the sound together, and was dropped
whole for the priming in it: a re-encoded AC-3 track began 27 ms late, MP2 14 ms.
And a range's fade-out is put on when the range closes, aimed at the last sample
the recording had rather than at the range's end. A range cut to the pictures at the end
of a recording runs on past the sound, and a ramp aimed at its end stopped at a quarter
of its level when the sound ran out; measured on a joined fixture, the last 80 ms
before the sound ends went from −40 dB to −69 dB.
A trap hit while implementing smart mode: audio frames straddling a segment boundary were submitted twice, once from each of the two adjacent segments, and the error accumulated one AAC frame (21.3 ms) at a time. Fixed by adopting the same exclusive ownership rule as the copy path.
tests/run_audio_tests.sh measures A/V sync sample by sample on material with a 2 ms
impulse every 0.5 s. Steady tones cannot reveal a sync error.
The boundary error is not the only thing copying leaves behind. The frame a boundary falls inside is copied whole, and what is inside it includes the audio from the side that was cut away. That is the last syllable of a commercial arriving over the opening of the programme: up to 21.3 ms of it, once per seam, every seam.
Smart mode re-encodes only the frames the boundaries fall inside. Each is built from the recording's own samples, with everything outside the kept range faded to silence over a millisecond. No other frame is touched. On real material — Nippon TV, two intervals, four boundaries — 5602 of 5606 frames are the recording's own bytes; 4 were rewritten.
The boundary itself does not move. Two frames cannot occupy one instant: MP4 lays its samples end to end, and MPEG-TS rejects a timestamp that goes backwards. So an interval is always a whole number of frames long. Opening an interval on the frame the boundary falls inside would lose nothing in itself, but that frame reaches back into the interval before it, where the previous interval's own overrunning last frame already is. Every mode therefore keeps whole frames and centres the error. What smart mode changes is what fills the rest of the straddling frame.
One neighbour is re-encoded with it, as a guard. AAC frames overlap by half a window, so the decoder rebuilds each frame from that frame and its neighbour, and a re-encoded frame sitting directly against a copied one is the one place the two halves can disagree. Silencing part of a frame is a transient, and a transient is what makes an encoder switch to short windows — which is exactly that disagreement. The guard carries the recording's own samples, unmasked, and keeps the switch one frame away from the copied material. When the straddling frame is not the one the interval opens on (the nearest-frame rule may pick the next), the guard has nothing to guard and is not re-encoded either.
When the far side is already silent, nothing is re-encoded. A commercial break is
cut in the silence around it, so the far half of the straddling frame is usually silent
already, and replacing it would remove nothing. The test is whether the samples outside
the range peak above −60 dBFS. On real material, a cut that takes out one commercial
block — all four boundaries inside the silence — re-encodes no frames at all, and the
output is byte-identical to copy. The re-encode only engages where a cut lands in
the middle of sound.
This design came from measuring TMPGEnc MPEG Smart Renderer 6.1's output against its source. It re-encodes not one audio frame: of the 67499 frames in a 24-minute cut, 67492 (99.99%) carry the recording's own coded audio, and the seven that do not sit nowhere near a join. All three of its joins are inside digital silence at −91.0 dBFS, where cutting on a frame boundary loses nothing. (It does re-pad every frame to a strict 256 kbps CBR, so only 24.7% of its frames are byte-identical — the audio data inside them is untouched.)
Whether a frame can be replaced at all comes down to what the encoder for that codec will take and what it hands back.
It has to take what these buffers hold. Everything here is planar float, which is what the AAC and AC-3 encoders want. Blu-ray LPCM wants 16- or 32-bit integers and MP2 wants 16, and an encoder handed a format it does not list will not open at all. So the encoder is asked what it takes, and the recording's own sample width is tried first. That is not only about keeping the samples: the LPCM encoder frames 16-bit sound 240 samples at a time and 24-bit sound 360, so a 16-bit recording asked for as 24-bit comes back framed differently from itself.
Its frame length has to be the recording's. A frame written here stands in for one of the recording's, so it has to be the same number of samples. Every encoder here has a frame length of its own — AAC's is 1024 samples, AC-3's 1536, MP2's 1152 — and LPCM has none at all, which is how it fits any recording's.
And its packets have to land where the recording's frames do, which is a question about the delay. An encoder announces one and then stamps its output by it: fed frames at 0, 1024, 2048 …, libavcodec's AAC encoder answers with packets at −1024, 0, 1024 …, so the packet covering the first frame fed comes out at 0 and the two grids agree. AAC's delay is a whole frame. Every other encoder here delays by part of one — AC-3 by 256 samples of 1536, MP2 by 481 of 1152, LAME by 1105 of 1152 — and their packets straddle the recording's frames rather than standing in for them:
| encoder | frame | delay | packets come out at |
|---|---|---|---|
aac |
1024 | 1024 | −1024, 0, 1024, 2048 … |
ac3, eac3 |
1536 | 256 | −256, 1280, 2816, 4352 … |
mp2 |
1152 | 481 | −481, 671, 1823, 2975 … |
libmp3lame |
1152 | 1105 | −1105, 47, 1199, 2351 … |
That used to end smart rendering for all three: the packets fell on no multiple of the frame length, they were discarded, and the boundary frames were copied.
The answer is to move the grid rather than the packets. The encoder is fed the tail
of the frame before the run — as many samples as it takes for the delay and the
lead-in together to come to a whole frame, so 1280 samples ahead of an AC-3 run and 671
ahead of an MP2 one — and from there its packets land on the recording's own frames,
one for one. That is audio::lead_in, and it is the whole of what makes AC-3, E-AC-3
and MP2 smart-rendered rather than copied. tests/run_audio_smart_tests.sh asks each
codec the same three questions: were the boundary frames re-encoded at all, is what
lies past the cut inside the last of them silent, and — the one a wrong lead-in would
fail — is the sound still where it was.
A constant-rate codec is written at its own rate. AC-3 and MP2 spend the same number of bytes on every frame, and those bytes are the rate: read off the frames being replaced rather than taken from the container, it is exact where the container's figure is an average, and the frame written here comes out the length of the frame it replaces. So a constant-rate track stays constant across a patch, which is the one thing a receiver reading it as one is entitled to.
Two codecs are not asked at all: DTS and TrueHD are lossless while libavformat's encoders for them are not, so their frames are carried through untouched — see Reading a disc.
Nor is FLAC or ALAC of any width but 16 or 24 bits. Both encoders here are
lossless, and both write those two widths only: 16-bit frames where the decoder hands
over 16-bit samples, 24-bit frames where it hands over 32-bit ones. A frame of one width among frames of another is not one
libavcodec will decode ("switching bps mid-stream"): a 32-bit FLAC track, and a 20-bit
one -- which is what 20-bit LPCM converted to FLAC comes to -- lost two frames, about a fifth
of a second, in every range, and --verify called the sound short (0.76 s over
three ranges). An ALAC frame does not state its width at all, so there the patch would
have decoded as noise. Such a track is carried through whole, as DTS and TrueHD are --
width_not_written in audio.rs -- and the
cut says so; its boundaries land on whole frames, and a fade asked for is not on it.
16 and 24-bit FLAC and ALAC are patched as before. The width is read from the stream,
which states it the same way in .mkv, .mka, .mp4, .m4a and .flac.
One case still loses the low bits: --sound-only into a .flac re-encodes the whole
track (the file opens on a header that states its length and checksum), and a 32-bit
track is written at 24 bits there. An .ogg or .oga of FLAC opens on the same
header, and is re-encoded the same way; up to 0.8.44 it was copied, and a player
read ten seconds kept out of a forty-second recording as forty.
So smart rendering reaches AAC, AC-3, E-AC-3, MP2 and Blu-ray LPCM — everything lossy a broadcast or a disc carries that there is an encoder for. DTS and TrueHD are copied on purpose, and anything with no encoder at all is copied with a note saying so.
A copied track is bytes from the recording, and for most codecs a frame is readable on its own. TrueHD's is not. It carries its format in a major sync that recurs through the stream — about every 106 ms on the Blu-ray measured here — and everything between two of them is read against the last one seen.
That matters at every seam. At the start of a file a decoder simply waits for the next sync, which is what a decoder joining a broadcast does anyway. At a seam it is already reading, and the frame arriving is from another part of the recording entirely: the restart header stops matching, the matrix count runs past what the format allows, and what comes out is noise or nothing. On a two-track Blu-ray cut in two places, each seam was worth 79 and 58 complaints from the decoder, and 76 ms of sound that never arrived.
So a track that has to be joined at a sync waits for one at every kept
range, not only at the file's start — AudioTrack::joins_at_sync in
cut.rs. The frames dropped while it
waits are up to one sync interval, and the time they occupied is counted as
spent, so what follows still lands where the recording had it and the muxer is
never handed a timestamp behind the one before it. Silence at a seam is worth
having in exchange for a decoder that stays in step; LPCM, which has no such
state, is unaffected and measures to under a millisecond across the same cut.
The same track in a .mkv was written as no sound at all, and said nothing
about it. A TrueHD access unit is 1/1200 of a second; Matroska keeps time in
milliseconds and stores the units with no duration at all, leaving that to be
worked out from the timestamps. A frame with no length is a frame the writer
has nowhere to put -- the output lays its sound end to end -- so every one of
them was left out, under a cut that declared the track and said it was
carrying it through byte for byte.
So the length is measured before the cut starts, in assumed_frame, and
over the whole run rather than gap by gap: the gaps themselves come out as
a ragged mixture of one millisecond and none, and every way of picking one of
them answers 1 ms, which is a fifth long. The first timestamp and the last,
over the frames between, answer 0.8333 ms.
The timestamps are still the container's, and still too coarse to place the
frames by: a sixth of them carry the instant of the frame before. So a track
measured this way is laid from where it has reached rather than from the
instant on the packet, and two frames at the same instant are only refused
where the output will not take them: a transport stream, and an MP4 (below).
Laid end to end, a measured track never hands either two. Measured on a
three-second .mkv cut to two: 70,072 bytes of TrueHD out of 105,064, into a
.ts, a .mp4 and a .mkv alike -- and nought before this.
An MP4 refuses two frames at one instant as a transport stream does. Every
muxer libavformat does not mark as lenient about timestamps -- .mp4, .m4v
and .mov among them -- stops at a frame stamped with the instant of the frame
before, with a bare "Invalid argument". MPEG program streams (.mpg, .vob)
hand such frames over in two ways. An AC-3 frame whose start straddles two
packs can come with the next pack's stamp, so that two frames share it: ten in
90 seconds of a file ffmpeg -f vob wrote, none on a pressed DVD. And sought
into the middle of an MP2 frame, the MPEG audio parser can lock onto a false
sync inside the frames and hand over pieces of them with made-up stamps until a
stamped pack steps back onto one of them; a steady tone, a line-up tone, keeps
it locked. Up to 0.8.30 either stopped a cut of such a recording to .mp4
outright.
The AC-3 frame is now put back where it belongs as it is read. The first of
the two is really the frame before: it comes a frame past where its track has
reached, with a frame's hole behind it. So on a program stream a sound frame
that comes a frame past where its track has reached is held back until the
track's next frame is read. Where that one carries the same stamp, the held one
is stamped where the track had reached, end to end, and both are kept;
otherwise it goes on as it came (LateFrames in
input.rs). Nothing else is held, and on
any other container nothing at all. The cut's segments, copied and re-encoded
alike, a program-stream recording conformed as a reel of a join, and
--sound-only all read through it. Before, a .ts or .mp4 kept the late
frame in the slot of the next and left the real one out, a frame of sound lost
and a frame of silence in its place each time, and a Matroska file got both at
one instant. On the file with ten of them, a cut of 1 to 89 seconds wrote 2,741 frames
with nine holes and the note; it now writes 2,750 with none and no note.
So does the decode that makes the smart-rendered frames at a range's edges
(decode_on in audio.rs). Up to 0.8.31 it
read the frames as they came, so what it made was keyed on the stamp the two
frames shared, while the cut looks it up by the one put back: a range ending on
the frame after a late one got the late frame's sound, re-encoded, in that
frame's place, and an edge inside a restamped frame found nothing made for it
and copied the frame whole, up to 32 ms of sound the cut had taken away.
What the read cannot put back, such as the MP2 parser's pieces, is still left
out where the muxer would refuse it, counted in the note that names the frames
that do not follow, as into a .ts (Writer::push_audio in
cut.rs); a Matroska file, which takes
both, still gets both. Where a frame is left out
that way, the sound written so far is taken to reach as far as the frame
written at that instant does: counted only by the frames it wrote, it fell a
frame short, and the next range opened a frame early, laying a frame of the
sound the cut took away over the last one.
The average is the container's arithmetic too. A TrueHD .mkv that ffmpeg or
SmartCut wrote starts on gaps of 13 or 75 ms where 0.83 is due, so the average
of its first frames is a third or more long; laid end to end at it, one 60 s
range of sound ran out over 63.5 s and the next range opened 1.8 s late. So each
frame is laid by its own length, read off the frame as the next section reads a
rounded one (sound::Lengths) and taken where it lies within a tick of the
average, which stands only where no length can be read.
A recording joined on is measured on its own. The length measured is the
master's, and a master whose frames carry their lengths has none measured. A
Blu-ray title's TrueHD has a length on every frame and a Matroska file's has
none, so an .mkv joined after a title was written with no sound for the whole
of it -- 150 s of it on the pair measured, with a note and --verify calling it
off. Where the master had nothing to measure, each recording joined on is
measured off its own timestamps (the head of each reel in cut_into), and laid
end to end as the master's would be. A join whose master had a length measured
is written as before.
Re-encoding the same track lost frames as well. A Matroska file's TrueHD is
stamped to the millisecond and comes a frame every 0.83 ms, so about one packet
in six shares its stamp with the one before -- 19,985 of 119,911 in the file
measured. The re-encoder passed over any packet stamped no later than the last,
which is how a packet handed over twice after a seek is told apart, and so one
frame in six was never decoded and the rest were laid end to end: a 30 s range
written with --audio-codec aac came to 26.35 s, and joined onto a master with
AAC, 100 s of the recording came to 88.3 s and ended 11.8 s adrift. A packet
with the same stamp is now passed over only where it was read from the same
place in the file or before it (audio::Reencoder::take), and as before where
either place is unknown.
--verify now looks at where the sound is laid as well. It used to count the
decoded samples and nothing else, so a track with every sample in it passed
however far apart they were stamped: the 0.8.26 cut of a 60 s range of a .mkv's
TrueHD, whose last packet sat 21 seconds past the last picture, was called OK. A
track that ends more than half a second after the last picture ends is now
reported as off (Sound::past in
verify.rs); on correct outputs the most
measured was 0.051 s.
--verify reads the head of a track as the writers read it. A range from
the head was read through a seek, and a seek lands on the first key picture,
where a broadcast's sound starts half a second to a second before it. That sound was
taken for sound the recording does not have and allowed for -- 0.896 s on a
broadcast remuxed into an .mkv, between 0.6 and 1.2 s on transport streams
and Blu-ray clips, 1.85 s on a remuxed recording joined to itself -- so a file
of sound alone that had lost that much at its head passed: a ten-second range
written as 9.15 s was called OK. Each track's head is now read with
sound::sound_from_head, as the sound-only writer reads it (see the head of a
Matroska file),
and only what is really missing is allowed for: 0.06 to 0.15 s on the same
files, and none on the remuxed one. The verdicts on correct outputs did not
change; that 9.15-second file now reads as off. Reading the head as the writer
does means sharing what the writer gets wrong, though: on a .mkv whose open
lost its whole first cluster to an unreadable WebVTT cue, both lost the same
0.26 s, and the short file was called OK. Both now find that cluster's sound
through the open that lost nothing (the sound in that lost
cluster).
Where that first cluster holds nothing but the cue and the sound, the
sought read missed it as the open did, and a file 1.15 s short was called OK;
that shape is now found too.
The reads of a track stop by where the file stands. Each one reads on
until the pictures are well past where it is going, and took that from a
picture's presentation time. A recording with B pictures stores the picture
shown after a still before the pictures shown ahead of it: a screen recording
holding one picture for twelve seconds stores the next one, shown twelve
seconds on, just after the still and among the sound of the still's first
second. Every read that crossed it stopped there, so a sound-only file of
--keep 1-5 of such a recording held 1.1 s of sound, and --verify said a
correct cut had seconds less sound than it has, or passed one that had lost
it. They now go by the picture's decode time, which is the file's own order
and runs ahead of the sound by no more than the pictures held back for
reordering (audio::read_reached): --verify's reads, the sound-only file,
the sound's shape and the decode around a seam, and the cut's own copy.
The read of what a range holds waits for every track. Where --verify
measures how much sound the recording itself has in a kept range
(held_in), the read stopped once the pictures were 1.5 s past the range's
end. A muxer may lay the sound well behind the pictures -- ffmpeg's MPEG-PS
writer puts it some 3 s back -- so the sound of the range's last seconds had not
been met yet, and was allowed for as sound the recording lacks: 3.2 s on an
.mpg cut into two ranges, enough to let a cut that lost that much pass. The
read now goes on until every track has come to the range's end as well, and
stops 10 s past it in any case, since a track that has stopped in the
recording has nothing more to come.
And the last picture is as long as the track says. --verify took the last
picture to be up for a frame of the average, which is where it puts the end of
the pictures to measure the sound against. A variable-rate cut ending on a still
writes that picture up to where the range ends (see the Rust
core), and with
sound beside it the sound was said to be laid seconds past the pictures. The
length now comes from the end the container states for the picture track where
that is more than a frame and a half past the last picture: an MP4's track
length, or the DURATION libavformat's Matroska muxer writes for each track. A
cut of a variable-rate recording takes the track at its word wherever it states
an end, however close that end is to the last picture: a range ending just after
a picture of a screen moving ends that picture some 40 ms on, and counted a
frame of the average past it (most of a second on a screen recording) a faithful
cut's sound was called OFF. Where nothing states an end, as in a transport
stream, the last picture is taken to be up for no longer than the shortest gap
measured between the recording's pictures. A cut of a constant-rate recording
reads as it always has.
Nothing may finish having written a declared track and no frames of it. Every other way of losing one ends in an error: a codec the container has no box for stops the header, an encoder that will not open stops the setup. That one said nothing, and the file played with the sound listed and silent, so it is now the end of the cut. A track that holds nothing because the kept ranges hold none of it says so and names the way out, which is to leave the track out.
Matroska and WebM keep time in milliseconds, and a frame that does have a
length is handed over with it rounded down: 21 ms for an AAC frame of 21.33 at
48 kHz, 10 for DTS's 10.67, 92 for a 4096-sample PCM packet at 44.1 kHz. The
cut sums those lengths to know how far the sound has reached, so every range
came up a fraction of a millisecond short per frame. The next range took the
shortfall for sound still owed and opened a frame early, on top of the one
before, and a player lays sound end to end: every cut of an ordinary AAC, PCM,
DTS or Vorbis .mkv put the rest of its sound later. Over eleven ranges it
grew from 41 ms to 137 on AAC and from 21 to 95 on DTS, and the run reported
frames lost that were not.
So where the container's clock is that coarse, each frame's length is read off
the frame itself (sound::Lengths, the reading the sound-only writer lays its
frames by): libavcodec's count of the samples in a frame where every frame is
one length, FLAC's block header, Opus's first byte, Vorbis's block sizes, and
for DTS the core header — its block count of 32 samples each at the core's own
rate, since libavcodec answers nought for DTS and a DTS-HD Master Audio track
stated at 96 kHz carries 48 kHz core frames. A reading is taken only within
one tick of the length the container stored. The same eleven ranges now stay
within a frame of the recording on all of these, and a joined reel is read by
its own stream's parameters. A transport stream's or an MP4's clock already
says the length exactly; those are left alone, and their cuts are byte for
byte what they were.
A cut joins two instants that were never next to each other, and what the sound does there is a step — a click at best, half a word at worst. A fade takes the level down into the join and brings it back out of it. What is heard is a pause rather than a jump.
The default is 0, and nothing fades. The reason is what it costs: what a fade fades is the programme. Ask for a second and a second either side of every seam is quieter than it was recorded. A cut of a recording is meant to be the recording, shorter, so this is something to be asked for rather than something to be decided here.
Only where two ranges meet. The first range's head and the last range's tail are the beginning and the end of the file, not a seam; fading there would be this program deciding how the recording opens and closes.
The length is clamped to half the range. Two seconds at both ends of a three second range would leave no instant at full level, and what came out would be the shape of the fade rather than the sound. Where the two ends overlap, the quieter wins — and that is not only taste. Each end is written as its own run of frames, so a sample inside both fades has to come out the same whichever run wrote it.
The shape is a raised cosine, the same curve the de-click at a frame boundary uses: unlike a straight line it leaves silence and arrives at full level with no corner at either end.
Only sound this program writes. Under smart rendering every frame the fade runs over is rewritten; under a whole re-encode every frame is rewritten anyway. A copied track is copied — not one byte is rewritten — so nothing fades, and the same goes for sound carried through whole because re-encoding it would lose what makes it lossless (TrueHD, DTS-HD MA, FLAC or ALAC of another width than 16 or 24 bits). Where it cannot be applied the cut says so rather than quietly doing nothing.
Smart rendering rewrites the frame a boundary falls inside and the guard beside it. A fade extends that run to whatever the fade reaches — some 47 frames of AAC for one second, 32 of AC-3. The window of frames decoded around the edge grows with it, and it has to cover both the fade and the frames the encoder needs beyond it: stopping at the end of the fade leaves the run with no frame to flush through, and the whole patch is dropped. That happened during development, and what came out was a cut that faded out and never faded back in.
Smart rendering puts a condition on the guard frame beside the straddler: use it only if the straddling frame was itself written, and otherwise write the recording's own bytes. A straddler that was not written leaves the guard with nothing to guard.
Under a fade that condition has to come off. For a frame the fade runs over, the recording's own bytes are not an equally good answer — they are the one frame without the fade on it. Left in place, the condition stands one untouched frame exactly at the seam: full level, in the one place the fade was asked to take the level away.
Found by measuring. AC-3 and MP2 left 32 ms of full level at the join; AAC did
not. Read in 100 ms windows it looks like nothing worse than a shallow dip, and
only at 10 ms does it resolve into a single frame. tests/audio_fade.py exists
to watch that frame.
One cut in real material (broadcast TS, AAC 48 kHz stereo) with
--audio-fade 1.0, read as RMS per 100 ms: the level starts down a second
before the seam, reaches −55 dB just before it, and comes back to its old height
a second after. The middle of the ramp measures −6.25 dB against the unfaded
cut, which is the raised cosine's own midpoint (−6.02 dB) to within 0.2 dB.
With --audio-fade 0, the default, the output is byte for byte what the
binary from before this feature wrote.
A Japanese broadcast carries MPEG-2 AAC: the ADTS ID bit is 1, profile LC,
48 kHz, with a CRC. FFmpeg's AAC encoder produces MPEG-4 AAC, and every muxer that
frames raw AAC for us writes ID = 0. The ADTS muxer has a write_mpeg2 option, but
MPEG-TS has no way to pass it down to the ADTS muxer it uses internally. Left
alone, a cut comes out as a stream that is MPEG-2 nearly everywhere and MPEG-4 for one
frame per seam, which the tools downstream of a recording read as malformed rather than
as the recording they were handed.
So the ADTS headers are written here instead (adts.rs). The profile, sampling
frequency, channel configuration and ID are read off the recording's own frames and
put in front of the encoder's output. Every muxer involved leaves a packet that
already begins with a sync word alone, which is what makes this work: MPEG-TS passes
it through untouched, and MP4 runs it through aac_adtstoasc exactly as it does the
source's own frames.
- The default follows the recording (
--aac auto), so broadcast material stays MPEG-2. - Only the frames SmartCut writes are framed here; copied frames keep their own
headers. So a version that disagrees with the recording cannot be delivered while
anything is being copied — it would make a stream that is two kinds of AAC at once —
and the request is refused with a note instead. Under
--audio-mode reencodenothing is copied and it is honoured. The tests check both. - The payload is kept inside MPEG-2 AAC LC as well: perceptual noise substitution, an MPEG-4 tool, is switched off.
- The frames written here carry no CRC (
protection_absent = 1). It is a per-frame field, so such a frame sits legally among frames that have one, and writing a CRC that was subtly wrong would be worse: a decoder that checks it would throw the frame away. --audio-mode reencodeuses the same framing, so a whole track re-encoded into a transport stream also comes out MPEG-2 AAC — a combination FFmpeg on its own cannot be asked for.
tests/run_aac_tests.sh walks the output's ADTS frame by frame, checks they are all
MPEG-2 LC, and counts how many are the recording's own bytes. Neither question can be
put to a decoder; both are about the bytes.
There is more on why this matters downstream in broadcast workflow compatibility.
A 4K broadcast carries the same codec framed the other way the standard allows. Its
sound is MPEG-4 AAC LC at 48 kHz in LOAS/LATM frames: a sync word, a length, and an
AudioMuxElement that states the stream's configuration in bits rather than in a fixed
header. A transport stream declares it as stream type 0x11 where ADTS AAC is 0x0F,
and libav gives the wrapper a codec id of its own, aac_latm.
Two things follow, and both used to end smart rendering for such a track:
- No encoder answers to
aac_latm. It is a framing, not a codec. What writes a frame for one of these tracks is the plain AAC encoder —audio::encoder_for— and what puts the wrapper back on islatm.rs. - The wrapper has to be written here, for the same reason the ADTS header does: the encoder hands back a raw frame and every muxer leaves a packet alone once it holds one. A payload spliced in with no sync word in front of it is not a frame at all.
So latm.rs writes the one shape a recorder uses — one program, one layer, AAC LC at
1024 samples, the configuration restated in every frame — and reads the recording's own
frames first to be sure that is the shape they are in. A rate written out in full, or a
channel arrangement described inside the frame, is declined, and those tracks are copied
as they were before. aac::Framing is which of the two a track uses; everything above
this line works the same either way.
AAC asked for by name is declared as the framing it goes out in. Asked of one of
these tracks (--audio-codec aac, or AAC in the output settings) into a .ts, the track
is re-encoded and framed as LATM again, and up to 0.8.23 the map was rewritten to say
ADTS (0x0F) over it: a player looking for ADTS sync words found nothing in it. The map
says 0x11 now, in a .ts, a .m2ts and a disc's clip index alike.
Measured on a 4K recorder's disc: a two-range cut re-encodes four frames per track at the seams, the whole track decodes with no errors, and the largest sample-to-sample step across the seam is the same size as the ones either side of it.
libavformat describes a track from the first frames it meets, a few megabytes into the file. On a broadcast recording those frames are regularly not the programme's. A tuner is told to start early, so the file opens on the end of whatever was on before — and the sound of that can be a different shape. A bulletin read in mono ahead of a documentary in stereo is an ordinary evening's television.
Three seconds of it was enough to have a fifty minute stereo track called mono, and everything downstream believed it:
- the encoder opened for the frames at a seam was opened with one channel, so two frames of every boundary came back mono among the stereo ones around them;
- the ADTS header written in front of them said one channel as well, which is what a transport stream's decoder reads;
- a whole-track re-encode folded the programme's two channels into one from beginning to end;
- and a BDAV clip index written from the cut described the track as mono, because that is the number it is given.
The bitrate is read off the same frames, and is wrong for the same reason. A stereo
opening in front of a 5.1 programme has the container state 208 kbit/s for a track
carried at 386, and a whole-track re-encode then spends a stereo programme's rate on six
channels. So it is given up along with the rest of the opening's description rather than
guessed at — measuring the real one means reading a second of sound at each place looked
at, a hundred times what this check costs — and derived_bit_rate, which is what a codec
is worth at a given channel count, stands in: 384 kbit/s for 5.1 AAC, which is what the
recording was carried at. That stand-in also stopped being a flat 192 kbit/s, a figure
that is stereo's; it follows the channel count upward now, and never downward.
So the frames are asked again. audio::settled_shape seeks to four places spread
through the recording, decodes one frame at each, and takes the channel count and the
rate a majority of them agree on. Where that disagrees with the probe, the recording's
own answer replaces it and a note says so once. The whole check costs about 10 ms on a
5.8 GB recording — four seeks and four frames — and it is asked only of a recording made
off the air. A disc's clip is a transport stream too, and is one programme from its
first frame: nothing in it can disagree with its own opening, and inside an image every
one of those seeks is a walk of the volume as well.
The frames a range opens on can be the other programme's, and copying them through carries the fault into the output. A cut asked to begin a second before its programme did keeps those frames — keeping them is what it was asked for — so the output opened on mono ahead of a stereo track, and everything that describes a file from its first frames was back where it started: libavformat called the cut mono, and a BDAV clip index written from it said mono too.
Smart rendering already rewrites the frames a boundary falls inside, and the run it
rewrites at the head of a range is now extended over every frame that is not the track's
own shape. foreign_reach measures that run off the frames themselves — each decoded
frame is asked what shape it arrived in, before anything conforms it — and takes it from
the range's start to the first frame that is the track's own. The encoder that was going
to write the boundary writes those frames as well, in the track's shape, by the same
swresample pass that conforms every other frame it is handed. What it costs is that
those frames stop being an exact copy: sixty milliseconds of the previous programme on
the recording measured here, two seconds in the fixture. What it buys is a track that is
one shape from its first frame to its last.
Two limits, and past either the frames are carried as they are and a note says so. More than ten seconds of it is a range that is the other programme rather than one that opens a moment early. And a change of rate is not a change this can absorb: the decoder hands back samples on the rate they were written at, and frames on another grid cannot stand in for the recording's.
A copy rewrites nothing, so the clip index is measured too. --audio-mode copy
carries every frame byte for byte, the other programme's included, and there the index
beside the stream is the only place the track's own shape can be stated. bdav::read_clip
asks settled_shape of the stream it is indexing rather than taking the probe's answer
for it: the check above is deliberately not asked of a disc's clips, a recorder's clip
being one programme from its first frame, and a clip this program had just written was
taken in by that exception through the shape of its path. What reached CLIPINF was the
opening's answer all over again — and that is the number a player configures its decoder
from, and the number every tool that reads the disc reports. Measured on a 55 minute
recording that opens on three and a half seconds of mono: the disc's index said one
channel, and says three — stereo — now, whether the cut rewrote the opening or copied
it.
Three things follow from taking a majority rather than a first answer:
- A recording that really is mono stays mono. Every place looked at agrees with the probe, and nothing is corrected.
- A recording holding more than one programme is left alone. The places looked at do not agree among themselves, there is no such thing as its own shape, and the probe's answer is as good as any other.
- The frames the cut writes are framed as the cut writes them, not as the recording
opened.
frame_asrestates the channel count and the rate in the ADTS or LATM header whether or not anything asked them to change, andboundary_patchesis handed that header rather than the one read off the head of the file. Stereo samples behind a header announcing one channel was the fault in its plainest form, and it was twice per seam.
tests/run_audio_head_tests.sh builds that recording in miniature — four seconds of
mono concatenated to forty of stereo, on one PID — and checks the cut of it three ways:
the track is described as stereo, every ADTS header in the output says two channels
while all but the boundary frames are still the recording's own bytes, and a whole-track
re-encode comes out stereo. Beside it the same shape with 5.1 behind a stereo opening,
where the re-encode is checked for its rate as well as its channels; and a guard, a
recording that is mono throughout, which has to come out mono. Against the binary from
before the fix, seven of the twelve checks it had then fail.
Beside them a range that starts inside the mono opening, which is the case the frames themselves have to answer: the output is stereo from its first frame, every ADTS header in it says two channels, and what those frames carry is the tone the other programme was carrying rather than a hole where it was — both channels, 3 dB under the mono it came from, which is what the conversion does. Then the same range onto a disc, both ways — smart, where the clip opens on the programme's shape, and copied, where it opens on the other programme's and the index says the programme's anyway — and the all-mono guard onto a disc as well, where the index has to say mono.
The preview player had the same fact to reckon with, and did not. The sound the
window plays goes through one swresample context, built from the first frame it decodes
and configured for that frame's format, layout and rate. swresample will not take a
frame that is not the shape its context was built for: at the change of programme it
answered Input changed, the error ended the playback thread, and the sound stopped at
the instant the programme began while the picture played on. Measured on a fifty minute
recording that opens on three seconds of mono, 151 frames converted and every one after
them failed. playback_audio::resample now asks each arriving frame what shape it is
and builds a fresh context where it differs from the last, which costs one allocation at
the change and carries the whole recording.
And so did the writing side. The frames a seam is rebuilt from go through a context
of the same kind, built for the first frame of the run. A range wholly inside the
programme hands it one shape and nothing is ever noticed; a range that spans the change
of programme, written to a layout that is neither of the two — 5.1 asked for outright —
has both of them to convert, and swresample refused the second the same way. The cut
ended outright at the instant the programme began. audio::conform asks each frame what
shape it arrived in and builds a fresh context where that differs from the last, and the
suite gained the range that spans the change: it comes out as 5.1, and the sound runs the
whole way across it rather than stopping where the shape did.
Some recordings carry 5.1, and a good many of the places they end up do not want it: a television that folds it badly, a player that puts the dialogue in the centre channel and never plays it, a phone.
Asking for stereo is asking for the track to be rebuilt, and this is the one audio
setting that decides the mode rather than obeying it. No frame of a 5.1 recording
can be spliced into a stereo track, so smart and copy have nothing to offer here: a
channel count that is not the recording's makes the cut a whole-track re-encode, and
SmartCut says so before it starts.
The fold itself is swresample's, so the rematrixing coefficients are libav's own: centre at −3 dB into both sides, the surrounds into their own side, the LFE dropped. What comes out is what a player downmixing the recording would have produced, which is the point. The recording is being adapted to the player, not reinterpreted.
- In and out at the same rate, which is what keeps it a per-frame operation. swresample hands a frame's samples back one for one and holds nothing over, so the sample window each keep-range is trimmed against still means what it meant on the source's own clock, and the boundaries stay sample-accurate.
- The ADTS header has to be rewritten too. A transport stream says how many
channels a frame has in the frame's own header, and the header these frames inherit is
the recording's. Left alone it announces 5.1 over a stereo payload, and a decoder that
believes the header ahead of the payload gets neither.
AdtsFormat::with_channelsputs thechannel_configright. A count with no configuration of its own — 7 channels, say — is left alone, because such a stream describes itself in a program config element inside the frame instead. - The output stream's parameters come from the encoder once the channels have changed, framed output or not: the recording's parameters describe a track the file no longer contains. This costs the framing nothing, since a packet that already begins with a sync word is passed through whatever the extradata says.
- A derived bitrate comes down with the channel count. 384 kbit/s is what the 5.1
cost, not what the stereo it was folded into is worth, so the rate taken from the
recording is scaled by the fold and floored at 128 kbit/s.
--audio-bitrateis taken as given.
tests/run_downmix_tests.sh uses a 5.1 fixture with a different tone in every channel,
so the fold is read out of the spectrum rather than off the metadata: the centre has to
arrive in both output channels, the left surround in the left only, and the LFE
nowhere. It checks the ADTS channel_config as well, and finishes by measuring A/V
sync through the fold on a 5.1 impulse fixture — 0.83 ms at worst, against the 1 ms bar
a whole-track re-encode is held to.
Eight channels go through every path six do: copied, spliced at the seams, re-encoded whole, folded, and played in the editor. What differs is which codecs can take them and how many ways there are to arrange them.
- AAC and linear PCM write 7.1; AC-3, E-AC-3 and DTS stop at 5.1. That is the
encoders' own limit, not a choice made here, and it is asked of them the way every
other limit is (
opens_at). A 7.1 recording asked for as AC-3 at "Same as the input" is refused, and says it is refused over eight channels; the window greys AC-3 and DTS out until 5.1 or fewer is chosen. Smart rendering of a 7.1 E-AC-3 track cannot rewrite its seam frames for the same reason, and copies them whole with a note. FLAC and Opus are carried as they are. - The track's own arrangement is kept where the encoder takes it. Two layouts of
eight channels are not the same channels —
7.1(wide)has its extra pair at the front rather than the sides — and the encoder used to be opened with the default for the count whatever the recording said. A whole-track re-encode then had swresample remix one into the other, and the seam frames, whose samples are copied across by position, came out of the wrong speakers. The encoder is now opened with the recording's own layout first (encoder_layout'skeep), and with the default only where it refuses that one. A fold goes to the plain layout for the count it folds to. - Playback on Linux puts the channels in ALSA's order. libav, Windows and macOS all
order a layout by its channels' bits — front left, front right, centre, LFE, back
left, back right, then the sides. ALSA puts the rears before the centre: front left,
front right, rear left, rear right, centre, LFE, and the sides after those for 7.1. The
PulseAudio and PipeWire plugins every desktop routes "default" through read it that
way too. Handed libav's order, a 5.1 recording played its centre from the rear left and
its surrounds from the centre and the LFE, and a sound server folding that down to two
speakers put the dialogue on one side at a fraction of its level.
to_card_orderreorders 5, 6 and 8 channels on the way into the ring buffer; this was a fault for 5.1 long before 7.1 was offered. Four channels cannot be put right by reordering: libav's four is 4.0, with a centre and a back centre, and ALSA's is quad, two pairs and no centre. They are mixed to quad on the way instead (card_layout), so a 3/1 broadcast's dialogue comes from the front pair rather than the rear left. Fixed in 0.8.2. - The window offers 7.1ch. It sits with the other counts and is greyed out the same way — a count above the narrowest track in the list would spread a recording into channels it was never sent with. The ladder's ceiling for eight channels is the codec's top rung.
tests/run_surround71_tests.sh does for 7.1 what the downmix suite does for 5.1: a tone
per channel, and the spectrum read back. AAC in a transport stream, an MP4 and a
Matroska file, FLAC, Opus and PCM in Matroska, and Blu-ray LPCM in an m2ts are cut,
re-encoded and folded to 5.1 and to stereo; every output's pictures are decoded as well,
and AC-3 at 7.1 has to be refused with a reason. FFmpeg's E-AC-3, TrueHD and DTS
encoders stop at 5.1 and fold without being asked, so the disc codecs at 7.1 have no
fixture here; nor does 7.1(wide), which FFmpeg's AAC encoder writes with a program
config element and its own decoder reads back as eight channels in no named order.
A sample has three properties, and the channel count is only the first. The other two are how often the sample is taken and how wide it is written, and both behave the way a downmix does: each decides the mode rather than obeying it. Samples on a 44.1 kHz grid cannot be spliced in among a 48 kHz recording's frames, and a 16-bit sample is not a 24-bit one. Either setting therefore makes the cut a whole-track re-encode, and SmartCut says so before it starts.
The resampler is swresample's again, but where the downmix's is per-frame this one is one context for the whole track, and that difference is the whole design.
A resampler is a filter. The output grid does not land on the input grid, so part of a sample is always held back between calls, and a context rebuilt per frame would drop that remainder every time — which gives you a click at every frame boundary rather than a track.
So the two stages are kept apart. First the channels are put right, frame by frame, on the way in: there the input and output rates are the same, so the sample window each keep-range is trimmed against still counts in the recording's own samples. Then the kept samples, already spliced end to end, run through one resampler on their way to the encoder. Whatever that filter is still holding at the end of the track is pushed out with a flush; without it the last few milliseconds would simply be missing.
Three places have to hear about the new rate, and missing any one of them is audible:
- The samples themselves, which is the resampler's job.
- The output stream's declaration. Its time base is the rate, and a re-encoded packet's timestamp is its sample count divided by that rate. It has to be the output's rate, not the recording's, or an MPEG-TS packet lands at the wrong second.
- The ADTS header on every frame, for AAC in a transport stream. A track resampled
to 44.1 kHz whose frames still announce 48 plays 9% fast to any decoder that believes
the header ahead of the payload.
AdtsFormat::with_rateputs the sampling frequency index right, the same waywith_channelsputs the channel configuration right.
Not every codec supports every rate. AC-3 has three — 32, 44.1 and 48 kHz — MP2 has
those and their halves, and Blu-ray LPCM has 48, 96 and 192. An encoder handed a rate it
does not list refuses to open at all. That is the same failure an unlisted sample format
or channel layout produces, and it is answered the same way: by asking the encoder.
audio::writable_rate moves the rate asked for to the nearest one the codec has,
preferring the higher of two equally near, and the cut reports which rate it settled on.
The question is asked before anything is opened, so the stream is declared at the rate
its packets will actually arrive at. A lossless track carried through whole is not asked
it, since no encoder is opened for it: up to 0.8.25 a 96 kHz DTS-HD Master Audio track
was held to the list of libavcodec's DTS encoder, which stops at 48 kHz, and came out
re-encoded whole as a 48 kHz DTS core.
A width only means something where the samples themselves are being written. Every lossy encoder here takes a float and spends a bitrate; how many bits the sound had beforehand is not a number any of them has anywhere to put, so a width asked of AAC is declined with a message rather than quietly ignored.
Where it does mean something, it means a great deal. In an MP4 it picks the codec
outright — pcm_s16be or pcm_s24be, since there is no single box for PCM of any
width. In a transport stream it picks the sample format the Blu-ray LPCM encoder is
opened with: 16-bit, or 32-bit carrying 24. In an MKV or a MOV, a recording's own plain
PCM is written at the width asked for, in its own byte order; up to 0.8.23 it was
re-encoded, said to be at that width, and came out at its own again, because each of
those encoders writes one width only. Float samples asked for as 16 bit, and 8-bit
samples asked for as 24, went the same way up to 0.8.24; they are written as 16- and
24-bit integers now. That is also the one way into an MP4 or a Matroska file for the
signed 8-bit samples an old QuickTime file holds (pcm_s8): neither has a tag for
them, and up to 0.8.25 a cut into either stopped with a bare "Invalid argument" after
creating the file. It is refused before the cut now, and the message names
--audio-bits 24. Matroska does not name the channel layout of its PCM either, and an
MP4 has to state one, so a copy of a .mkv's PCM into an .mp4 stopped on its first
packet with the same "Invalid argument"; it is given the ordinary layout for its
channel count now. The width also decides the file's size on its own:
channels × width × rate, with nothing else in it.
It also decides the frame length. The Blu-ray LPCM encoder frames 16-bit sound 240 samples at a time and 24-bit sound 360, which is why the width was already being settled before any of this became a setting. A 16-bit recording asked for as 24-bit comes back framed differently from itself, and a frame that does not line up with the recording's own cannot stand in for one.
tests/run_audio_format_tests.sh reads the rate out of all three places, checks that a
rate a codec does not have comes back as the nearest it does, reads each width off the
size of the file it produced, and finishes by measuring A/V sync through a resample on
the impulse fixture — 0.02 ms at worst, against the 1 ms bar a whole-track re-encode is
held to.
A cut of a broadcast is AAC because the broadcast was, and for most of what this program is for that is the end of it. It is not the end of it when the file has somewhere to go afterwards: a disc player and an AV receiver want AC-3 or DTS, and an editor that will encode again wants the samples themselves rather than a second generation of something lossy.
Naming a codec is the second setting that decides the mode rather than obeying it,
for exactly the reasoning that applies to a downmix. The only frame that can be copied
is one already in the codec being written, so a codec that is not the recording's leaves
nothing to splice: smart and copy have nothing to offer, the cut becomes a
whole-track re-encode, and SmartCut says so before it starts. Asking for the codec the
recording already carries is not a conversion and changes nothing; the mode stands.
| Asked for | Into a transport stream | Into MP4, MKV, MOV |
|---|---|---|
aac |
AAC, ADTS framed | AAC |
ac3 |
AC-3 | AC-3 |
dts |
DTS | DTS |
lpcm |
Blu-ray LPCM (pcm_bluray) |
pcm_s16be, or pcm_s24be from a deep source |
Four things had to be settled to make that table true.
- The ADTS framing is AAC's alone. Everything this program encodes into a transport stream is framed, because a re-encoded frame has to be the same shape as the copied frames beside it. But ADTS is AAC's framing and nothing else's, and six bytes of it in front of an AC-3 frame is six bytes a decoder will try to read as a frame. The framing is therefore conditional on the codec being written, not on the codec that arrived.
- The programme map has to describe what is actually in the file. A downmixed track is
still the codec it was, so the recording's own map entry stays true of everything but the
channels; a track re-encoded as AC-3 is not, and an entry that says MPEG-2 AAC will have
a player hand AC-3 frames to an AAC decoder. Where the codec changed, the entry is
written from what the cut produced: 0x0F for AAC (0x11 where it goes out in
LATM), 0x81 and an
AC-3registration descriptor, 0x82 for DTS, 0x80 for LPCM. - LPCM in a transport stream needs the programme registered as HDMV. There is no
standard way to carry raw PCM in an MPEG-2 transport stream; there is only Blu-ray's,
which is a private stream that means LPCM because the programme registers itself as HDMV
and not otherwise. libavformat's muxer writes that registration when it is asked for
Blu-ray's own framing and not when it is asked for a plain
.ts, where it falls back to declaring private data of no stated kind — which reads back asbin_dataand decodes to nothing. The bytes are identical either way, so the cut writes the registration and the 0x80 into the map it was already rebuilding. Into an MP4 or an MKV there is no such problem: those have a box for plain PCM, and the samples go in big-endian at the recording's own width. - The encoder's own idea of a channel layout. Six channels are not one arrangement:
libav's default for six puts the surrounds at the back, FFmpeg's DTS and Blu-ray LPCM
encoders list only the arrangement with them at the sides, and
avcodec_open2refuses to open on a layout the encoder does not list. That is the same failure as an unlisted sample format, and it is answered the same way: ask the encoder, and resample into what it says. Smart-rendered patch frames are the exception. Those go back among the recording's own frames, so the layout asked for is the layout that arrived, and the only conversion is one of format.
Bitrates are the codec's, not the recording's. Following the recording's rate is right
while the codec is the recording's and meaningless once it is not: 384 kbit/s is what a
5.1 broadcast's AAC cost, it is not what the same programme is worth as AC-3, and as DTS
it is not a rate the format has. So where the codec changed, the derived rate is what that
codec is ordinarily carried at for that many channels — AC-3 at 96 / 192 / 448 kbit/s for
mono, stereo and surround, DTS at 768 and 1536 — and --audio-bitrate is taken as given
whatever the codec.
LPCM has no rate to choose at all. Its size is arithmetic — channels × bit depth × sample rate — and the encoder throws away any figure handed to it. So the window greys the control out and puts the result of that arithmetic in it: what an uncompressed track costs is the one thing the control can usefully say, and it can say it exactly.
All three numbers are the output's rather than the recording's. The channels are what is
being written, so a fold halves the figure; the rate is what --audio-samplerate asked
for, or the recording's; and the depth is what --audio-bits asked for, or what
[audio::pcm_bits] settles — 24 only where the recording has more than 16 bits in it,
which off a broadcast it never does. And there is a third term in a transport stream:
Blu-ray's LPCM writes its channels in pairs and pads an odd count with a silent one, so a
mono track in a .ts costs two channels' worth of bytes and the window says 1536 kbit/s
rather than 768.
DTS is behind libavcodec's experimental flag. Its encoder refuses to open unless the caller says that is understood, which the cut does, once, for that codec. Here "experimental" is libav's account of how much attention the encoder has had; what comes out is a DTS stream a receiver decodes without complaint.
Not every combination of codec, channels, rate, width and bitrate is a file. Blu-ray LPCM has 48, 96 and 192 kHz and nothing between. DTS is written mono, stereo, quad, 5.0 or 5.1 and in no other count, because those are the arrangements its encoder lists, and a DTS frame carries a fixed number of samples and has to be long enough to describe every channel in it — which puts a floor under the bitrate that moves with the channel count and with the sample rate. 5.1 at 48 kHz is not written under about 670 kbit/s; the same track at 32 kHz is, because there the frame covers more of a second.
None of that is worth discovering at the end of an export, and none of it is worth keeping a table of either: the answers belong to whatever FFmpeg the build was linked against, and a table here would go quietly out of date. So the question is answered by opening an encoder and seeing what happens.
[audio::opens_at] is that question. Everything an encoder can refuse over is settled
before it by asking the encoder — the rate in encoder_rate, the arrangement in
encoder_layout, the sample format in encoder_format. Each either finds something the
encoder listed or finds nothing, and finding nothing is now reported plainly rather than
left to avcodec_open2 and its "Invalid argument". What is left after all three is the
bitrate, which no encoder lists, and that is what opens_at is for.
Two callers read it. The first, [cut::writable_sound], answers for a whole screen:
given the audio tracks a list of clips will write, the settings currently held and the
lists the window offers, it says which of those answers a cut could actually be given.
It works one list at a time, judging each against the other settings as they stand, and
every track has to agree. The output settings screen greys out the rest. Where the
settings currently held cannot themselves be written — a project made before there was a
floor to know about — they are let go one at a time, bitrate first and codec last, so
that the lists are judged against something writable rather than all coming back empty.
The window adds one refusal of its own, and it is not a question for an encoder: nothing
above what the recording has. Channels, rate and width can all be written above what
came in — an encoder opens perfectly well at six channels when handed two — but none of
the three brings anything with it except size. soundCeiling in app.js takes the least
of each across every track the list will write, greys out everything above it, and drops
a control whose answer no longer fits. audioChannelsOut, audioRateOut and
audioBitsOut do the same arithmetic again where the answers are handed to the cut, so
that a count chosen against a 5.1 recording cannot reach the cut of a stereo one that
joined the list later. Being arithmetic rather than an encoder open, it is answered on
the spot rather than off the thread. And it is the window's rule rather than the
engine's: the command line will still resample upwards if you ask it to.
The cut reads it too, for the callers a window does not cover. A bitrate the encoder
will not open at is no reason to stop a cut: what the codec is ordinarily carried at is
above every one of these floors and is a figure the format has, so that is written
instead and reported as it goes — the same shape of answer writable_rate gives for a
rate a codec does not support.
tests/run_audio_codec_tests.sh asks for each of the four codecs into each of four
containers, and checks the track is the codec asked for, that every channel of a 5.1 fixture still
carries the tone it went in with, and that a transport stream's map declares the codec
that is in it. It also asks for 384 kbit/s of 5.1 DTS, which is under the floor, and for
three channels of it, which is a count DTS has no arrangement for: the first is written
at 1536 kbit/s with a note and the second is refused in a sentence that mentions
channels. The unit tests beside writable_sound cover the lists themselves.
A bilingual broadcast sends its sound on two separate PIDs. Only one used to be
picked up, by best(Type::Audio), so the second language was never even read.
Source.audios now lists every sound track the recording carries. Source.audio
stays, as the main one: commercial detection, preview playback and the .aac sidecar
each read a single track, and which one that is remains a real question.
A broadcast announces what it is sending when the programme starts, and a recorder starts before that. The head of the file is the end of whatever was on before, with one sound track in it — so the second track is missing from the map, from the stream, or from both, and libavformat stops probing after five megabytes, which on a Japanese broadcast is about two and a half seconds.
Measured over 400 broadcast recordings: 37 carry two sound tracks, and 22 of those name the second only past where the probe stops — one of them 7.2 megabytes in. Those 22 arrived as ordinary single-track recordings. The track was not listed, not described, and not carried into the cut.
The deeper probe that was being spent on recordings with no captions — see
input.rs and The captions the head of the file
does not mention in
broadcast-ts.md —
is now spent on every broadcast recording. The sound cannot be asked the cheap question the captions were
asked — "did none arrive?" — because three quarters of the recordings that turn out to
have two tracks look exactly like the ones that have one until the deeper probe is
spent. Measured at a fifth of a second on a file read cold off a network share, and
nothing at all on one already in the page cache. Not on a disc: a clip's map is written
once by an authoring tool, and everything it carries is in the first copy of it — a
recorder's clip being the exception, below.
The recording's own map is read further for the same reason. si::stream_tags is
handed the PIDs the demuxer found, and a map that does not name them all is a map read
before the broadcaster had finished announcing the programme — so the window opens from
four megabytes to sixteen, once, exactly as read_service already did for the captions.
Unless told otherwise, libavformat follows the first map it reads and no other: a
map replaced later by one naming another sound track does not make it a stream.
scan_all_pmts, which reads every map, is set by ffmpeg and ffprobe and asked
for in input.rs only where it is needed. A commentary track was lost two ways:
| Recording | What happened |
|---|---|
| A programme with commentary that a recorder wrote to a BD-RE | The clip opens on a map naming the picture, the main sound and the captions, and half a second in that map is replaced by one adding the commentary. The disc's own listing named it; the cut wrote the recording as one with a single sound track, and every cut of those programmes left it out |
| A broadcast recording whose map names its commentary late | The deeper probe lists the track but cannot describe it. Named 6 MB in, with its first frame past 7.6 MB, it was left out under a note saying the map names it but it never appears — which is not true |
On a transport stream whose table names one service, every map is now read and
the recording opened again (input::sound_named_later) where either holds:
- a sound track the open came back with has no rate or no channel count. The deeper probe goes with it, and the time libavformat spends looking at each stream goes from its five seconds to twenty;
- on a disc's clip, a copy of the map in the first 5 MB names as sound a PID the open does not have. That is as far as libavformat's own probe reads, so the bytes come back out of the page cache.
Never where the table names more than one service: reading every map there also turns up the phone-sized copy of the programme the multiplex carries beside it (see Reading a disc).
The second open keeps the first one's start and length (keep_the_clock). On one
programme the commentary's first frame is stamped 127 seconds ahead of everything
else; described now, it became the recording's start, the whole timeline slid two
minutes and two minutes of nothing opened it.
Of 68 heads of programmes whose map comes late, 7 gained their commentary and
lost the false note, and nothing else changed; 61 heads picked at random and the
heads of twelve stations' recordings open and cut exactly as before. On discs, the
five recorder BD-RE titles with commentary gained it, and every other title lists
and analyses as before. What it costs: every transport-stream title on a disc reads
its first 5 MB once more per open, and a broadcast recording whose commentary is
named late is opened three times per read — --analyze with a saved index on a
network share went from 0.2 s to 0.4 s.
Two tracks arrive for one of two reasons, and the sound cannot say which. A programme sent in two languages carries the original and the dub; a programme sent with commentary for a viewer who cannot see the picture carries the programme and the commentary. Both are two stereo AAC tracks at the same rate, described identically by the demuxer — of the 37 recordings above, 34 name Japanese on both tracks.
What separates them is the broadcaster's own name for each track, and that is in the programme description: the audio component descriptor of ARIB STD-B10, tag 0xC4, in the event information table. It carries the component tag the map names the stream by, a flag saying whether this is the main sound, the language, and the text a listing would print — 日本語, 英語, 日本語（解説）.
si::sound_tracks reads it, and passes over a description that names fewer
component tags than the map carries: the description on the air when the file opens
can be the previous programme's, and a programme with one sound track describes one.
It is passed over rather than half believed.
Of those that name every track, the one on the air is taken, not the first to arrive. The present and the following description come as two sections on their own schedules, and the following one — the programme after this one — is as likely to come first. A programme with commentary followed on the air by a ball game, whose second track is the stadium's sound, had its commentary named 球場音声: 7 of 54 two-track recordings in the corpus took the next programme's description this way. So the window is read to its end, and the answer is the description of the event last seen present; failing that, any present one; failing that, the first, which is then the following programme's — a recording that opens before its programme, on one whose description names fewer tracks — and the right one.
A programme on the air that describes no sound at all names nothing. It has
not described fewer tracks than the map carries; it has said nothing about them.
That is what a cut SmartCut wrote with its sound folded, resampled or in another
codec looks like read back: the description of the programme on now goes out
with its audio component descriptors taken off, since they no longer describe
the track (only the description of the programme on now is
trimmed),
while the following programme's is left whole. Taken as the fallback above,
that next programme named the cut's tracks: a two-track drama cut with
--audio-codec ac3 read back with the next programme's names on its tracks (the
language, which the map states, stayed right). Such a file's tracks now go
unnamed, and the map's tags decide the main one, as on a recorder's clip.
Asked only where there is more than one track — on the other 363 recordings there is one track, it is the main sound whatever any table says, and the read would buy nothing. Measured over the 37: the first description naming every track arrives between 3.6 and 15.3 megabytes in, so the window is sixteen.
The main track is then the broadcast's own answer rather than best(Type::Audio),
which weighs the disposition flags a container carries and on a broadcast is picking
the wider of two identical tracks. On a two-track recording measured here it named the
second — so a preview opened on the commentary, and the commercial detector listened to
it.
A transport stream whose recording describes none of its sound — a recorder's
BD-RE clip carries no programme description at all — still has its map, and the map
names each track by its component tag: ARIB gives the main sound 0x10 and the second
0x11. So the main track there is the one with the lowest tag, or, where the map tags
none, the first sound the map lists that is not marked for viewers who cannot hear or
see the programme (by_map in outline_of,
lib.rs). A DVB broadcast marks its audio
description that way, and one that lists it first had it named the main sound. Left to
best, which breaks a tie between two stereo AAC tracks on the bit rate, 5 of the 10
titles with commentary on two recorder discs opened on the commentary. best is still
the answer for every other container, and for a transport stream whose map could not
be read. The main track decides which track the editor's track list marks as the main
one and which one the detections listen to; what a cut writes is the same either way.
On the way out, each track is cut on its own. That is the whole point, and the reason the writer's state was split from one set into one set per track. Two tracks have their frames at different instants, so:
- a boundary falls inside a different frame on each, meaning a different number of frames get re-encoded;
- each accumulates a different drift from range to range;
- and they need not even be framed the same way — one MPEG-2 AAC, the other MPEG-4.
Use one track's answer for the other and the other slips a frame per range. So an
AudioTrack holds everything true of one track alone — its output stream index, how far
it has been written, the previous frame's pts, its table of re-encoded boundary
frames, its re-encoder — and the Writer holds a list of them.
Options like --audio-channels apply to every track, but the decision is per track:
only a track that actually has to be folded becomes a whole-track re-encode, and a
track that was already stereo is still copied.
A track that is not wanted is dropped with --drop-stream <stream index>, or in the
GUI with the cut editor's track menu. The default is to write them all: a track
nobody asked about is a track that was in the recording, and dropping it silently would
be this program deciding what the recording is for.
The re-encoding path was a frame out, and had been all along. It counted the encoder's
delay itself, from initial_padding, dropped that many opening packets, and
subtracted the same amount from their timestamps — taking it off twice.
A libav encoder declares its delay by stamping its output: the first packet comes out
at pts = -1024, covering the window that reaches back before anything was fed, and the
next at pts = 0, covering the first frame that was. The packet says where it belongs,
so the fix is to believe it: drop what lands before zero and take the timestamp as the
position. The A/V length skew in run_audio_tests.sh under reencode, which peaked at
21.3 ms — exactly one frame — now peaks at 16.0 ms and averages 6.4 ms.
Two encoders do not stamp what they hold, and each is corrected by hand in
audio::Reencoder. libavcodec's DTS encoder runs its input through a
512-sample analysis bank and declares a delay of nought, so every packet holds
the frame fed before it, stamped as its own: a track re-encoded as DTS was
10.7 ms behind its pictures end to end and lost its last frame. Its packets are
now placed 512 samples earlier (undeclared_delay), and a frame of silence at
the end pushes the last of the track out. An Opus encoder states its delay
a second time, as the pre-skip in its OpusHead, for a player to drop off the
head; the packet that delay sits in is never written here, so a whole track
re-encoded as Opus played 6.5 ms early. The pre-skip is written as nought.
Every other encoder here declares its delay exactly, measured with an impulse
at 48 and 44.1 kHz.
Impulse material can only answer "how does it behave on material we made". "When a real broadcast recording is cut, is the sound that comes out really the sound from that place?" is a different question.
So tests/run_audio_content_tests.sh decodes 6 seconds of the output and finds, by
cross-correlation, where in the recording it matches. Dropped, muted or misplaced audio
all fail this.
What matters is not the absolute offset but whether the offset changes from interval to interval. Decoder priming and the container start time apply equally everywhere, so a constant offset belongs to the measurement. If it changes, that is a bad seam.
Measured on two intervals of real material (Nihonkai TV, 30 minutes):
| Audio | Container | Correlation | Offset spread across seams |
|---|---|---|---|
| copy | MP4 | 1.000 | 31.0 ms |
| copy | TS | 1.000 | 31.0 ms |
| sample-accurate | MP4 | 1.000 | 0.0 ms |
| sample-accurate | TS | 1.000 | 0.0 ms |
The sound is unquestionably there — correlation 1.000 across every interval, with RMS within 1 dB of the source. On top of that, leaving audio on copy shifts the sound by up to about 30 ms per seam, and it stacks up with the number of seams. AAC can only be cut on a 21.3 ms frame grid, so the boundary rounding shows through directly. The same 31.0 ms appears in MP4 and in TS, so this is a property of the cutting side, not of the muxing.
Re-rendering sample-accurately brings it to 0.0 ms (--audio-mode reencode). It is
nonetheless kept out of the GUI, for two reasons, both from measurement:
- On actual commercial cuts, copy showed no drift. The 31.0 ms above comes from two intervals deliberately placed on boundaries that are not lossless points. Commercial detection snaps its boundaries to access points, and across a five-interval export of real material all six seams were a constant +5.7 ms, unchanged from interval to interval. The offset is per-seam rounding, not something that stacks, and placing the boundaries well removes almost all of it.
- Re-rendering changes the shape of the ADTS and breaks downstream tools. Japanese
broadcast AAC is
ff f8(MPEG-2, with CRC) while ffmpeg can only writeff f1(MPEG-4, no CRC). Indexing software written for ARIB fails to read it.
The judgement is that what is lost — downstream tools cannot read the file — outweighs
what is gained, an improvement of roughly 0 ms. It is still in the engine and the CLI,
so --audio-mode reencode is there if it is needed.
What this writes is the audio file the cut would have had: the ranges, the joins and the fades already in it, and no pictures anywhere.
A writer of its own rather than a switch in crate::cut. That writer is
built around the pictures — a range is chosen at an entry point, a segment is
copied or re-encoded by what the pictures need, the progress is the writing
head moving through them, and the output's clock is a grid of fields. None of
it has anything to say about an audio file. Asked to write no pictures it
would still read them, plan them, and spend the whole of a run's time on
material it then threw away: a two hour recording is ten gigabytes written to
keep three hundred megabytes. Measured on two recordings of eleven minutes,
the sound alone is written in under a second against four for the video.
Every decision is still made once. The track is planned by
cut::plan_audio, which is the one place that settles what a track becomes;
the frames at a boundary come from audio::boundary_patches, which is the one
place that re-encodes them; a whole-track re-encode goes through
audio::Reencoder. The sound of an audio-only run and the sound inside an
ordinary cut are the same sound because they are made by the same code. What
this module adds is the walk: which packets each range claims, and where they
go on a timeline that has no pictures in it.
Frames are laid end to end, as the picture writer lays them. They are whole, so a range's worth of them is a whole number of frames however the range was chosen, and end to end is the only arrangement that neither overlaps nor drifts.
Each range opens against where the sound so far has got to. Whole frames
leave every range up to half a frame off its own length, and until 2026-10-16
the next range opened on the frame nearest its own start whatever the ranges
before had come to. The errors added up: eleven ranges of an AC-3 track
ended 129 ms later than the sound of the same keeps cut with pictures, and AAC,
Vorbis, PCM and DTS went the same way. The cut with pictures carries
the difference over (pick_from in cut::take_audio), and the sound alone
now carries it by the same rule: before each range it measures how far the
sound written runs from the ranges before laid end to end, and chooses the
range's first frame against its start moved by that much. Every range lines up
within a frame with the sound of the video cut. Where the sound is more than
half a second behind, the track had a hole rather than a drift — a broadcast's
sound stops before its pictures — and the count starts over, as the picture
writer's does, so sound the cut took away is never pulled back in. A single
range has nothing before it and is written exactly as it was.
The track's very first frame is counted too. The cut with pictures lays it
at its own instant, some milliseconds off the range's start, and sums how
far the sound has reached from there, so the drift every later range opens
against carries that offset. The sound alone took it for the rounding of the
opening rule and counted nothing; from the second range on the two counts
differed by it, and where that crossed the middle of a frame a range opened a
frame away from the cut's. The first frame's offset is now counted the way the
cut counts it, and where a range's ends sit on pictures (as the editor sets
them) every range after the first opens on the frame the cut opens it on.
(Times on a millisecond clock, as in a .mkv, still part by a fraction of a
millisecond a range, and over many ranges a head can come out a frame off.) A
command-line range whose ends fall between pictures, --keep 5-12 say, is
moved onto them first, as the cut with pictures moves its sound
(on_its_pictures, which plans the range as the cut does and asks
cut::sound_span): until 2026-11-10 the cut counted such a range in whole
pictures and the sound-only writer in seconds, and its later heads could open
a frame apart. The first range's own head keeps its own rule (below).
A hole in the track is never taken for drift. Where a range's own sound starts late (the recording's sound begins after its pictures, or the track breaks off at the range's head) or breaks off inside the range, the missing stretch is left out of the count, so the next range does not open early on sound the cut removed. Sound laid end to end cannot hold the hole, so the rest of the file sits that much earlier than the cut with pictures, which keeps it as silence. Where the sound's own times step back (seen about a second before some recorder seams), the frames that repeat earlier sound are skipped, as the cut with pictures drops them.
A range kept from the head starts where the cut's sound starts. An
ffmpeg-made .mkv stamps its AAC's first frame, the encoder's priming, at
-0.021 s, ahead of a first picture at 0. The cut with pictures starts the range
on that picture and claims the frame nearest it, so the priming frame is not in
its sound. Read from the range's own start, the sound alone opened on it, and
the whole file ran one frame (21 ms) late against the cut's sound: 282 frames
for 281 in six seconds. Such a range now starts where the cut starts it
(primed_from). Only that frame goes: the half second a broadcast sends ahead
of its first picture is the recording's own and is kept as before.
A recording joined on starts on its first picture. The sound a recording
sends ahead of its first picture is its own only at the head of the file;
joined on, it is at the head of nothing, and the cut with pictures starts that
recording on its first picture. The sound alone now does the same
(joined_lead, where the range runs from before the recording's first entry
point). Up to 0.8.44 the lead went in at the join, on the end of the
recording before -- 0.89 s of a satellite recording's -- and a join's fade-in
was spent on it, so the programme came in at full level.
The frames of a join's fade-in are decoded from a seek, and a seek to the head
lands on a picture, where a broadcast's sound starts half a second before it.
So in smart mode, a joined transport stream kept from its head had its fade-in
left off the sound-only file's first half second, everything before the first
picture copied at full level, where the cut with pictures, which starts on that
picture, had the fade whole. Where the frames from the seek begin after the
edge, they are now read on from a fresh open, as the writer reads them
(decode_around in audio.rs).
An overlapping crossing takes the same seconds out of the sound as it does
in a video run. A dissolve, a wipe or a slide shows the head of the next clip
while the one before it is still sounding, so the video's writer starts that
clip later by the crossing's length, sound and all
(cut::ranges_with_transitions). The sound alone now makes the same cut with
the same arithmetic — each end gives at most half its range, and the pair is
held to the smaller. Before 0.8.1 it did not, and the file came out a
crossing's length longer at every join. A fade takes nothing from either side.
A recorder's seam takes the same clock out of the sound as out of the
pictures. The picture writer plans a range across a seam as two, and leaves
out the clock from where the pictures before it stop to the first entry point
after it (Source::unwritten).
sound::written splits every range the same way, keeping the piece after such
a stretch only where it is more than half a picture long, as the planner keeps
it only where an entry point lies inside it. Up to 0.8.23 the ranges were taken
whole: kept across a seam whose clock runs on for 18.9 seconds with no picture
in it, a .wav came out 35.0 seconds long, 18.9 of them silence laid in by the
re-encoder, where the cut with pictures is 16.1. --verify measures a file of
sound alone against the ranges this function returns, so the two cannot come
to differ.
A joined recording whose sound is another shape re-encodes the track. The
output stream is declared as the master's (the first recording's unless
--master or the reference clip names another), and packets of another
codec, rate or channel count cannot be copied into it. Before 0.8.5 the
master was not asked, and the file always took the first recording's shape. The video's writer
re-encodes only the reels that differ; with one track and no pictures,
re-encoding all of it comes to the same file.
Not where the master's sound is TrueHD, MLP or DTS and no codec was named
(--audio-codec). That sound is carried through as it is, and nothing here
re-encodes it (audio::carried_whole), so the rule is the cut with pictures':
a recording whose sound is the same codec in another shape -- 2.0 TrueHD
joined onto 5.1, 44.1 kHz DTS onto 48 kHz, a DTS core onto DTS-HD Master
Audio -- is carried frame for frame, its frames saying what they are; one whose
sound is another codec is refused before anything is written, in the cut's
words (see joining). Up to 0.8.27 both were re-encoded as any
other shape is: a DTS track went whole through libavcodec's encoder, which
writes the core alone, the master's own frames included, and a TrueHD one
stopped on the encoder's "Experimental feature".
The file's name picks the container, and the container picks the codec where
it insists: carriage answers for the containers a cut goes into, where
big-endian PCM is what an MP4 has a box for, and a .wav wants its samples
the other way round. Asked for the wrong one it says "Function not
implemented" — after creating the file. So a PCM target is taken to whatever
that container writes PCM as — keeping its width, since the muxer names its
sixteen-bit default and a 24-bit track would otherwise be cut down to it, and
keeping float samples float: up to 0.8.25 an .aiff, .au or .caf was
handed float sound as sixteen-bit integers, --audio-bits 24 or not. The
CLI writes a .wav as linear PCM whatever the recording's own sound is. Where
that sound is 32-bit float and no width is asked for, a .wav or .aiff of it
alone is written in 32-bit float; up to 0.8.44 it was 24-bit integers, which
clip whatever went over full scale. 64-bit float goes to 16-bit integers, as
before. Matroska holds float samples little-endian only, where it holds
integers either way round, so a QuickTime file's big-endian 32-bit float
(fl32) asked for as an .mka is turned the other way round, as a .wav
holds it, which loses nothing; up to 0.8.45 it was refused. And a
container with no room for the codec at all — MP2 in an .m4a — says only
"Invalid argument", once the header is being written; avformat_query_codec is
asked first, so the run stops naming the codec and the file.
One track. A bilingual recording has two, and an audio file with two tracks in it is a thing only some containers hold; the first kept track is written and the run says so.
It is asked for in the container list rather than beside it, because it answers that list's question -- what kind of file comes out -- and a file with no pictures in it is not a variant of one that has them.
Six controls: one above five, though the five are on screen only some of the time. Audio picks the mode — smart rendering, copy through, re-encode everything — and under it Audio codec, Audio channels, Sample rate, Bit depth and Audio bitrate describe that encode. They exist for two cases: a 5.1 recording that has to play somewhere that will not take it, and a cut that has somewhere to go after this program is done with it.
Audio codec offers the recording's own, AAC, AC-3, DTS and linear PCM. See Choosing the codec for what each is for and what had to be settled to write them.
Audio channels offers the recording's own count, 1ch, 2ch, 5.1ch or 7.1ch, named by
the count rather than by the direction. Which way it goes is the recording's to decide,
not the setting's: one list can hold a 5.1 recording and a stereo one, where 2ch folds the
first and leaves the second alone. The readouts say which happened. (The engine takes any
count from 1 to 8 through --audio-channels; the window offers the ones that get asked
for. See 7.1.)
Sample rate offers the recording's own, 96, 48, 44.1 and 32 kHz: what a Blu-ray's
LPCM is carried at, what a broadcast uses, what a CD uses, and what a small file gets
away with. Like a downmix, it is either a whole-track re-encode or nothing at all. Unlike
a downmix, the codec may not have the rate, in which case the cut writes the nearest one
it does have and says which. Two cases among the four offered fall outside a codec's
list. First, Blu-ray LPCM has 48, 96 and 192 kHz and nothing between, so 44.1 asked of a
.ts comes back as 48 while 96 is written as asked — and the window does that arithmetic
too (writableRate), because the figure it shows in place of an uncompressed track's
bitrate has to be the size of the file that will actually be written. Second, 96 kHz
belongs to AAC and Blu-ray LPCM alone: AC-3 has 32, 44.1 and 48, and DTS goes no higher
either. There the engine tells the window so, and the window greys the rate out rather
than silently substituting another — a rate asked for outright and written as something
else would make the screen say something untrue. See
The rate and the width.
96 kHz is on the list for the recordings that have it, and the ceiling below decides which those are: a Blu-ray carrying LPCM or lossless sound at 96 can be asked for 96, and a broadcast sampled at 48 has it grey, alongside the channel counts and the widths it never had either.
Bit depth offers the recording's own, 16 and 24 bit, and is the one control here that does not answer to the mode alone: it is greyed out unless linear PCM is being written. Every other codec writes a description of the sound rather than the sound itself, and has nowhere to put a sample width. That makes it the mirror image of the bitrate below it — exactly one of the two is grey at any time, because an uncompressed track's size is arithmetic nobody chooses, and a lossy track's sample width is a number nobody can name.
None of them is on screen unless the mode is "Re-encode everything", because they describe
an encode and the other two modes do not run one over the whole track. copy runs none at
all, and smart runs one on two frames per boundary, where the whole point is that they
come out the same shape as the frames they are spliced between. Five dead rows under a mode
that has no use for any of them are five rows to read past every time the panel is opened,
and the panel has grown, so under the other two modes they are simply not there. They are
put away rather than cleared: what they hold is still there when the mode comes back
round to wanting it. While they are away, what they hold does not reach the cut either —
the window sends these settings only when it is re-encoding, so the screen and the file
always agree.
Bit depth and bitrate are the exception within the encode, and are greyed out rather than hidden. Both belong to the encode being written, and which of the two is live changes with the codec, so hiding and showing them would make the panel jump around while somebody is choosing a codec.
The engine takes a harder line, and has to: there, audio_channels,
audio_sample_rate and audio_bits each force a whole-track re-encode whatever the mode
says. A stereo frame cannot be spliced among a recording's 5.1 ones, nor a 44.1 kHz one
among 48 kHz ones, and there is no honest way to half-do it. The window never puts the
engine in that position; the CLI can, and prints a note when it does.
Audio bitrate on "Leave it to the engine" means the engine derives one from the recording, bringing it down with the channel count, since what 5.1 cost is not what the stereo it was folded into is worth.
The rungs are the codec's own, and where the ladder stops moves with the channel count:
| Codec | Rungs | 1ch | 2ch | 5.1 | 7.1 |
|---|---|---|---|---|---|
| AAC | 64 … 640 kbit/s | 192 | 384 | 640 | 640 |
| AC-3 | the format's own table, 64 … 640 | 192 | 384 | 640 | — |
| DTS | the format's own table, 384 … 1536 | 768 | 1536 | 1536 | — |
| linear PCM | — | — | — | — | — |
AAC's rungs are where a broadcast puts them, with room over the top; AC-3's are where a disc puts them, and 640 is the format's own limit; DTS has two rates anyone uses, 768 and 1536, and the ladder is the way between them. AC-3 and DTS both carry the rate as a number out of a table the format defines, so only the rates in that table are offered — a figure between two of them is one the encoder would round wherever it liked. "Same as the input" is not in the table at all: what a clip carries is not known until it is read, and for a broadcast it is AAC, so AAC's ladder is the one shown.
Linear PCM's row is empty because there is nothing to choose in it. Its size is channels × width × rate, and Bit depth and Sample rate are where those two are chosen. The control goes grey and the setting is cleared with it — a figure left standing behind a greyed-out control would come back as an answer nobody gave the moment the codec changed — and in its place the control shows what the track will actually cost, worked out as above.
The setting is one answer for a whole list, and the clips in it need not cost the same: a stereo recording beside a 5.1 one, both read as "same as the input", give two figures. The control then names the range rather than picking one of them, which is the same reasoning as the ceiling above taking the widest track in the list. The format panel's own audio line has only one clip to describe, so there the figure is always exact.
The top of the stereo ladder is headroom rather than a promise, because the encoder has a ceiling of its own and does not announce it. Asked for more than it can spend, it simply writes less; asked for a great deal more, it writes less than it otherwise would have, because the rate control overshoots and backs off. Driven with noise at 48 kHz, so that the encoder rather than the material is what runs out, mono levels off near 218 kbit/s and stereo near 250. Ask for 384 in stereo and what comes back is whatever the encoder found worth spending.
"Same as the input" has no count of its own, and the setting is one answer for a whole list, so the widest track in the list decides — the widest the ceiling could have to cover. A count with no ceiling of its own takes the next one up, a 4-channel recording being nearer 5.1 than stereo. A rate the list stops offering — because the channel count came down under it, or because a project was written when the rungs were different — is moved to the nearest rung at or below it rather than thrown away.
The panel's drop-downs open upward. File settings are the bottom panel of the
window, so a native popup there has nowhere to go: the bitrate ladder, sixteen rungs of
it, ran off the bottom of the screen with most of itself out of reach. Where a native
popup opens is the platform's decision rather than ours, so the popup is drawn here
instead (opensUpward) — but only the popup. The <select> stays where it was and goes
on holding the answer, so reading a setting off a control, putting one back when a
project opens, and translating the options all keep working untouched. What is drawn
carries two marks rather than one: the answer the control holds, and the cursor, which
the mouse and the arrow keys move together. The one thing to watch is that suppressing
the native popup costs the click its focus, so the focus is handed back by hand — a
control that cannot be used from the keyboard after being clicked would be worse than a
popup in the wrong place.
Two places say what will happen, because there are two screens:
- The format panel names it as part of the audio line —
Audio: re-encoded (5.1ch → 2ch, 48 kHz → 44.1 kHz, 192 kbps), and quick properties says what each row is written as —Audio: ac3, 48000 Hz, 5.1ch (default), the underlined words being the row's own choice — so a 5.1 clip can be picked out of a list of stereo ones before the output has been written rather than after. - The output screen is about pictures, and its best line is "nothing re-encoded, the whole
clip is copied losslessly". That stops being true of the file as soon as the audio is
re-encoded end to end, so the audio's own fate is appended to it:
(the audio is downmixed 5.1ch → 2ch and re-encoded).
Every one of these settings travels in the project file, which needs no code: the file carries whatever keys the settings object has.