Skip to content

Latest commit

 

History

History
299 lines (242 loc) · 16.3 KB

building.md

File metadata and controls

299 lines (242 loc) · 16.3 KB

Building

← Documentation ・ ← SmartCut ・ 日本語

SmartCut builds on Linux (Debian 13 / Ubuntu 24.04 or newer). The Windows build is cross-built from Linux too.

Requirements

FFmpeg must be from the 7.1 series. ffmpeg-sys-next selects its bindings by version, so a different series gives you a mismatched API.

# Build essentials (bindgen uses clang / libclang)
sudo apt install build-essential pkg-config cmake clang libclang-dev

# FFmpeg 7.1 development headers
sudo apt install libavcodec-dev libavformat-dev libavutil-dev \
                 libavfilter-dev libavdevice-dev libswscale-dev libswresample-dev

# Prerequisites for the Tauri GUI (skip if you are only building the core)
sudo apt install libwebkit2gtk-4.1-dev libgtk-3-dev libayatana-appindicator3-dev \
                 librsvg2-dev libsoup-3.0-dev libjavascriptcoregtk-4.1-dev \
                 patchelf desktop-file-utils xdg-utils

# Rust
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
Item Version
FFmpeg 7.1 series (developed against 7.1.5)
Rust 1.98 or newer
WebKitGTK 4.1 (Tauri v2)
Node.js 24 LTS (used to bundle the GUI)

Building

# Rust core and CLI
cd rust && cargo build --release
# -> rust/target/release/smartcut

# GUI
cd gui/src-tauri && cargo build --release
# -> gui/src-tauri/target/release/gui

# The GUI under the name it ships as
cd gui/src-tauri && cargo tauri build
# -> gui/src-tauri/target/release/smartcut

The cargo crate is called gui, and it is mainBinaryName in tauri.conf.json that renames the binary, so the name smartcut only appears on a bundling build.

For how the AppImage, tar.gz, deb and the Windows installer are produced, see Distribution.

Tests

bash tests/run_tests.sh               # Python E2E                                     13
bash tests/run_rust_tests.sh          # Rust E2E (+11 with the container index)         29
bash tests/run_audio_tests.sh         # A/V sync (+10 with copy and reencode)            5
bash tests/run_audio_content_tests.sh # is real material's audio in the right place      6
bash tests/run_aac_tests.sh           # what the output's AAC frames are made of         8
bash tests/run_downmix_tests.sh       # where 5.1 goes when it is folded to stereo       9
bash tests/run_surround71_tests.sh    # 7.1 through every codec, each channel in place  14
bash tests/run_audio_codec_tests.sh   # writing the sound as another codec entirely     39
bash tests/run_audio_smart_tests.sh   # smart rendering, codec by codec                 24
bash tests/run_audio_format_tests.sh  # the rate and the width the samples are written  23
bash tests/run_audio_head_tests.sh    # a recording whose opening is another programme  22
bash tests/run_preview_tests.sh       # does a scrub show the time you asked for         7
bash tests/run_index_tests.sh         # does the index answer as the walk did           38
bash tests/run_proxy_tests.sh         # can the proxy stand in for the recording        22
bash tests/run_scene_tests.sh         # scene detection vs commercial boundaries         1
bash tests/run_ts_layout_tests.sh     # TS provenance and sequence headers               5
bash tests/run_broadcast_tests.sh     # captions, tables, data broadcast, dual audio    17
bash tests/run_bilingual_tests.sh     # a second sound track that comes late or goes early 9
bash tests/run_cm_tests.sh            # commercial detection vs a human's answer         5
bash tests/run_disc_tests.sh          # a BDAV and a BDMV disc, as folders and as .isos 38
bash tests/run_bdav_tests.sh          # writing a disc: the index, the image, and both   72
bash tests/run_udf_tests.sh           # the image as a filesystem, beside real discs     50
bash tests/run_dvd_tests.sh           # a DVD-Video disc, as a folder and as an .iso    23
bash tests/run_bd_audio_tests.sh      # the sound a disc carries, written out             39
bash tests/run_vc1_tests.sh           # the VC-1 encoder, put through a decoder          4
bash tests/run_vp9_av1_tests.sh       # VP9 and AV1: cut, and still the source's bytes   5
bash tests/run_sar_tests.sh           # SAR: 44 exports + 16 outlines                  60
bash tests/run_av1_mp4_tests.sh       # AV1 MP4 seam levels and Matroska controls       21
bash tests/run_vfr_tests.sh           # variable rate, both ways round                   8
bash tests/run_source_clock_tests.sh  # source clocks, range ends and VFR copying       95
bash tests/run_demux_tests.sh         # what a reader loses by reading what it does not  9
bash tests/run_transrate_tests.sh     # fitting a disc: the identity, the size, the cost 15
bash tests/run_join_tests.sh          # several recordings into one file, and the seams  17

Every suite passes whole. Anything that fails is a regression.

CI

tests/ci.sh runs every suite above that works on synthetic material, one after another, and names the ones that failed at the end. It leaves out the suites that only ever read real recordings (aac, audio_content, broadcast, cm, preview, pulldown, scene, ts_layout) and the VC-1 one, which wants a disc. Name suites to run only those, as bash tests/ci.sh join vfr. The whole set at once is too much for a small machine.

GitHub Actions (ci.yml) runs it on every push to main and every pull request, in Debian 13 for the FFmpeg 7.1 above, on a build with overflow checks and debug assertions on. Beside it, one job runs clippy and the unit tests and checks the GUI's scripts as the modules they load as, and another builds the CLI and the GUI for Windows and cuts the same synthetic recording as Linux does. The two cuts have to be the same file to the byte.

Fixtures

The synthetic fixtures (H.264 / HEVC / open GOP / 29.97 fps / MPEG-2 TS) are generated by run_tests.sh into /tmp/smartcut-fixtures/, so run that first. The suites that reuse them (run_rust_tests.sh, run_index_tests.sh, run_proxy_tests.sh) stop with run tests/run_tests.sh first to generate fixtures rather than quietly skipping half their checks.

run_disc_tests.sh builds a whole disc of each dialect out of mpeg2.ts — the stream remuxed into 192 byte packets, index files written around it by disc_index.py, and a UDF image wrapped over each by genisoimage, which it needs installed.

run_bdav_tests.sh goes the other way: it writes a disc of two recordings out of the same fixture, opens it with SmartCut's own reader, and then has bdav_index.py check every number in the index against the stream it is about — the entry point map followed into the file point by point, the arrival times, and what the clip index says the streams are. With a real broadcast recording in ~/media it also checks that the programme's name, the channel and the moment it went out survive a round trip through a disc. The last of it is the image: each UDF revision is written, opened again by this program, and -- where 7-Zip is installed -- unpacked by a reader written by somebody else and compared against the folder it was made of.

run_udf_tests.sh looks at the image the other way round: not as a disc that opens but as a filesystem that has to be well-formed before anything opens it. udf_shape.py reports the three layers a reader goes through -- the anchors and the descriptors they point at, the partition and the entries inside it, and where each file's bytes actually lie -- and the suite checks the invariants that hold for any image at all: an anchor at each end, every file inside the partition, nothing written over anything, no extent longer than its own length field, the streams beginning on a 64 KB cluster, and the tree matching the folder the image was made of file for file and byte for byte. The image for a recorder to go on editing (--iso-access overwritable) is checked the same way, and for being the size of a whole BD-RE while taking only what it holds.

It then prints the same report for one image per other writer it can find under $SMARTCUT_DISCS (~/Documents/claude/TMPGEnc unless set), so the places where our dialect differs from a recorder's or a burner's are in front of you rather than waiting on a disc that will not play. So the count at the top follows the images on the machine it is run on -- 50 here, where five other writers are represented, and 45 where there are none. Those columns are never failures: a recorder writes an overwritable partition where an image file is read-only, keeps three anchors to a burner's two, and leaves deleted file identifiers in its directories, and all of it is legal. The comparison earned its place immediately -- it is what found the mirror of the metadata partition being described by an entry written beside the original's, where one unreadable cluster would have taken both.

run_dvd_tests.sh builds two DVDs out of the same stream, muxed to MPEG program streams by ffmpeg's dvd muxer: an ordinary one, whose title set is written in two files the way the format made discs write it, and one whose title was multiplexed in two halves, so that its clock starts again in the middle. dvd_index.py fills in the navigation packs — ffmpeg leaves their presentation times at zero — and writes the .IFO tables around them.

run_bd_audio_tests.sh builds one clip per codec a disc carries — LPCM at 16 and at 24 bits, DTS, TrueHD, E-AC-3 — out of the same mpeg2.ts, and cuts each into a .ts, an .m2ts and an .mp4.

run_audio_codec_tests.sh asks for each of the four codecs the window offers — AAC, AC-3, DTS, LPCM — into each of four containers, and checks three things of each: the track is the codec that was asked for, every channel still carries the tone it went in with, and a transport stream's own programme map declares the codec that is actually in it. It also asks for two things that cannot be written — 5.1 DTS under its bitrate floor, and three channels of DTS, which is a count it has no arrangement for — and checks that the first is raised to what DTS is ordinarily carried at and the second is refused in a sentence that says why. Those are the answers the output settings screen greys out, told to it by writable_sound.

run_audio_format_tests.sh covers the other two properties of a sample: the rate it is taken at and the width it is written with. A resample has to reach the samples, the stream's declaration and, for AAC in a transport stream, the ADTS header on every frame; a rate the codec does not have must come back as the nearest one it does, with a message saying so. A width only means anything where samples are written down, so it is honoured for LPCM — where it decides the file's size outright — and declined with a message everywhere else.

run_audio_tests.sh and run_downmix_tests.sh build their own fixtures into the same directory: an impulse train, and a 5.1 track with a tone per channel. The codec and format suites reuse the second of those; nothing else wants either.

Every suite writes its output next to the fixtures, under $TMPDIR. The proxy suite needs several GB of that on real material — one proxy of half an hour of broadcast TS is 2.3 GB — which is more than a small /tmp tmpfs holds. If it fails with No space left on device, point TMPDIR at a directory on disk:

TMPDIR=~/tmp bash tests/run_proxy_tests.sh

Suites that read real material

run_audio_content_tests.sh, run_aac_tests.sh, run_preview_tests.sh, run_index_tests.sh, run_proxy_tests.sh, run_scene_tests.sh, run_cm_tests.sh, run_ts_layout_tests.sh and run_broadcast_tests.sh. (The index and proxy suites also run on synthetic material; the preview suite has nothing to run without real recordings.)

They look in ~/media by default, which SMARTCUT_MEDIA overrides. The audio comparison needs numpy, and SKIPs without it.

run_vc1_tests.sh reads real material too, and is named separately because what it needs is particular: a recording written in VC-1, which in practice means a Blu-ray pressed before about 2010. It is named by SMARTCUT_VC1 — a whole .m2ts or a piece of one — and without it the suite exits rather than pretending to pass. It also wants the encoder's own example binary, which an ordinary cargo build --release does not produce:

cd rust && cargo build --release --examples          # -> target/release/examples/vc1enc
SMARTCUT_VC1=~/media/disc.m2ts bash tests/run_vc1_tests.sh

Three of the four checks encode pictures out of that recording at a coarse, a middle and a fine quantizer, decode each with libavcodec and measure what comes back; the fourth takes a ten second cut whose ends fall between access points, so that a head and a tail both have to be written, and checks that the copied stretch between them is a verbatim run of the recording's own bytes. SMARTCUT_VC1_AT moves where the reference pictures are taken from (20 s in by default), for a disc that opens on something too flat to measure.

Swapping implementations

Environment variables let you swap implementations and check that the result is the same:

SMARTCUT_INDEX=container bash tests/run_rust_tests.sh    # take the index from the container
SMARTCUT_AUDIO=copy      bash tests/run_audio_tests.sh   # never touch the audio (smart is the default)
SMARTCUT_AUDIO=reencode  bash tests/run_audio_tests.sh   # sample-accurate audio
SMARTCUT_BYTE_SEEK=0     bash tests/run_preview_tests.sh # seek by timestamp again

The container-index run answers 11 of the 15: the four transport-stream checks skip themselves with the container has no seek table for this stream, which is the fact that makes the walk exist.

Environment variables

The seek index can be built from the CLI too. Pass the same path twice and the second run skips the walk:

smartcut rec.ts --seek-index /tmp/rec.scix --scenes
Variable Default
SMARTCUT_BYTE_SEEK on 0 or off goes back to aiming at a timestamp and reading forward from seek_margin seconds early

The proxy is off by default, and tunable the same way:

Variable Default
SMARTCUT_PROXY off 1 or on to build one. The preview, the strip and playback then read from it
SMARTCUT_PROXY_WIDTH 1280 Proxy width in square pixels. Higher looks better and takes longer to build. The cap is 1920x1080; for tall material, where the height hits the cap first, the width comes down accordingly
SMARTCUT_PROXY_QUALITY 22 Quality, in x264 CRF terms. Lower is better; 18–24 is the useful range
SMARTCUT_PROXY_ENCODER auto Comma-separated list of encoders to try (mpeg4, for instance)

Width and quality are part of the cache hash, so each setting builds its own proxy.

In the window, the first two of those are also rows in 環境設定, along with SMARTCUT_FFMPEG_LOG and SMARTCUT_CLEAN_JOINS. The environment is what each of them starts at, which is what a recording named on the command line is opened with; the frontend sends what it has stored as soon as it has read it, and that wins. See Using the GUI.

Validating against real material

tests/verify_real.py <src> <out> <ranges> checks frame count, alignment, bit-exact ratio, timeline, interlacing and A/V length difference in one go. For the results, see Validation.

Running in a virtual machine

WebKitGTK's compositor draws nothing on a machine without a GPU, and never updates after the first paint — which looks exactly like a freeze. The app defaults WEBKIT_DISABLE_COMPOSITING_MODE=1, so normally you do not have to think about it.

The same freeze arrives a second way, this time on a click into a text field. With GTK's XIM input-method module in the window, WebKitGTK stops painting the moment an <input> takes focus. The program keeps running underneath: the state behind the stale pixels goes on changing, and a 1px resize brings it all back at once. (Screenshots that went stale while driving the GUI with xdotool were this, not xdotool.) XIM is what GTK falls back to when GTK_IM_MODULE is unset, which is every desktop where an IME was never set up, so the app defaults GTK_IM_MODULE=gtk-im-context-simple. That module cannot compose Japanese, so on a machine with a working IME, set GTK_IM_MODULE to it (fcitx, ibus); where it is set explicitly, the app leaves it alone.