Everything needed to get tests/acceptance/ pytest running, and to fix it
when it breaks. For how the framework is built see
acceptance-design.md; for MTL itself see
build.md and run.md.
.local_install rule — read this firstPytest resolves every binary it launches under .local_install/, a tree
separate from the build/ + /usr/local install that gtest/KahawaiTest
uses. This is hardcoded in tests/acceptance/mtl_engine/const.py
(PREFIX = ".local_install") and is the single most common reason a host
“with MTL built” still cannot run a test.
| Pytest launches | Resolved path |
|---|---|
| RxTxApp | .local_install/mtl/bin/RxTxApp |
| MtlManager | .local_install/mtl/bin/MtlManager |
| FFmpeg / ffprobe | .local_install/ffmpeg/bin/ |
| GStreamer plugin | .local_install/gstreamer/gstreamer-1.0/ |
A working build/manager/MtlManager or tests/tools/RxTxApp/build/RxTxApp
does not satisfy pytest. Symptom when the tree is missing: the
mtl_manager fixture errors with Failed to start MtlManager on host while
gtest runs fine.
root@ even on a single host, because NIC binding and hugepage setup
need it. There is no unprivileged mode..local_install, not taken from the system packages..github/scripts/acceptance_setup.sh is the single entry point. It composes
acceptance_setup_base.sh (apt, ICE driver, DPDK, MTL + plugins into
.local_install, hugepages, CPU governor) and setup_acceptance.sh (NFS
media, localhost root SSH, venv, generated configs). No agent or MCP client
is required.
./.github/scripts/acceptance_setup.sh status # read-only probe, changes nothing
./.github/scripts/acceptance_setup.sh setup # interactive: asks PF, NFS source, EBU
Non-interactive, e.g. for CI:
./.github/scripts/acceptance_setup.sh setup --auto \
--pf-bdf=0000:c9:00.0 \
--nfs-source=10.0.0.5:/mnt/NFS/mtl_assets/media
It prints a ready-to-run pytest command when it finishes. -h lists all
flags (--base-only, --pytest-only, --check-only, --ffmpeg/
--no-ffmpeg, --gstreamer/--no-gstreamer, --ebu-ip/--ebu-user for
compliance capture, --nr-hugepages, --build-mode). Both phases are
idempotent — re-running on a prepared host is safe and takes seconds.
Cold setup takes roughly 7–10 minutes. Do not interrupt and retry during the first run.
Only needed to debug or customise one stage; the script above automates all of it.
cd tests/acceptance
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt
pip install -r common/integrity/requirements.txt
configs/gen_config.py generates both files; hand-edit only to adjust.
configs/topology_config.yaml — hosts, connections, and NICs:
hosts:
- name: host
instantiate: true
role: sut # "sut" or "client" (dual-host only)
network_interfaces:
- pci_device: 8086:1592 # vendor:device, NOT a BDF
interface_index: 0
connections:
- ip_address: 127.0.0.1
connection_type: SSHConnection
connection_options:
port: 22
username: root # must be root
password: '' # required key even when using a key file
key_path: /home/<you>/.ssh/id_rsa
pci_device takes the vendor:device ID (lspci -nn | grep Ethernet), not
the 0000:18:00.0 bus address. Supplying a BDF raises
Incorrect format ... PCIDevice.
configs/test_config.yaml — paths and per-run behaviour:
build: /home/<you>/Media-Transport-Library/
mtl_path: /home/<you>/Media-Transport-Library/
media_path: /mnt/media
interface_type: VF # VF | PF | VFxPF | 2VFxPF | 3VFxPF
capture_cfg:
enable: false # true enables EBU LIST compliance capture
pcap_dir: /mnt/ramdisk/pcap
capture_time: 5
sniff_pci_device: 8086:1592 # vendor:device of the capture NIC
phc_sync: true # discipline the local PHC to system TAI
ebu_server: # omit to leave compliance disabled
host: 10.0.0.9
user: <user>
password: <password>
ramdisk:
media: {mountpoint: /mnt/ramdisk/media, size_gib: 32}
pcap: {mountpoint: /mnt/ramdisk/pcap, size_gib: 768}
Capture needs a second NIC port in a different IOMMU group from the one
carrying traffic. Without ebu_server, tests still run — the compliance
verdict is simply skipped.
Production hosts mount an NFS share at media_path. Tests skip rather
than fail when an asset is missing. To generate a local set instead:
cd tests/acceptance/common
./gen_frames.sh
This needs an FFmpeg build with text filters; No such filter: 'drawtext'
means the system FFmpeg is too minimal (sudo apt install ffmpeg).
The ST2110-40 GStreamer suites stage inputs and outputs on
ramdisk.media.mountpoint — mount and size that ramdisk or they spill
into /tmp.
Run from tests/acceptance/, as root, using the venv interpreter:
cd tests/acceptance
sudo -E ./venv/bin/python3 -m pytest \
--topology_config=configs/topology_config.yaml \
--test_config=configs/test_config.yaml \
-m smoke -v
sudo python3 uses the system interpreter and fails with
No module named pytest. The config arguments are registered by the local
conftest.py, so running from another directory fails with
unrecognized arguments.
$PY -m smoke -v # by marker
$PY tests/single/st20p -v # by folder
$PY tests/single/st40p -k multicast -v # by substring
$PY "tests/single/st20p/test_fps.py::test_st20p_fps[|fps = p60|-Penguin_1080p-|application = rxtxapp|]"
$PY <selector> --collect-only -q # dry run
Quote selectors containing [ ] or |.
Suite markers:
| Marker | Use |
|---|---|
smoke |
Smallest set that proves the host works |
nightly |
Bulk single-host coverage |
performance |
Dual-host capacity sweeps, long and hardware-bound |
base_performance |
Added at collection to a test ID that holds 1080p and 59fps |
dual |
Requires two hosts |
ptp |
Uses MTL’s internal PTP (phc2sys suppressed) |
Neither performance marker selects a tests/single/ test. Select the
tests/single/performance/ modules by path.
Descriptive markers: verified (see below),
refactored (uses the Application-based harness),
tx_side/rx_side/tx_and_rx (which side a test validates, and therefore
whether a compliance verdict is meaningful), and allow_wide_compliance
(accept ST 2110-21 wide instead of failing).
tests/ contains single/ and dual/ only.
| Option | Effect |
|---|---|
--time N |
Seconds each test runs (default 15) |
--keep all\|failed\|none |
Retain result media files |
--dmesg clear\|keep |
Whether to clear the kernel ring before each test |
--media, --build, --nic, --dma |
Override the config values |
--num_sessions N, --sch_quota N |
Performance sweeps only |
--sut_app_args='ARGS' |
Performance sweeps only: extra RxTxApp arguments for the measured app |
--shard K/N |
Run only shard K of N of the selected tests; the N shards together run each test once |
Logs land in tests/acceptance/logs/<UTC-timestamp>/ (root-owned) with a
logs/latest symlink — per-test logs plus MtlManager and application output
and the rendered JSON configs.
sudo grep -E "EAL|hugepage|VF|RxTxApp|Traceback|err:" logs/latest/*.log | head -40
An HTML run report:
$PY -m smoke --template=html/index.html --report=report.html
A performance report across timestamped log directories:
./venv/bin/python3 common/generate_report.py logs/
Start with ./.github/scripts/acceptance_setup.sh status — it checks kernel,
hugepages, governor, ICE driver, NIC PFs, .local_install build state, NFS,
venv, and configs in one read-only pass, and usually names the broken stage
directly. Re-run setup --base-only or setup --pytest-only to repair it.
| Symptom | Cause and fix |
|---|---|
No module named pytest |
Used sudo python3. Use sudo -E ./venv/bin/python3. |
unrecognized arguments: --topology_config |
Not in tests/acceptance/. |
Failed to start MtlManager on host |
.local_install tree missing — re-run setup. A system-wide build/ install does not count. |
RxTxApp: command not found |
Same cause: build into .local_install, not tests/tools/RxTxApp/build/. |
error while loading shared libraries: librte_*.so |
Stale linker cache. sudo ldconfig, then check ldd .local_install/mtl/bin/RxTxApp. |
Media file not present ... (ERROR) |
NFS unmounted or empty. Re-run setup with the NFS source. |
Incorrect format ... PCIDevice / sniff_pci_device ... not found |
A BDF was used where vendor:device is required. |
SSHConnection.__init__() missing ... 'password' |
Add password: '' to the topology connection. |
Permission denied (publickey) for root@127.0.0.1 |
Public key not in /root/.ssh/authorized_keys. |
ValueError: q must be exactly 160, 224, or 256 bits long |
DSA key. ssh-keygen -t rsa -b 2048 -f ~/.ssh/id_rsa && ssh-copy-id root@localhost. |
netsniff-ng: command not found |
sudo apt install -y netsniff-ng. |
EBU server configuration not found, test still PASSED |
Data path passed, compliance skipped. Configure ebu_server + capture_cfg. |
Test hangs past --time + ~30 s |
Stale process: sudo pkill -9 RxTxApp MtlManager ffmpeg gst-launch-1.0. |
| EAL hugepage or VF binding errors | Hugepages exhausted, or VFs not bound to vfio-pci. Re-run setup. |
RxTxApp segfault in iavf_tm_node_add |
In-tree ice loaded instead of the out-of-tree build. Re-run setup. |
A segfault anywhere other than iavf_tm_node_add is a real MTL/DPDK bug —
capture a backtrace (coredumpctl gdb RxTxApp) and report it rather than
working around it.
The exhaustive symptom table, kept in sync with the engine source, is in mtl-acceptance-tests.instructions.md.
@pytest.mark.verified is a claim about evidence, not just a passing run. A
test may carry it only when all three hold:
rss_enabled) in the logs agree with the test
inputs.