isolate.sh [--] <command...> runs one command at nice −20 inside a cgroup v2
cpuset partition that exists only while the command runs. The NoCtx runner
(tests/integration_tests/noctx/run.sh) uses it for the cases in NOCTX_ISOLATE_CASES,
and the acceptance rx_timing tests for RxTxApp (cpu_isolation="require").
Background on static and dynamic CPU isolation: doc/isolation.md.
| Variable | Set by | Values |
|---|---|---|
MTL_ISOLATE |
caller | off (default) runs the command as is; try; require |
MTL_ISOLATE_PORTS |
caller | Comma-separated PCI BDFs, the MTL primary port (port P) first |
MTL_CPU_ISOLATION |
wrapper, for the command | exclusive or none |
If the partition is unavailable, try prints one WARNING with the reason and
runs the command unconfined; require prints isolate.sh: ERROR ... (<why>)
and exits without running it: 3 when another wrapper holds the lock, else 1.
Usage errors exit 2. Otherwise the command’s exit status is the
wrapper’s. CI (.github/scripts/gtest.sh) runs NoCtx with MTL_ISOLATE=require.
sudo -n).cpuset controller (/sys/fs/cgroup/cgroup.controllers). If
it is not enabled for the root’s children (cgroup.subtree_control),
isolate.sh enables it, which restricts nothing; systemd may turn it off
again, and the next wrapper re-enables it.isolated value of cpuset.cpus.partition, Linux 6.1 or later.
cpuset.cpus.exclusive (Linux 6.7) is written when present. Validated on 6.8.cpuset.cpus,
such as k3s’s kubepods.slice (AllowedCPUs=0-255): the kernel then
refuses the partition. isolate.sh never changes other cgroups to make room./sys/fs/cgroup/mtl-isolate-<pid> with
cpuset.cpus = the chosen CPUs (and cpuset.cpus.exclusive),
cpuset.cpus.partition = isolated, and reads the state back. Those CPUs
leave every other cgroup and load balancing, so no foreign task (containers,
pods) runs there. The cgroup gets cgroup.pressure=0, which stops its PSI
averaging.cpuset.mems is the ports’ nodes. MTL binds each
scheduler to its port’s node, so every port node needs CPUs. Setup fails if a
port’s node cannot be resolved or has too few cores.mtl_init()
binds the main thread to port P’s node (numa_bind). Setup fails unless that
CPU is on port P’s node, i.e. port P’s node must hold the partition’s
lowest-numbered CPU. Do not pass --lcores: EAL then pins its main lcore to CPU 0,
outside the cgroup./proc/irq/*/smp_affinity_list) are moved off the partition where the
kernel allows it, /proc/irq/default_smp_affinity excludes it (VF MSI-X
vectors are only allocated when the ports open), vm.stat_interval is
raised to 120 s, and kernel.numa_balancing is set to 0 where the kernel
has it: its scan runs as task work on the busiest threads, the MTL
schedulers, and stalls them for up to ~400 µs. The originals are written to
/run/mtl-isolate-<pid>.restore first and restored on exit.On exit the wrapper terminates what is left in its cgroup, removes it and
restores the saved settings. If the wrapper is SIGKILLed, a watcher in its own
session does this, so a kill of the wrapper’s session or process group does not
reach it, and it ignores SIGPIPE in case its stderr pipe is already gone. If
both die, the next wrapper or isolate.sh --sweep (root) does; it exits 1 if
a partition is left, 3 if the lock is held. .github/scripts/gtest.sh sweeps
before and after the NoCtx run, the acceptance suite at session start, and
task cleanup at the start of each CI test job. /run is
tmpfs, so a reboot also resets the settings. Wrappers and sweeps serialize on
flock /run/mtl-isolate.lock (a wrapper waits 10 s, then gives up), and remove
the stale mtl-isolate-* cgroups of dead wrappers.
nohz_full=), RCU
callbacks and expedited grace periods (rcu_nocbs=), per-CPU work queued on
the CPU that triggered it, and managed or per-CPU IRQs.warm_pkts packets, ~120 µs at 2160p59), or when there is no pad plan
(unmeasured pad interval, unknown burst bucket) or a stall inside a pad burst
(over 2 µs) and WAIT_TARGET holds it until its target.