Skip to content

CR: 4689720 | ContextualClassifier relocates OCI container init process into urm.slice/focused.apps, orphaning docker-<CID>.scope and breaking docker exec #333

Closed
#334
Closed
CR: 4689720 | ContextualClassifier relocates OCI container init process into urm.slice/focused.apps, orphaning docker-<CID>.scope and breaking docker exec#333
#334
Assignees
kartnemavarunsinghal29
Labels
bugSomething isn't working

Description

@xinlon-z
Xinzheng Long (xinlon-z)
opened on Sep 28, 2026

Summary

On devices with urm.service running, an OCI container's init/entrypoint process (e.g. python3 as the container's PID 1) gets migrated into /urm.slice/focused.apps shortly after the container starts. This orphans the transient system.slice/docker-<CID>.scope unit systemd created for the container — it becomes empty and is garbage-collected by systemd — which breaks docker exec with a setns/cgroup-open failure.

Affected devices

  • Hamoa-iot-evk
  • Purwa-iot-evk

Environment

  • META Build: Hamoa_IOT.UBUN.1.0.r1-00102-STD.INT.DESKTOP-1
  • OS: Ubuntu 26.04, kernel 7.0.0-1014-qcom, aarch64
  • Docker 29.1.3, containerd 2.2.2, runc 1.4.0, systemd 259
  • liburm1 0.4.6-1
  • docker info: Cgroup Driver: systemd, Cgroup Version: 2

Steps to reproduce

sudo systemctl start urm.service   # ensure URM is running
docker run -dit --name repro python:3.12-slim
PID=$(docker inspect -f '{{.State.Pid}}' repro)
cat /proc/$PID/cgroup
docker exec repro /bin/true

A/B confirmation:

  • systemctl stop urm.service → new containers stay in system.slice/docker-<CID>.scope, docker exec works (rc=0).
  • systemctl start urm.service → 100% reproducible on every new container, on both Hamoa-iot-evk and Purwa-iot-evk.

Detailed logs / evidence

1. docker exec failure

$ docker exec -it python-slim-test bash
OCI runtime exec failed: exec failed: unable to start container process: error starting setns process: can't open cgroup: openat2 /sys/fs/cgroup/system.slice/docker-<CID>.scope: no such file or directory: unknown

2. Container process has been relocated out of its docker scope

$ PID=21442
$ cat /proc/$PID/cgroup
0::/urm.slice/focused.apps

Reproduced live on device with a second container:

$ cat /proc/22756/cgroup
0::/urm.slice/focused.apps
$ cat /proc/22756/comm
python3

3. The docker-managed transient scope has been garbage-collected

$ systemctl status docker-<CID>.scope
Loaded: not-found
Active: inactive (dead)

On a still-intact container (created before URM picked it up), for comparison:

$ systemctl status docker-75070824a9fe1d07a2b04bce0d90de25e582187bbccb172ea5c9ac170c5ac115.scope
● docker-75070824a9fe1d07a2b04bce0d90de25e582187bbccb172ea5c9ac170c5ac115.scope - libcontainer container 75070824a9fe1d07a2b04bce0d90de25e582187bbccb172ea5c9ac170c5ac115
     Loaded: loaded (/run/systemd/transient/docker-75070824a9fe1d07a2b04bce0d90de25e582187bbccb172ea5c9ac170c5ac115.scope; transient)
  Transient: yes
     Active: active (running) since Mon 2026-09-28 07:09:34 UTC; 53min ago
     CGroup: /system.slice/docker-75070824a9fe1d07a2b04bce0d90de25e582187bbccb172ea5c9ac170c5ac115.scope
             └─22313 python3

4. focused.apps cgroup actually contains the container's PID

$ cat /sys/fs/cgroup/urm.slice/focused.apps/cgroup.procs
16358
17074
22756          # <- the container's python3, confirmed via /proc/22756/comm above

5. Process tree (confirms it's a standard OCI container init process, not a stray host process)

$ pstree -sp 22756
systemd(1)---containerd-shim(22732)---python3(22756)

$ ps -eo pid,ppid,comm | grep -E 'containerd|shim|python'
  19216       1 containerd
  22289       1 containerd-shim
  22313   22289 python3
  22732       1 containerd-shim
  22756   22732 python3

6. URM service log noise around the relocation/eviction (journalctl / serial console)

urm[22557]: [URM_SYSCALL_FAILURE] [ERROR] removeProcessFromCGroup: Call to write, Failed with Error: No such process
urm[22557]: [RESTUNE_SIGNAL_REGISTRY] [ERROR] getSignalConfigByIdAndType: Signal with: ID [0x00020001] and Type [0x00000000] not found in the registry. Dropping Request.

These two errors repeat continuously (multiple times per second) for the lifetime of the service.

7. urm.service state at time of repro

$ systemctl status urm.service
● urm.service - URM Service
     Loaded: loaded (/usr/lib/systemd/system/urm.service; enabled; preset: enabled)
     Active: active (running) since Mon 2026-09-28 07:14:42 UTC; 47min ago
    Process: 22556 ExecStartPre=/usr/libexec/urm/initscripts/post_boot/post_boot.sh (code=exited, status=203/EXEC)
   Main PID: 22557 (urm)
     CGroup: /urm.slice/urm.service
             └─22557 /usr/sbin/urm

Note: ExecStartPre for post_boot.sh fails with 203/EXEC (script file not present on the rootfs — /usr/libexec/urm/initscripts/post_boot/post_boot.sh: No such file or directory), though this does not prevent urm.service from reaching active (running).

8. Relevant on-device classifier configuration at time of repro

$ cat /etc/urm/common/InitConfig.yaml
InitConfigs:
  - CgroupsInfo:
    - Name: "init.scope"
      Create: false
      ID: 1
    - Name: "system.slice"
      Create: false
      ID: 2
    - Name: "user.slice"
      Create: false
      ID: 3
    - Name: "urm.slice/focused.apps"
      Create: true
      ID: 4

$ cat /etc/urm/classifier/classifier-blocklist.txt
ls,cp,sh,mv,rm,id,mkdir,dirname,touch,find,irqbalance,cpuUsage.sh,which,sudo,cpu.sh,ln,uname,gst-plugin-scan,urmCli,UrmComponentTests,UrmIntegrationTests
grep,sed,awk,cat,tail,head,less,more,sort,uniq,wc,cut,tr,tee
ps,top,sleep,journalctl,xargs,logrotate,systemctl

$ grep -A2 'apply_mode' /etc/urm/common/PropertiesConfig.yaml
  - Name: urm.classifier.apply_mode
    # Possible values: ACCEPT_AND_PERSIST, ACCEPT_AND_REPLACE
    Value: "ACCEPT_AND_PERSIST"

Impact

Any containerized workload (Docker/containerd/Podman, or any tool relying on a systemd transient scope + its own cgroup path) is at risk of having its process silently relocated out of its runtime-managed cgroup by URM, breaking exec-style operations and any cgroup-based resource accounting the container runtime depends on — not just docker exec specifically. Reproduced consistently on both Hamoa-iot-evk and Purwa-iot-evk.

Activity

  1. Kartik Nema (kartnema)
    added
    bugSomething isn't working
    on Sep 28, 2026
  2. Kartik Nema (kartnema)
    assigned
    Kartik Nema (kartnema)
    and
    Varun Singhal (varunsinghal29)
    on Sep 28, 2026
  3. Varun Singhal (varunsinghal29)
    changed the title [-]ContextualClassifier relocates OCI container init process into urm.slice/focused.apps, orphaning docker-<CID>.scope and breaking docker exec[/-] [+]CR: 4689720 | ContextualClassifier relocates OCI container init process into urm.slice/focused.apps, orphaning docker-<CID>.scope and breaking docker exec[/+] on Sep 28, 2026

  4. varunsinghal29 commented on Sep 29, 2026

    @varunsinghal29
    Varun Singhal (varunsinghal29)
    on Sep 29, 2026
    Member

    Hi Xinzheng Long (@xinlon-z)
    we have solved this issue in this PR : #334
    please have a look.
    Now URM, not moving the Docker container processes to urm.slice/focused.app cgroups

    ### attaching the logs below [with verification]

    `root@qcom:/home/qcom# sudo docker ps -q | xargs -I {} sudo docker inspect --format 'ID: {{.Id | printf "%.12s"}}  Name: {{.Name}}  PID: {{.State.Pid}}  Status: {{.State.Status}}' {}
ID: cfe6334113e0  Name: /repro  PID: 4631  Status: running
root@qcom:/home/qcom#
root@qcom:/home/qcom# systemctl status urm
● urm.service - URM Service
     Loaded: loaded (/usr/lib/systemd/system/urm.service; enabled; preset: enabled)
     Active: active (running) since Tue 2026-09-29 09:59:02 UTC; 1h 25min ago
 Invocation: f8abbe26a21d42cfa4bc064f87bf7eef
    Process: 4347 ExecStartPre=/usr/libexec/urm/initscripts/post_boot/post_boot.sh (code=exited, status=203/EXEC)
   Main PID: 4348 (urm)
      Tasks: 17 (limit: 37443)
     Memory: 17.2M (peak: 18.6M)
        CPU: 784ms
     CGroup: /urm.slice/urm.service
             └─4348 /usr/sbin/urm

Sep 29 09:59:02 qcom systemd[1]: Starting urm.service - URM Service...
Sep 29 09:59:02 qcom (post_boot.sh)[4347]: urm.service: Unable to locate executable '/usr/libexec/urm/initscripts/post_boot/post_boot.sh': No such file or directory
Sep 29 09:59:02 qcom systemd[1]: Started urm.service - URM Service.
Sep 29 09:59:02 qcom urm[4348]: [URM_SERVER_INIT] [INFO] main: Starting URM, PID = [4348]
Sep 29 09:59:02 qcom urm[4348]: [URM_SERVER_INIT] [INFO] parseUtil: Proceeding with [prop-common] Config Parsing, Path: [/etc/urm/common/PropertiesConfig.yaml]
Sep 29 09:59:02 qcom urm[4348]: [URM_SERVER_INIT] [INFO] parseUtil: [prop-common] Configs successfully parsed
Sep 29 09:59:02 qcom urm[4348]: [URM_SERVER_INIT] [INFO] fetchMetaConfigs: Detected Target: []
Sep 29 09:59:02 qcom urm[4348]: [URM_SERVER_INIT] [INFO] fetchMetaConfigs: Detected Target: []
root@qcom:/home/qcom#
root@qcom:/home/qcom#
root@qcom:/home/qcom#
root@qcom:/home/qcom# ls /sys/fs/cgroup/system.slice/ | grep docker
**docker-cfe6334113e0c1f17e04b15d74c258204b5261de073144728411cab4e82e1c85.scope
docker.service
docker.socket**
root@qcom:/home/qcom#
root@qcom:/home/qcom# sudo docker exec repro /bin/true
root@qcom:/home/qcom#

    No issue is coming.

  5. Kartik Nema (kartnema)
    linked a pull request that will close this issueclassifier: skip Docker container processes to fix docker exec failure #334on Sep 30, 2026
  6. Varun Singhal (varunsinghal29)
    closed this as completedin #334on Sep 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions