sysupgrade: the watchdog keeper never runs on Xiongmai (xm_wdt is /dev/wdt and cannot be fed by write) #2611

Summary

sysupgrade's RAM-phase watchdog keeper (watchdog_keep, general/overlay/usr/sbin/sysupgrade) protects the flash from a watchdog reset after majestic dies mid-write. It does nothing on xm510/xm530/xm550. Two things stop it, and fixing the first alone doesn't help:

1. Wrong device. WDOG=${WDOG:-/dev/watchdog}, but xm_wdt.ko registers a misc device named wdt, so XM has /dev/wdt and no /dev/watchdog. watchdog_owner() returns 1 at [ -c "$WDOG" ], so wdog_userspace_owned is never set and no keeper is started. The pre-flash report prints Watchdog: none, even though majestic holds /dev/wdt and feeds it.
2. No write() feed. Even pointed at /dev/wdt, the keeper's printf 1 >&9 feeds nothing. xm_wdt.ko's file_operations has only open, release and unlocked_ioctl, and no write (the relocations against .data in both the xm510 and xm530 builds of the module show this). Feeding is only possible through its private ioctl, _IO('W', 2) = 0x5702, which no busybox applet can issue. busybox watchdog writes to the device and uses the standard WDIOC_* numbers, so it doesn't work here either.

Why it matters

When majestic arms the watchdog it asks for 300 s:

Watchdog armed: asked 300 s, assuming 300 s, fed every 10.000s, magic close NOT supported by this driver

wdt_release is a no-op (the driver has no magic close), so if majestic dies during the flash (the SIGBUS-after-erase case the keeper exists for), the SP805 counts down and resets the board about 300 s after majestic's last feed. The keeper's own comment describes the result: a camera that comes back with a part-written rootfs from an upgrade whose log looked fine. On XM nothing guards against that today.

I have not measured how long a full sysupgrade takes on an XM board, so I can't say how often the 300 s window is hit in practice. The mechanism above was read from the module and from majestic's log, not observed as a failed upgrade.

What the driver does (from disassembly of general/package/xiongmai-osdrv-xm530/files/kmod/xm_wdt.ko)

The hardware is an ARM PrimeCell SP805:

Options

• a. Keeper stands the dog down on XM. When the owner is gone and the device is /dev/wdt, run rmmod xm_wdt (the module's exit stops the SP805) instead of trying to pet it.
  • Pros: shell only, no new binary.
  • Cons: it gives up the property the keeper deliberately keeps on other SoCs, where an armed dog recovers a camera whose final reboot can't exec (sysupgrade --no_reboot (-x) corrupts the running rootfs and locks out SSH after an on-device flash #2231). It also needs rmmod and the module's path to still work from the ramfs.
• b. Small ioctl feeder. A tiny C helper, packaged with source in the xm osdrv packages (or a sysupgrade helper), that feeds /dev/wdt with 0x5702. The keeper uses it when WDOG=/dev/wdt. This keeps the "armed until reboot" behaviour. It has to live in the RAM root like the rest of the flash phase.
• c. Replace xm_wdt with the upstream sp805_wdt driver. That gives a real /dev/watchdog with write() feeding, WDIOC_* and magic close, and the hardware stays idle until opened. That would also have made XM530 reboots after ~ 1 minute  #743 impossible. It needs the AMBA device registered for the SoC, which is kernel work for OpenIPC/linux, plus a majestic change to open /dev/watchdog on XM.

Whichever is chosen, WDOG needs to resolve per platform (or probe /dev/watchdog, then /dev/wdt). That way the pre-flash Watchdog: line reports the truth on XM instead of none.

Related: #743 (xm_wdt starts counting at module load; fixed in majestic by stopping it when the watchdog is disabled).