** Tags added: kernel-daily-bug -- You received this bug notification because you are subscribed to linux in Ubuntu. Matching subscriptions: Bgg, Bmail, Nb https://bugs.launchpad.net/bugs/2165791 Title: linux 7.0.0-30-generic: list_del corruption + fatal Oops in ep_remove_safe/remove_wait_queue, triggered by the snapd 2.76 -> 2.76.3 upgrade Status in linux package in Ubuntu: New Bug description: Ubuntu 26.04, linux-image-7.0.0-30-generic 7.0.0-30.30, on a LENOVO 82VG (IdeaPad 1, BIOS KSCN31WW 05/09/2024), amd64. The machine hard-locked in the middle of an unattended-upgrades run. The crash reproduced a second time during manual recovery. Both crashes happened in snapd, in the epoll teardown path, and both were tied to the snapd 2.76 -> 2.76.3 package transition -- not to steady-state operation. TIMELINE Aug 20 14:25 kernel 7.0.0-30 installed. Ran 8 days, zero panics. Aug 28 05:45 unattended-upgrades starts a ~120 package transaction. Aug 28 05:47 term.log ends at "Setting up snapd (2.76.3+ubuntu26.04)". Machine dies here. history.log has no End-Date for this transaction. dpkg left 51 packages in iU and snapd in iF. Aug 28 11:03 Manual recovery: `dpkg --configure -a` reconfigures snapd. -> list_del corruption WARNING, kernel tainted G W. Aug 28 11:14 Reboot requested. snapd receives SIGTERM. -> fatal Oops, kdump captured a 252 MB vmcore. Aug 28 11:20+ Three subsequent clean shutdowns, plus a multi-hour memtest86+ run. No further crashes. /proc/sys/kernel/tainted back to 0. RAM was ruled out: memtest86+ passed clean, and the same kernel had already run 8 days without incident before snapd 2.76.3 arrived. The failure is deterministic and tied to the package transition, not random. FIRST EVENT (warning, kernel still alive) list_del corruption. next->prev should be ffff8f518099a328, but was 0000000000000000. (next=ffff8f4fa4969b50) WARNING: lib/list_debug.c:65 at __list_del_entry_valid_or_report+0xc0/0x10b, CPU#4: snapd/284305 Not tainted 7.0.0-30-generic #30-Ubuntu PREEMPT(lazy) Call Trace: remove_wait_queue.cold+0x9/0x12 ep_remove_safe+0x3c/0xe0 do_epoll_ctl+0x599/0x840 __x64_sys_epoll_ctl+0x6c/0xb0 x64_sys_call+0x1be6/0x2390 do_syscall_64+0x105/0x5a0 A second, symmetric warning followed immediately (lib/list_debug.c:62, prev->next should be ffff8f4f2004b068, but was 0000000000000000), i.e. both directions of the list entry had already been zeroed. SECOND EVENT (fatal, ~11 minutes later, on snapd termination) BUG: unable to handle page fault for address: ffffffff3226cb80 Oops: Oops: 0002 [#1] SMP NOPTI CPU: 3 UID: 0 PID: 284675 Comm: snapd Kdump: loaded Tainted: G W 7.0.0-30-generic #30-Ubuntu PREEMPT(lazy) Hardware name: LENOVO 82VG/LNVNB161216, BIOS KSCN31WW 05/09/2024 RIP: 0010:native_queued_spin_lock_slowpath+0x2f5/0x370 Call Trace: __raw_spin_lock_irqsave+0x57/0x80 _raw_spin_lock_irqsave+0xe/0x20 remove_wait_queue+0x1b/0x80 ep_remove_safe+0x3c/0xe0 do_epoll_ctl+0x599/0x840 __x64_sys_epoll_ctl+0x6c/0xb0 x64_sys_call+0x1be6/0x2390 do_syscall_64+0x105/0x5a0 ? __memcg_slab_free_hook+0x113/0x180 ? kmem_cache_free+0x266/0x3f0 ? __fput+0x1a2/0x2d0 ? fput_close_sync+0x40/0xc0 ? __x64_sys_close+0x3e/0x90 ANALYSIS The fatal trace shows epoll_ctl(EPOLL_CTL_DEL) racing a concurrent close() on the same descriptor: the kmem_cache_free / __fput / fput_close_sync / __x64_sys_close frames sit alongside the ep_remove_safe path. This matches the known eventpoll use-after-free shape, where ep_remove() drops file->f_ep under the lock but keeps using the file object, while a concurrent __fput() frees the struct eventpoll underneath it. The subsequent list operation then writes into freed memory -- which is exactly what the two list_debug warnings reported (both list pointers zeroed), and what the page fault at ffffffff3226cb80 in the spinlock slowpath is the consequence of. The zeroed pointers in the warning, and the fact that the machine survived 11 more minutes in a tainted state before dying on the next snapd termination, are consistent with memory that was freed and then reused. IMPACT Total loss of the machine mid-upgrade, with dpkg left in a broken state (51 packages unconfigured). Recovering required a second boot and a manual `dpkg --configure -a`, which itself re-triggered the bug. ATTACHED / AVAILABLE /var/crash/linux-image-7.0.0-30-generic-202608281115.crash (48 KB apport report, contains the full VmCoreDmesg, 1675 lines) /var/crash/202608281115/dump.202608281115 (252 MB vmcore, preserved, available on request) /var/crash/202608281115/dmesg.202608281115 (165 KB) WORKAROUND None applied to the kernel itself. 7.0.0-29-generic is still installed and available as a fallback from the GRUB advanced menu. unattended-upgrades was disabled locally right after the crash, then re-enabled the same day, once it was established that snapd cannot in fact be upgraded unattended on this system: the installed snapd (2.76.3+ubuntu26.04) comes from resolute-updates, which is not among Unattended-Upgrade::Allowed-Origins, and resolute-security only carries the older 2.76+ubuntu26.04.3. ADDITIONAL EVIDENCE (same day, after recovery) Steady-state snapd operation does NOT reproduce this bug. On the same kernel, after recovery, all of the following completed with no warning and no Oops: - 8x `snap remove --purge <snap>` - 23x `snap remove <snap> --revision=<rev>` - 2x `snap set system <option>` (daemon reconfiguration) - 1x `snap forget <snapshot set>` - 1 full clean reboot, i.e. snapd receiving SIGTERM at shutdown -- which is precisely the operation that produced the fatal Oops on Aug 28 11:14. /proc/sys/kernel/tainted stayed at 0 throughout, and the current boot logs contain no list_del, BUG: or Oops entries. This narrows the trigger considerably: the fault appears to require the package transition itself -- dpkg reconfiguring and restarting snapd across the 2.76 -> 2.76.3 version change -- rather than epoll teardown during ordinary snapd shutdown, which by itself is not sufficient to reproduce it. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2165791/+subscriptions
Комментариев нет:
Отправить комментарий