Can confirm this happens on my machine with the 147 kernel also. I let Claude have a look at it, and this are it's findings: Also reproducible on 6.8.0-147.147 (noble) -- the bad hunk is still present, so this is not fixed by the -147 update. System: Dell Latitude 5530, BIOS 1.29.1, Ubuntu 24.04. VPN: NetworkManager-l2tp (kl2tpd / go-l2tp) + strongSwan, i.e. L2TP over IPsec. Last known good: 6.8.0-142.142. I have not tested -146 myself; I went straight from -142 to -147. Reproducer: activate an L2TP/IPsec connection. The machine hard-freezes within 30-120 seconds of ppp0 coming up (no panic written to disk, no shutdown sequence in the journal, power cycle required). Happened on 2/2 attempts on -147; reverting to -142 makes it go away completely. WHAT THE JOURNAL SHOWS The first warning fires in the same second ppp0 is created, from the L2TP UDP RX softirq path: ------------[ cut here ]------------ refcount_t: addition on 0; use-after-free. WARNING: CPU: 0 PID: 2872 at lib/refcount.c:25 refcount_warn_saturate+0x12e/0x150 CPU: 0 PID: 2872 Comm: kworker/0:5 Not tainted 6.8.0-147-generic #147-Ubuntu Hardware name: Dell Inc. Latitude 5530/0RN5CP, BIOS 1.29.1 04/10/2025 Workqueue: events xfrm_trans_reinject RIP: 0010:refcount_warn_saturate+0x12e/0x150 Call Trace: <TASK> l2tp_tunnel_get_session+0xc8/0xd0 [l2tp_core] l2tp_udp_recv_core+0xb1/0x350 [l2tp_core] ? __pfx_l2tp_udp_encap_recv+0x10/0x10 [l2tp_core] l2tp_udp_encap_recv+0x2c/0x54 [l2tp_core] udp_queue_rcv_one_skb+0x278/0x550 udp_queue_rcv_skb+0x4f/0x80 udp_unicast_rcv_skb+0x7a/0xa0 __udp4_lib_rcv+0x627/0x6f0 udp_rcv+0x25/0x40 ip_protocol_deliver_rcu+0xd8/0x210 ip_local_deliver_finish+0x77/0xa0 ip_local_deliver+0x6e/0x120 ? __pfx_ip_local_deliver_finish+0x10/0x10 xfrm4_rcv_encap_finish2+0x3c/0x60 xfrm_trans_reinject+0xe2/0x170 process_one_work+0x181/0x3a0 worker_thread+0x18b/0x330 kthread+0xef/0x120 ret_from_fork+0x44/0x70 ret_from_fork_asm+0x1b/0x30 </TASK> ---[ end trace 0000000000000000 ]--- Immediately followed by "refcount_t: underflow; use-after-free" in l2tp_session_dec_refcount+0xbb/0xd0 [l2tp_core] and "refcount_t: saturated; leaking memory" in l2tp_tunnel_get_session+0xbc/0xd0 [l2tp_core], all from the same RX path. Two seconds later the TX side starts tripping over the same freed object, and from then on it warns on *every* packet: WARNING: CPU: 3 PID: 1094 at net/l2tp/l2tp_ppp.c:166 pppol2tp_xmit+0x1e1/0x220 [l2tp_ppp] CPU: 3 PID: 1094 Comm: systemd-resolve Tainted: G W 6.8.0-147-generic #147-Ubuntu Call Trace: <TASK> ppp_push+0xaf/0x110 ppp_send_frame+0x1e8/0x530 __ppp_xmit_process+0x88/0x100 ppp_xmit_process+0x3d/0xb0 ppp_start_xmit+0x159/0x200 dev_hard_start_xmit+0x62/0x1e0 __dev_queue_xmit+0x709/0x910 neigh_direct_output+0x13/0x20 ip_finish_output2+0x196/0x480 ... That is the WARN_ON(session->magic != L2TP_SESSION_MAGIC) inside the inlined pppol2tp_sock_to_session(). I counted 704 of these in ~2 minutes (24 in the first minute, 90, then 241 in the third). Each one dumps the full module list and a stack trace, which is what ultimately wedges the box. ANALYSIS I compared the shipped modules from -142 and -147: * l2tp_core.ko is byte-identical between the two (objdump -d output diffs clean apart from the filename), so l2tp_core itself did not change. * l2tp_ppp.ko changed in exactly two functions: pppol2tp_connect (+7 bytes, one added store, from the ppp RCU-free backport) and pppol2tp_ioctl (+126 bytes). The pppol2tp_ioctl change is the -146 backport of l2tp: pppol2tp: hold reference to session in pppol2tp_ioctl() (upstream stable patchset 2026-08-21, LP: #2164796 // CVE-2026-53262) and I believe it has been applied incorrectly for this tree. Disassembly of pppol2tp_ioctl in 6.8.0-147: 1a78: mov 0x18(%r8),%rbx ; rbx = sock->sk 1aa6: lock xadd %eax,0x80(%rbx) ; sock_hold(sk) -> ref on the SOCKET 1ac1: mov 0x278(%rbx),%r13 ; session = sk->sk_user_data 1ad1: cmpl $0xc04eb7d,0x0(%r13) ; session->magic == L2TP_SESSION_MAGIC ... 1b0c: call l2tp_session_dec_refcount ; -> ref dropped on the SESSION It takes the reference on the socket (6.8's pppol2tp_sock_to_session() still does sock_hold(sk)) but releases it on the session. Counting relocations in the two builds: l2tp_tunnel_get_session l2tp_session_dec_refcount 6.8.0-142 1 2 6.8.0-147 1 3 In -142 both dec_refcount calls pair with the single l2tp_tunnel_get_session() in the PPPIOCGL2TPSTATS peer-session path. In -147 there are three, and the two extra ones are on the generic exit paths, where nothing ever took a session reference. The matching sock_put(sk) only happens on the !session / bad-magic error paths, so on every *successful* ioctl the socket refcount is leaked and the session refcount is decremented once too often. The ioctl handler in -147 accepts 0x40047452, 0x40047459, 0x80047453, 0x8004745a and 0x80487436, i.e. PPPIOCSMRU, PPPIOCSFLAGS, PPPIOCGMRU, PPPIOCGFLAGS and PPPIOCGL2TPSTATS. pppd issues PPPIOCSMRU and PPPIOCSFLAGS on the PPPoL2TP session socket during link setup, so session->ref_count is driven to zero within a second of the tunnel coming up. The session is freed while still linked into the tunnel, l2tp_tunnel_get_session() then hits "addition on 0" on the next inbound packet, and pppol2tp_xmit()'s magic check fails forever after -- exactly the sequence above. Suggested fix: on 6.8, where pppol2tp_sock_to_session() returns with a sock_hold() on sk, those exit paths need sock_put(sk) rather than l2tp_session_dec_refcount(session). The upstream commit applies to a tree where the lookup helper hands back a session-refcounted pointer, which is why it does not transplant as-is. (Caveat: this is binary-level analysis of the shipped modules, I have not built a test kernel to confirm it.) -- You received this bug notification because you are subscribed to linux in Ubuntu. Matching subscriptions: Bgg, Bmail, Nb https://bugs.launchpad.net/bugs/2169477 Title: [regression] 6.8.0-146: l2tp/pppol2tp use-after-free -> kernel panic / soft lockup (L2TP/IPsec, xl2tpd) Status in linux package in Ubuntu: Confirmed Bug description: Package: linux (Ubuntu Noble) Affected: linux-image-6.8.0-146-generic 6.8.0-146.146 Last good: 6.8.0-142.142 (also 6.8.0-139.139) == Summary == After upgrading from 6.8.0-142 to 6.8.0-146, a host acting as an L2TP/IPsec server (LNS) and client (LAC) at the same time crashes within 1-10 minutes of boot: kernel panic ("Fatal exception in interrupt") or a permanent soft lockup. 11 crashes in ~2 hours, all on 6.8.0-146, all inside l2tp_core/l2tp_ppp. Booting back into 6.8.0-142 fixes it completely (no warnings at all since). Among the l2tp modules, only l2tp_ppp.ko changed between 142 and 146: l2tp_core srcversion: D5976517E8DDA0096C46F58 (139, 142, 146 - identical) l2tp_ppp srcversion: 410F7BD8BFBF7AF73E5803E (139, 142) 5FE6350B4BF3E0647293B12 (146) Suspected commit (146 changelog, LP: #2164796, CVE-2026-53262): "l2tp: pppol2tp: hold reference to session in pppol2tp_ioctl()" All failures are refcount errors on the L2TP session object and the faults that follow from them, which suggests an unbalanced session reference in the backport. == Environment == Ubuntu 24.04.5 LTS, x86_64 Hardware: YF ADLNN01 (Intel N150), BIOS 5.27 12/17/2025, 4x Intel I226-V (igc) xl2tpd 1.3.18-1build2, ppp 2.4.9-1+1.1ubuntu4, strongswan 5.9.13-2ubuntu4.24.04.5 Kernel-mode L2TP (l2tp_ppp), L2TP over IPsec transport (ESP in UDP 4500), xl2tpd configured with one [lns default] (several clients) plus one [lac] with redial=yes, redial timeout=30. == Reproduction == 1. Boot 6.8.0-146 with xl2tpd (LNS + LAC) over strongswan IPsec. 2. Let 2-3 L2TP tunnels come up; reconnects / session teardown happen. 3. Within 1-10 minutes: refcount warnings, then panic or soft lockup. Not reproducible on 6.8.0-142 (4 boots, hours of uptime, zero warnings). == Crash signatures (6.8.0-146) == 1) refcount UAF on RX path (first symptom on every bad boot) refcount_t: addition on 0; use-after-free. WARNING: CPU: 1 PID: 42 at lib/refcount.c:25 refcount_warn_saturate+0x12e/0x150 Workqueue: events xfrm_trans_reinject l2tp_tunnel_get_session+0xc8/0xd0 [l2tp_core] l2tp_udp_recv_core+0xb1/0x350 [l2tp_core] l2tp_udp_encap_recv+0x2c/0x54 [l2tp_core] udp_queue_rcv_one_skb / __udp4_lib_rcv / udp_rcv followed by: refcount_t: underflow; use-after-free. (l2tp_session_dec_refcount+0xbb <- l2tp_udp_recv_core+0x103) refcount_t: saturated; leaking memory. 2) NULL pointer dereference on TX path -> panic BUG: kernel NULL pointer dereference, address: 000000000000002d CPU: 2 PID: 2608 Comm: pppd Tainted: G W 6.8.0-146-generic #146-Ubuntu xfrm_dst_check+0x24/0x40 __sk_dst_check+0x8f/0xa0 __ip_queue_xmit+0x2a0/0x510 l2tp_xmit_core.constprop.0+0x225/0x460 [l2tp_core] l2tp_xmit_skb+0x30/0xa0 [l2tp_core] pppol2tp_xmit+0x114/0x220 [l2tp_ppp] Kernel panic - not syncing: Fatal exception in interrupt (other instances: RIP _raw_spin_lock+0x13 / pppol2tp_xmit+0x1e1) 3) Soft lockup in tunnel deletion (host hangs, no auto-reboot) watchdog: BUG: soft lockup - CPU#1 stuck for 26s..130s! [kworker/u8:3:176] Workqueue: l2tp l2tp_tunnel_del_work [l2tp_core] RIP: _raw_spin_lock_bh+0x1b / l2tp_tunnel_del_work+0x98 l2tp_tunnel_del_work+0x8d/0x1b0 [l2tp_core] process_one_work / worker_thread / kthread preceded by a warning in pppol2tp_release+0x153 [l2tp_ppp] (sock close path). Some dumps also show faults in __mptcp_close_ssk / tcp_release_cb_override / xfrm_bundle_ok after the l2tp refcount warnings. These are most likely secondary memory corruption. == Boot statistics == 6.8.0-139: 16 days uptime, no issues 6.8.0-146: 13 boots in 2h, 11 pstore crash dumps (all l2tp), boots in between ended in hard hangs 6.8.0-142: 4 boots, 0 refcount warnings, 0 soft lockups, 0 l2tp traces == Attachments == pstore/*.txt - EFI pstore dumps (systemd-pstore). Inside each file the parts are in reverse order (PartN first). IPv4/MAC addresses redacted. == Workaround == Pin 6.8.0-142 in GRUB and hold linux-image-generic. To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2169477/+subscriptions
Комментариев нет:
Отправить комментарий