[Bug 2171020] Re: PCI/PM: Restore BARs after powering up during enumeration

I have created a test kernel for resolute: https://people.canonical.com/~mreed/hpe/lp_2133410_kdump_failure/resolute/ The stonking test kernel build is in progress. ** Description changed: - Gather the crash kernel, which hangs. This issue was reported by HPE + [ Impact ] + The issue is reproducible on Ubuntu 24.04.3 with 6.14.0-33 kernel. + + kdump fails during early crash-kernel boot on kernels that include: + 4d4c10f763d7 ("PCI: Explicitly put devices into D0 when initializing") + + This commit was the bisect result. + Reverting only this commit restores normal kdump behavior. + + + [Fix] + + https://git.kernel.org/pub/scm/linux/kernel/git/pci/pci.git/commit/?h=pm&id=f043083843cac1105206437d8c58e725b6412fad + Commit 4d4c10f763d7 ("PCI: Explicitly put devices into D0 when initializing") added pci_pm_power_up_and_verify_state() to pci_pm_init(). The helper ignores pci_power_up()'s return value indicating that BARs need restoring. Its resume callers restore saved configuration afterwards; pci_pm_init() does not. D3hot-to-D0 may reset a device and lose its BARs (PCI PM 1.2, sec 5.4.1). After enumeration powers it up, pci_enable_device() reads D0 and skips the full power-up path. Previously, a device still in D3hot at enable time took that path, including BAR restoration. - On AMD Venice SP7, two AMD xHCI controllers [1022:17d2] are in runtime + On and AMD server, two AMD xHCI controllers [1022:17d2] are in runtime D3hot with No_Soft_Reset clear before kdump. Diagnostic reads on v7.3-rc6-37 show the following for 0000:01:00.4 in the crash kernel: - - PMCSR BAR0 - before the platform power call 0003 f4100004 - after the platform power call 0003 f4100004 - after native D0 power-up 0000 00000004 +                                PMCSR BAR0 +   before the platform power call 0003 f4100004 +   after the platform power call 0003 f4100004 +   after native D0 power-up 0000 00000004 The upper BAR DWORD stays zero. pci_power_up() returns 1, but the BAR address remains zero while dev->resource[0] still contains 0xf4100000. Without BAR restoration, the crash kernel hangs during boot. The last console output includes: - pci 0000:01:00.4: enabling device (0000 -> 0002) +   pci 0000:01:00.4: enabling device (0000 -> 0002) Restoring both controllers' BARs gives matching readbacks and allows kdump to save a vmcore and reboot. An AMD Turin xHCI [1022:1557] shows the same BAR loss; restoring its BAR also allows kdump to complete. Read the initial power state first. Otherwise, pci_power_up() may request a BAR restore based on a cached PCI_UNKNOWN state, even when the device is already in D0. For devices already in D0, only synchronize the platform state so that ACPI calls such as _REG still run. For other states, use pci_set_full_power_state() to power up the device and restore BARs when needed. Impact : kdump to save a vmcore and reboot with this fix. Without it , it hangs. See description above. + + [ Test Plan ] + + [ Where problems could occur ] + + [ Other Info ] ** Summary changed: - PCI/PM: Restore BARs after powering up during enumeration + [SRU] PCI/PM: Restore BARs after powering up during enumeration ** Changed in: linux (Ubuntu Resolute) Assignee: (unassigned) => Michael Reed (mreed8855) ** Changed in: linux (Ubuntu Stonking) Assignee: (unassigned) => Michael Reed (mreed8855) -- You received this bug notification because you are subscribed to linux in Ubuntu. Matching subscriptions: Bgg, Bmail, Nb https://bugs.launchpad.net/bugs/2171020 Title: [SRU] PCI/PM: Restore BARs after powering up during enumeration Status in linux package in Ubuntu: In Progress Status in linux source package in Resolute: In Progress Status in linux source package in Stonking: In Progress Bug description: [ Impact ] The issue is reproducible on Ubuntu 24.04.3 with 6.14.0-33 kernel. kdump fails during early crash-kernel boot on kernels that include: 4d4c10f763d7 ("PCI: Explicitly put devices into D0 when initializing") This commit was the bisect result. Reverting only this commit restores normal kdump behavior. [Fix] https://git.kernel.org/pub/scm/linux/kernel/git/pci/pci.git/commit/?h=pm&id=f043083843cac1105206437d8c58e725b6412fad Commit 4d4c10f763d7 ("PCI: Explicitly put devices into D0 when initializing") added pci_pm_power_up_and_verify_state() to pci_pm_init(). The helper ignores pci_power_up()'s return value indicating that BARs need restoring. Its resume callers restore saved configuration afterwards; pci_pm_init() does not. D3hot-to-D0 may reset a device and lose its BARs (PCI PM 1.2, sec 5.4.1). After enumeration powers it up, pci_enable_device() reads D0 and skips the full power-up path. Previously, a device still in D3hot at enable time took that path, including BAR restoration. On and AMD server, two AMD xHCI controllers [1022:17d2] are in runtime D3hot with No_Soft_Reset clear before kdump. Diagnostic reads on v7.3-rc6-37 show the following for 0000:01:00.4 in the crash kernel:                                PMCSR BAR0   before the platform power call 0003 f4100004   after the platform power call 0003 f4100004   after native D0 power-up 0000 00000004 The upper BAR DWORD stays zero. pci_power_up() returns 1, but the BAR address remains zero while dev->resource[0] still contains 0xf4100000. Without BAR restoration, the crash kernel hangs during boot. The last console output includes:   pci 0000:01:00.4: enabling device (0000 -> 0002) Restoring both controllers' BARs gives matching readbacks and allows kdump to save a vmcore and reboot. An AMD Turin xHCI [1022:1557] shows the same BAR loss; restoring its BAR also allows kdump to complete. Read the initial power state first. Otherwise, pci_power_up() may request a BAR restore based on a cached PCI_UNKNOWN state, even when the device is already in D0. For devices already in D0, only synchronize the platform state so that ACPI calls such as _REG still run. For other states, use pci_set_full_power_state() to power up the device and restore BARs when needed. Impact : kdump to save a vmcore and reboot with this fix. Without it , it hangs. See description above. [ Test Plan ] [ Where problems could occur ] [ Other Info ] To manage notifications about this bug go to: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2171020/+subscriptions