[Bug 2089012] Re: virtio-net slowdown and error message ens3: bad gso: type: 1, size: 1448

Thanks. Unfortunately, I can't replicate the performance slowdown on
dev, only on prod. I do see the dmsg errors on dev. So I can't tell
the performance impact of the diverse ethtool commands.

On prod I notice that the machine at the *other* end of the network
connection had a much higher interrupt load (x10) with 5.15.0-125. This
went back to normal after downgrading to 5.15.0-124.

--
You received this bug notification because you are subscribed to linux
in Ubuntu.
Matching subscriptions: Bgg, Bmail, Nb
https://bugs.launchpad.net/bugs/2089012

Title:
virtio-net slowdown and error message ens3: bad gso: type: 1, size:
1448

Status in linux package in Ubuntu:
Confirmed

Bug description:
On Ubuntu Jammy, when upgrading from linux-image-5.15.0-124-generic to
linux-image-5.15.0-125-generic, we notice on multiple Nutanix VMs:

* A severe slowdown in network performance
* A neverending stream of error messages in dmesg:

[ 779.124760] ens3: bad gso: type: 1, size: 1398
[ 779.136866] ens3: bad gso: type: 1, size: 1398
[ 779.180993] ens3: bad gso: type: 1, size: 1448
[ 779.181002] ens3: bad gso: type: 1, size: 1448
[ 779.181005] ens3: bad gso: type: 1, size: 1448
[ 779.181073] ens3: bad gso: type: 1, size: 1448
[ 779.200088] ens3: bad gso: type: 1, size: 1448
[ 779.200096] ens3: bad gso: type: 1, size: 1448
[ 779.200098] ens3: bad gso: type: 1, size: 1448
[ 870.706518] net_ratelimit: 8 callbacks suppressed

When downgrading to the old kernel, this problem disappears. Forcing
an upgrade to linux-image-5.15.0-126-generic does not help.

This looks a lot like
https://bugzilla.kernel.org/show_bug.cgi?id=219129 but if I
understand the bug tracker well, that patch has already been included
in this kernel.


Some details:

$ lspci -v |grep -i net
00:03.0 Ethernet controller: Red Hat, Inc. Virtio network device


$ lsb_release -rd
Description: Ubuntu 22.04.5 LTS
Release: 22.04

$ apt-cache policy linux-image-generic
linux-image-generic:
Installed: 5.15.0.126.125
Candidate: 5.15.0.126.125
Version table:
*** 5.15.0.126.125 500
500 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages
100 /var/lib/dpkg/status
5.15.0.125.124 500
500 http://archive.ubuntu.com/ubuntu jammy-security/main amd64 Packages
5.15.0.25.27 500
500 http://archive.ubuntu.com/ubuntu jammy/main amd64 Packages

ProblemType: Bug
DistroRelease: Ubuntu 22.04
Package: linux-image-5.15.0-125-generic 5.15.0-125.135
ProcVersionSignature: Ubuntu 5.15.0-125.135-generic 5.15.167
Uname: Linux 5.15.0-125-generic x86_64
AlsaDevices:
total 0
crw-rw---- 1 root audio 116, 1 Nov 19 12:49 seq
crw-rw---- 1 root audio 116, 33 Nov 19 12:49 timer
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay'
ApportVersion: 2.20.11-0ubuntu82.6
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord'
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/seq', '/dev/snd/timer'] failed with exit code 1:
CRDA: N/A
CasperMD5CheckResult: pass
CloudArchitecture: x86_64
CloudID: none
CloudName: none
CloudPlatform: none
CloudSubPlatform: config
Date: Tue Nov 19 13:00:32 2024
InstallationDate: Installed on 2022-06-20 (882 days ago)
InstallationMedia: Ubuntu-Server 22.04 LTS "Jammy Jellyfish" - Release amd64 (20220421)
IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig'
Lsusb:
Bus 001 Device 002: ID 0627:0001 Adomax Technology Co., Ltd QEMU USB Tablet
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 004 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 003 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Bus 002 Device 001: ID 1d6b:0001 Linux Foundation 1.1 root hub
Lsusb-t:
/: Bus 04.Port 1: Dev 1, Class=root_hub, Driver=uhci_hcd/2p, 12M
/: Bus 03.Port 1: Dev 1, Class=root_hub, Driver=uhci_hcd/2p, 12M
/: Bus 02.Port 1: Dev 1, Class=root_hub, Driver=uhci_hcd/2p, 12M
/: Bus 01.Port 1: Dev 1, Class=root_hub, Driver=ehci-pci/6p, 480M
|__ Port 1: Dev 2, If 0, Class=Human Interface Device, Driver=usbhid, 480M
MachineType: Nutanix AHV
PciMultimedia:

ProcEnviron:
TERM=screen
PATH=(custom, no user)
LANG=C.UTF8
SHELL=/bin/bash
ProcFB: 0 bochs-drmdrmfb
ProcKernelCmdLine: BOOT_IMAGE=/boot/vmlinuz-5.15.0-125-generic root=UUID=2dabc077-db09-4f81-bbf2-b8390e5bc560 ro ipv6.disable=1
RelatedPackageVersions:
linux-restricted-modules-5.15.0-125-generic N/A
linux-backports-modules-5.15.0-125-generic N/A
linux-firmware 20220329.git681281e4-0ubuntu3.35
RfKill: Error: [Errno 2] No such file or directory: 'rfkill'
SourcePackage: linux
UpgradeStatus: No upgrade log present (probably fresh install)
dmi.bios.date: 04/01/2014
dmi.bios.release: 0.0
dmi.bios.vendor: SeaBIOS
dmi.bios.version: nutanix-ahv-2.20220304.0.2619.el7
dmi.chassis.type: 1
dmi.chassis.vendor: Red Hat
dmi.chassis.version: RHEL 7.6.0 PC (i440FX + PIIX, 1996)
dmi.modalias: dmi:bvnSeaBIOS:bvrnutanix-ahv-2.20220304.0.2619.el7:bd04/01/2014:br0.0:svnNutanix:pnAHV:pvrRHEL7.6.0PC(i440FX+PIIX,1996):cvnRedHat:ct1:cvrRHEL7.6.0PC(i440FX+PIIX,1996):sku:
dmi.product.family: Red Hat Enterprise Linux
dmi.product.name: AHV
dmi.product.version: RHEL 7.6.0 PC (i440FX + PIIX, 1996)
dmi.sys.vendor: Nutanix

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2089012/+subscriptions