[Bug 2045560] Re: TCP memory leak, slow network (arm64)

Cool, thank you for details, all clear now.

--
You received this bug notification because you are subscribed to linux
in Ubuntu.
Matching subscriptions: Bgg, Bmail, Nb
https://bugs.launchpad.net/bugs/2045560

Title:
TCP memory leak, slow network (arm64)

Status in linux package in Ubuntu:
Fix Committed
Status in linux source package in Mantic:
Fix Committed
Status in linux source package in Noble:
Fix Committed

Bug description:
Hello! 👋

We have Ubuntu OS-based servers running in both AWS and Azure clouds.
These servers are handling thousands of connections, and we've been
experiencing issues with TCP memory usage since upgrading to Ubuntu
22.04.3 from 22.04.2.

$ cat /proc/net/sockstat
sockets: used 6642
TCP: inuse 5962 orphan 0 tw 292 alloc 6008 mem 128989
UDP: inuse 5 mem 0
UDPLITE: inuse 0
RAW: inuse 0
FRAG: inuse 0 memory 0

As shown in the output below, even after stopping all possible
services and closing all open connections, the TCP memory usage
remains high and only decreases very slowly.

$ cat /proc/net/sockstat
sockets: used 138
TCP: inuse 2 orphan 0 tw 0 alloc 3 mem 128320
UDP: inuse 3 mem 0
UDPLITE: inuse 0
RAW: inuse 0
FRAG: inuse 0 memory 0

I have attached a screenshot of linear TCP memory usage growth, which
we believe may indicate a TCP memory leak

When net.ipv4.tcp_mem limit is reached, it causes network slowdown

We've never had these issues before, and the only solution we've found
so far is to reboot the node. Do you have any suggestions on how to
troubleshoot further?

Thank you for any help or guidance you can provide!

ProblemType: Bug
DistroRelease: Ubuntu 22.04
Package: linux-image-6.2.0-1015-aws 6.2.0-1015.15~22.04.1
ProcVersionSignature: Ubuntu 6.2.0-1015.15~22.04.1-aws 6.2.16
Uname: Linux 6.2.0-1015-aws aarch64
ApportVersion: 2.20.11-0ubuntu82.5
Architecture: arm64
CasperMD5CheckResult: unknown
CloudArchitecture: aarch64
CloudID: aws
CloudName: aws
CloudPlatform: ec2
CloudRegion: us-west-2
CloudSubPlatform: metadata (http://169.254.169.254)
Date: Mon Dec 4 13:13:08 2023
Ec2AMI: ami-095a68e28e781dfe1
Ec2AMIManifest: (unknown)
Ec2Architecture: arm64
Ec2AvailabilityZone: us-west-2b
Ec2Imageid: ami-095a68e28e781dfe1
Ec2InstanceType: m7g.large
Ec2Instancetype: m7g.large
Ec2Kernel: unavailable
Ec2Ramdisk: unavailable
Ec2Region: us-west-2
ProcEnviron:
 TERM=xterm-256color
 PATH=(custom, no user)
 LANG=C.UTF-8
 SHELL=/bin/bash
RebootRequiredPkgs: Error: path contained symlinks.
SourcePackage: linux-signed-aws-6.2
UpgradeStatus: No upgrade log present (probably fresh install)

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2045560/+subscriptions