[Bug 2143083] [NEW] [SRU] Duplicated entries in /proc//mountinfo

Public bug reported:

[ Impact ]

Sometimes on a system that is mounting and unmounting filesystems frequently,
for example running lots of docker containers, the size of /proc/1/mountinfo,
can become very large -- 100s, to 1000s of entries or more -- with the vast
majority being a single entry duplicated many times.

This causes other problems on the system, due to systemd parsing the mount table
whenever it changes, and eating up a lot of memory, for example [0]. Waiting
long enough there are rare events where the length of mountinfo can go into the
millions of lines and lead to OOM and kernel panics.

[Fix]

Christian Brauner submitted a patch on the mailing list[1] (now merged upstream [2])
which fixes the issue by
a) using the unique mount ID as the pos for iterating the mounts
b) updating to the pos of the iterator before returning

[ Test Plan ]

To test whether this issue occurs, it is sufficient to rapidly mount and unmount
tmpfs rapidly and poll for duplicates in `/proc/1/mountinfo` this can be done
for example by the following pair of scripts:

repro.sh
--------
#!/bin/bash
counter=0
while true; do
echo -n "."
unique_name="tmpfs_$$_$counter"
mkdir -p "/tmp/$unique_name"
sudo mount -t tmpfs "$unique_name" "/tmp/$unique_name"
sudo umount "/tmp/$unique_name"
rmdir "/tmp/$unique_name"
((counter++))
sleep 0.1
done
-------

has-bug.sh
--------
#!/bin/bash
THRESHOLD=100
WAIT_MIN=30
WAIT_SECONDS=$((WAIT_MIN * 60))
SECONDS=0
while ((SECONDS < WAIT_SECONDS)); do
# Get mountinfo entries and count total
mountinfo="$(cat /proc/1/mountinfo)"
mountinfo_count=$(echo "$mountinfo" | wc -l)

if ((mountinfo_count > THRESHOLD)); then
echo "$(date): Mount count ($mountinfo_count) exceeds threshold ($THRESHOLD)"

# Find and log duplicate mount points with their counts
duplicates=$(echo "$mountinfo" | sort | uniq -cd)

if [[ -n "$duplicates" ]]; then
echo "Duplicate mounts :"
echo "$duplicates" | while read -r count mountpoint; do
echo " $mountpoint: $count occurrences"
done
fi
exit 0
fi

sleep 0.1
done
exit 1
------

In my testing, about 5 minutes is sufficient time for the bug to occur, so I've
set the timeout to 30min in the test plan out of an abundance of caution.

If the bug is present in the running kernel, `has-bug.sh` will print a message
about duplicate entries and exit success, otherwise it will time out and exit
with return code 1.

The high level test plan is:

1. Run `has-bug.sh` and `repro.sh` on unpatched kernel.
2. Observe that bug is present
3. Upgrade to patched kernel and restart
4. Run `has-bug.sh` and `repro.sh` on unpatched kernel.
5. Observe that bug is no longer present

[ Where problems could occur ]

* The patch to fix this modifies how iterator position is tracked when iterating
mounts. It therefore potentially affects anything that iterates through the mounts.

[ Other Info ]

[0]: https://github.com/systemd/systemd/issues/37939
[1]: https://lore.kernel.org/lkml/20260129-geleckt-treuhand-4bb940acacd9@brauner/
[2]: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=4a403d7aa9074f527f064ef0806aaab38d14b07c

** Affects: linux (Ubuntu)
Importance: Undecided
Status: New

** Affects: linux (Ubuntu Noble)
Importance: Undecided
Status: New

** Affects: linux (Ubuntu Questing)
Importance: Undecided
Status: New

** Affects: linux (Ubuntu Resolute)
Importance: Undecided
Status: New

** Also affects: linux (Ubuntu Questing)
Importance: Undecided
Status: New

** Also affects: linux (Ubuntu Resolute)
Importance: Undecided
Status: New

** Also affects: linux (Ubuntu Noble)
Importance: Undecided
Status: New

--
You received this bug notification because you are subscribed to linux
in Ubuntu.
Matching subscriptions: Bgg, Bmail, Nb
https://bugs.launchpad.net/bugs/2143083

Title:
[SRU] Duplicated entries in /proc/<pid>/mountinfo

Status in linux package in Ubuntu:
New
Status in linux source package in Noble:
New
Status in linux source package in Questing:
New
Status in linux source package in Resolute:
New

Bug description:
[ Impact ]

Sometimes on a system that is mounting and unmounting filesystems frequently,
for example running lots of docker containers, the size of /proc/1/mountinfo,
can become very large -- 100s, to 1000s of entries or more -- with the vast
majority being a single entry duplicated many times.

This causes other problems on the system, due to systemd parsing the mount table
whenever it changes, and eating up a lot of memory, for example [0]. Waiting
long enough there are rare events where the length of mountinfo can go into the
millions of lines and lead to OOM and kernel panics.

[Fix]

Christian Brauner submitted a patch on the mailing list[1] (now merged upstream [2])
which fixes the issue by
a) using the unique mount ID as the pos for iterating the mounts
b) updating to the pos of the iterator before returning

[ Test Plan ]

To test whether this issue occurs, it is sufficient to rapidly mount and unmount
tmpfs rapidly and poll for duplicates in `/proc/1/mountinfo` this can be done
for example by the following pair of scripts:

repro.sh
--------
#!/bin/bash
counter=0
while true; do
echo -n "."
unique_name="tmpfs_$$_$counter"
mkdir -p "/tmp/$unique_name"
sudo mount -t tmpfs "$unique_name" "/tmp/$unique_name"
sudo umount "/tmp/$unique_name"
rmdir "/tmp/$unique_name"
((counter++))
sleep 0.1
done
-------

has-bug.sh
--------
#!/bin/bash
THRESHOLD=100
WAIT_MIN=30
WAIT_SECONDS=$((WAIT_MIN * 60))
SECONDS=0
while ((SECONDS < WAIT_SECONDS)); do
# Get mountinfo entries and count total
mountinfo="$(cat /proc/1/mountinfo)"
mountinfo_count=$(echo "$mountinfo" | wc -l)

if ((mountinfo_count > THRESHOLD)); then
echo "$(date): Mount count ($mountinfo_count) exceeds threshold ($THRESHOLD)"

# Find and log duplicate mount points with their counts
duplicates=$(echo "$mountinfo" | sort | uniq -cd)

if [[ -n "$duplicates" ]]; then
echo "Duplicate mounts :"
echo "$duplicates" | while read -r count mountpoint; do
echo " $mountpoint: $count occurrences"
done
fi
exit 0
fi

sleep 0.1
done
exit 1
------

In my testing, about 5 minutes is sufficient time for the bug to occur, so I've
set the timeout to 30min in the test plan out of an abundance of caution.

If the bug is present in the running kernel, `has-bug.sh` will print a message
about duplicate entries and exit success, otherwise it will time out and exit
with return code 1.

The high level test plan is:

1. Run `has-bug.sh` and `repro.sh` on unpatched kernel.
2. Observe that bug is present
3. Upgrade to patched kernel and restart
4. Run `has-bug.sh` and `repro.sh` on unpatched kernel.
5. Observe that bug is no longer present

[ Where problems could occur ]

* The patch to fix this modifies how iterator position is tracked when iterating
mounts. It therefore potentially affects anything that iterates through the mounts.

[ Other Info ]

[0]: https://github.com/systemd/systemd/issues/37939
[1]: https://lore.kernel.org/lkml/20260129-geleckt-treuhand-4bb940acacd9@brauner/
[2]: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=4a403d7aa9074f527f064ef0806aaab38d14b07c

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2143083/+subscriptions