суббота

[Bug 2084979] [NEW] Device Mapper accepts writes even when the underlying devices are dead (offline or transport-offline)

Public bug reported:

**********
#### Steps to reproduce
**********

root@ubusrv:~# lsblk /dev/sda
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
sda 8:0 0 2G 0 disk
└─sda1 8:1 0 2G 0 part
root@ubusrv:~#

root@ubusrv:~# pvcreate /dev/sda1
Physical volume "/dev/sda1" successfully created.
root@ubusrv:~# vgcreate vgtmp /dev/sda1
Volume group "vgtmp" successfully created
root@ubusrv:~# lvcreate -n lvtmp -l 100%free vgtmp
Logical volume "lvtmp" created.
root@ubusrv:~# lvs
LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
ubuntu-lv ubuntu-vg -wi-ao---- 18.22g
lvtmp vgtmp -wi-a----- <2.00g
root@ubusrv:~#

root@ubusrv:~# cat /sys/block/sda/device/state
running
root@ubusrv:~# echo offline > /sys/block/sda/device/state
root@ubusrv:~# cat /sys/block/sda/device/state
offline
root@ubusrv:~#

root@ubusrv:~# l file.tar
-rw-r--r-- 1 root root 2.4G Oct 19 14:12 file.tar
root@ubusrv:~#


----------------------
1. Reading from faulty DM device: OK - expected behavior
root@ubusrv:~# dd if=/dev/mapper/vgtmp-lvtmp
dd: error reading '/dev/mapper/vgtmp-lvtmp': Input/output error
0+0 records in
0+0 records out
0 bytes copied, 0.00298229 s, 0.0 kB/s
root@ubusrv:~#

2. Reading from faulty/offline underlying sda: OK - expected behavior
root@ubusrv:~# dd if=/dev/sda1
dd: failed to open '/dev/sda1': No such device or address
root@ubusrv:~#

3. Writing to faulty/offline underlying sda: OK - expected behavior
root@ubusrv:~# dd of=/dev/sda1
dd: failed to open '/dev/sda1': No such device or address
root@ubusrv:~#

4. And writing to faulty DM device: not OK - UNEXPECTED behavior
root@ubusrv:~# dd if=file.tar of=/dev/mapper/vgtmp-lvtmp bs=10M count=204
204+0 records in
204+0 records out
2139095040 bytes (2.1 GB, 2.0 GiB) copied, 4.57038 s, 468 MB/s <<<<<<<<
root@ubusrv:~#
----------------------


******** Where this data has been stored?
Not in memory - not enough space:
root@ubusrv:~# free
total used free shared buff/cache available
Mem: 961Mi 361Mi 200Mi 1.3Mi 556Mi 599Mi <<<<
Swap: 2.0Gi 0B 2.0Gi
root@ubusrv:~#


And the same applies to iSCSI+MPIO.
----------------------
root@ubusrv:~# multipath -ll
mpatha (3600140582d6e7c50c0347908f0d094c2) dm-2 LIO-ORG,ubusrv-wrbk
size=4.4G features='1 queue_if_no_path' hwhandler='1 alua' wp=rw
`-+- policy='service-time 0' prio=50 status=active
|- 8:0:0:135 sdc 8:32 active ready running
`- 7:0:0:135 sdb 8:16 active ready running
mpathb (3600140512e2d3ad7ae048aba1ee0a33a) dm-3 LIO-ORG,ubusrv-wrth
size=3.9G features='1 queue_if_no_path' hwhandler='1 alua' wp=rw
`-+- policy='service-time 0' prio=50 status=active
|- 8:0:0:235 sdd 8:48 active ready running
`- 7:0:0:235 sde 8:64 active ready running
root@ubusrv:~#

#### poweroff iSCSI target/server

root@ubusrv:~# egrep 'mpath[ab]' /var/log/syslog | tail
2024-10-19T15:07:01.810687+02:00 ubusrv multipathd[382]: checker failed path 8:64 in map mpathb
2024-10-19T15:07:01.811658+02:00 ubusrv multipathd[382]: mpathb: remaining active paths: 1
2024-10-19T15:07:02.810464+02:00 ubusrv multipathd[382]: checker failed path 8:16 in map mpatha
2024-10-19T15:07:02.813151+02:00 ubusrv multipathd[382]: mpatha: Entering recovery mode: max_retries=12
2024-10-19T15:07:02.813182+02:00 ubusrv multipathd[382]: mpatha: remaining active paths: 0
2024-10-19T15:07:04.811717+02:00 ubusrv multipathd[382]: checker failed path 8:48 in map mpathb
2024-10-19T15:07:04.813169+02:00 ubusrv multipathd[382]: mpathb: Entering recovery mode: max_retries=12
2024-10-19T15:07:04.813206+02:00 ubusrv multipathd[382]: mpathb: remaining active paths: 0
2024-10-19T15:08:02.844668+02:00 ubusrv multipathd[382]: mpatha: Disable queueing
2024-10-19T15:08:04.845680+02:00 ubusrv multipathd[382]: mpathb: Disable queueing
root@ubusrv:~#

root@ubusrv:~# iscsiadm -m session -P 3 | grep Att
Attached SCSI devices:
Attached scsi disk sdb State: transport-offline
Attached scsi disk sde State: transport-offline
Attached SCSI devices:
Attached scsi disk sdc State: transport-offline
Attached scsi disk sdd State: transport-offline
root@ubusrv:~#

root@ubusrv:~# multipath -ll
mpatha (3600140582d6e7c50c0347908f0d094c2) dm-2 LIO-ORG,ubusrv-wrbk
size=4.4G features='0' hwhandler='1 alua' wp=rw
`-+- policy='service-time 0' prio=0 status=active
|- 8:0:0:135 sdc 8:32 failed faulty running
`- 7:0:0:135 sdb 8:16 failed faulty running
mpathb (3600140512e2d3ad7ae048aba1ee0a33a) dm-3 LIO-ORG,ubusrv-wrth
size=3.9G features='0' hwhandler='1 alua' wp=rw
`-+- policy='service-time 0' prio=0 status=active
|- 8:0:0:235 sdd 8:48 failed faulty running
`- 7:0:0:235 sde 8:64 failed faulty running
root@ubusrv:~#

root@ubusrv:~# cat /sys/block/sd[b-e]/device/state
transport-offline
transport-offline
transport-offline
transport-offline
root@ubusrv:~#

root@ubusrv:~# l /dev/mapper/mpath*
lrwxrwxrwx 1 root root 7 Oct 19 13:35 /dev/mapper/mpatha -> ../dm-2
lrwxrwxrwx 1 root root 7 Oct 19 13:35 /dev/mapper/mpatha-part1 -> ../dm-4
lrwxrwxrwx 1 root root 7 Oct 19 13:35 /dev/mapper/mpathb -> ../dm-3
lrwxrwxrwx 1 root root 7 Oct 19 13:35 /dev/mapper/mpathb-part1 -> ../dm-5
root@ubusrv:~#

********
root@ubusrv:~# dd if=file.tar of=/dev/mapper/mpatha-part1 bs=10M
240+1 records in
240+1 records out
2526904320 bytes (2.5 GB, 2.4 GiB) copied, 5.65064 s, 447 MB/s <<<<<<<<
root@ubusrv:~#
********
----------------------


Is it a Device Mapper feature?


--
Regards,
Mirek

ProblemType: Bug
DistroRelease: Ubuntu 24.04
Package: linux-image-6.8.0-47-generic 6.8.0-47.47
ProcVersionSignature: Ubuntu 6.8.0-47.47-generic 6.8.12
Uname: Linux 6.8.0-47-generic x86_64
AlsaVersion: Advanced Linux Sound Architecture Driver Version k6.8.0-47-generic.
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay'
ApportVersion: 2.28.1-0ubuntu3.1
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord'
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/timer', '/dev/snd/seq', '/dev/snd/pcmC0D0p', '/dev/snd/pcmC0D0c', '/dev/snd/hwC0D0', '/dev/snd/controlC0', '/dev/snd/by-path'] failed with exit code 1:
CRDA: N/A
Card0.Amixer.info: Error: [Errno 2] No such file or directory: 'amixer'
Card0.Amixer.values: Error: [Errno 2] No such file or directory: 'amixer'
CasperMD5CheckResult: pass
Date: Sat Oct 19 15:39:00 2024
InstallationDate: Installed on 2024-09-03 (46 days ago)
InstallationMedia: Ubuntu-Server 24.04 LTS "Noble Numbat" - Release amd64 (20240423)
IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig'
Lsusb:
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 001 Device 002: ID 0627:0001 Adomax Technology Co., Ltd QEMU Tablet
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Lsusb-t:
/: Bus 001.Port 001: Dev 001, Class=root_hub, Driver=xhci_hcd/15p, 480M
|__ Port 001: Dev 002, If 0, Class=Human Interface Device, Driver=usbhid, 480M
/: Bus 002.Port 001: Dev 001, Class=root_hub, Driver=xhci_hcd/15p, 5000M
MachineType: QEMU Standard PC (Q35 + ICH9, 2009)
ProcEnviron:
LANG=en_US.UTF-8
PATH=(custom, no user)
SHELL=/bin/bash
TERM=xterm-256color
ProcFB: 0 qxldrmfb
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-6.8.0-47-generic root=/dev/mapper/ubuntu--vg-ubuntu--lv ro
RelatedPackageVersions:
linux-restricted-modules-6.8.0-47-generic N/A
linux-backports-modules-6.8.0-47-generic N/A
linux-firmware 20240318.git3b128b60-0ubuntu2.4
RfKill: Error: [Errno 2] No such file or directory: 'rfkill'
SourcePackage: linux
UpgradeStatus: No upgrade log present (probably fresh install)
dmi.bios.date: 04/01/2014
dmi.bios.release: 0.0
dmi.bios.vendor: SeaBIOS
dmi.bios.version: 1.16.3-debian-1.16.3-2
dmi.chassis.type: 1
dmi.chassis.vendor: QEMU
dmi.chassis.version: pc-q35-8.2
dmi.modalias: dmi:bvnSeaBIOS:bvr1.16.3-debian-1.16.3-2:bd04/01/2014:br0.0:svnQEMU:pnStandardPC(Q35+ICH9,2009):pvrpc-q35-8.2:cvnQEMU:ct1:cvrpc-q35-8.2:sku:
dmi.product.name: Standard PC (Q35 + ICH9, 2009)
dmi.product.version: pc-q35-8.2
dmi.sys.vendor: QEMU

** Affects: linux (Ubuntu)
Importance: Undecided
Status: New


** Tags: amd64 apport-bug noble

--
You received this bug notification because you are subscribed to linux
in Ubuntu.
Matching subscriptions: Bgg, Bmail, Nb
https://bugs.launchpad.net/bugs/2084979

Title:
Device Mapper accepts writes even when the underlying devices are dead
(offline or transport-offline)

Status in linux package in Ubuntu:
New

Bug description:
**********
#### Steps to reproduce
**********

root@ubusrv:~# lsblk /dev/sda
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
sda 8:0 0 2G 0 disk
└─sda1 8:1 0 2G 0 part
root@ubusrv:~#

root@ubusrv:~# pvcreate /dev/sda1
Physical volume "/dev/sda1" successfully created.
root@ubusrv:~# vgcreate vgtmp /dev/sda1
Volume group "vgtmp" successfully created
root@ubusrv:~# lvcreate -n lvtmp -l 100%free vgtmp
Logical volume "lvtmp" created.
root@ubusrv:~# lvs
LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
ubuntu-lv ubuntu-vg -wi-ao---- 18.22g
lvtmp vgtmp -wi-a----- <2.00g
root@ubusrv:~#

root@ubusrv:~# cat /sys/block/sda/device/state
running
root@ubusrv:~# echo offline > /sys/block/sda/device/state
root@ubusrv:~# cat /sys/block/sda/device/state
offline
root@ubusrv:~#

root@ubusrv:~# l file.tar
-rw-r--r-- 1 root root 2.4G Oct 19 14:12 file.tar
root@ubusrv:~#


----------------------
1. Reading from faulty DM device: OK - expected behavior
root@ubusrv:~# dd if=/dev/mapper/vgtmp-lvtmp
dd: error reading '/dev/mapper/vgtmp-lvtmp': Input/output error
0+0 records in
0+0 records out
0 bytes copied, 0.00298229 s, 0.0 kB/s
root@ubusrv:~#

2. Reading from faulty/offline underlying sda: OK - expected behavior
root@ubusrv:~# dd if=/dev/sda1
dd: failed to open '/dev/sda1': No such device or address
root@ubusrv:~#

3. Writing to faulty/offline underlying sda: OK - expected behavior
root@ubusrv:~# dd of=/dev/sda1
dd: failed to open '/dev/sda1': No such device or address
root@ubusrv:~#

4. And writing to faulty DM device: not OK - UNEXPECTED behavior
root@ubusrv:~# dd if=file.tar of=/dev/mapper/vgtmp-lvtmp bs=10M count=204
204+0 records in
204+0 records out
2139095040 bytes (2.1 GB, 2.0 GiB) copied, 4.57038 s, 468 MB/s <<<<<<<<
root@ubusrv:~#
----------------------


******** Where this data has been stored?
Not in memory - not enough space:
root@ubusrv:~# free
total used free shared buff/cache available
Mem: 961Mi 361Mi 200Mi 1.3Mi 556Mi 599Mi <<<<
Swap: 2.0Gi 0B 2.0Gi
root@ubusrv:~#


And the same applies to iSCSI+MPIO.
----------------------
root@ubusrv:~# multipath -ll
mpatha (3600140582d6e7c50c0347908f0d094c2) dm-2 LIO-ORG,ubusrv-wrbk
size=4.4G features='1 queue_if_no_path' hwhandler='1 alua' wp=rw
`-+- policy='service-time 0' prio=50 status=active
|- 8:0:0:135 sdc 8:32 active ready running
`- 7:0:0:135 sdb 8:16 active ready running
mpathb (3600140512e2d3ad7ae048aba1ee0a33a) dm-3 LIO-ORG,ubusrv-wrth
size=3.9G features='1 queue_if_no_path' hwhandler='1 alua' wp=rw
`-+- policy='service-time 0' prio=50 status=active
|- 8:0:0:235 sdd 8:48 active ready running
`- 7:0:0:235 sde 8:64 active ready running
root@ubusrv:~#

#### poweroff iSCSI target/server

root@ubusrv:~# egrep 'mpath[ab]' /var/log/syslog | tail
2024-10-19T15:07:01.810687+02:00 ubusrv multipathd[382]: checker failed path 8:64 in map mpathb
2024-10-19T15:07:01.811658+02:00 ubusrv multipathd[382]: mpathb: remaining active paths: 1
2024-10-19T15:07:02.810464+02:00 ubusrv multipathd[382]: checker failed path 8:16 in map mpatha
2024-10-19T15:07:02.813151+02:00 ubusrv multipathd[382]: mpatha: Entering recovery mode: max_retries=12
2024-10-19T15:07:02.813182+02:00 ubusrv multipathd[382]: mpatha: remaining active paths: 0
2024-10-19T15:07:04.811717+02:00 ubusrv multipathd[382]: checker failed path 8:48 in map mpathb
2024-10-19T15:07:04.813169+02:00 ubusrv multipathd[382]: mpathb: Entering recovery mode: max_retries=12
2024-10-19T15:07:04.813206+02:00 ubusrv multipathd[382]: mpathb: remaining active paths: 0
2024-10-19T15:08:02.844668+02:00 ubusrv multipathd[382]: mpatha: Disable queueing
2024-10-19T15:08:04.845680+02:00 ubusrv multipathd[382]: mpathb: Disable queueing
root@ubusrv:~#

root@ubusrv:~# iscsiadm -m session -P 3 | grep Att
Attached SCSI devices:
Attached scsi disk sdb State: transport-offline
Attached scsi disk sde State: transport-offline
Attached SCSI devices:
Attached scsi disk sdc State: transport-offline
Attached scsi disk sdd State: transport-offline
root@ubusrv:~#

root@ubusrv:~# multipath -ll
mpatha (3600140582d6e7c50c0347908f0d094c2) dm-2 LIO-ORG,ubusrv-wrbk
size=4.4G features='0' hwhandler='1 alua' wp=rw
`-+- policy='service-time 0' prio=0 status=active
|- 8:0:0:135 sdc 8:32 failed faulty running
`- 7:0:0:135 sdb 8:16 failed faulty running
mpathb (3600140512e2d3ad7ae048aba1ee0a33a) dm-3 LIO-ORG,ubusrv-wrth
size=3.9G features='0' hwhandler='1 alua' wp=rw
`-+- policy='service-time 0' prio=0 status=active
|- 8:0:0:235 sdd 8:48 failed faulty running
`- 7:0:0:235 sde 8:64 failed faulty running
root@ubusrv:~#

root@ubusrv:~# cat /sys/block/sd[b-e]/device/state
transport-offline
transport-offline
transport-offline
transport-offline
root@ubusrv:~#

root@ubusrv:~# l /dev/mapper/mpath*
lrwxrwxrwx 1 root root 7 Oct 19 13:35 /dev/mapper/mpatha -> ../dm-2
lrwxrwxrwx 1 root root 7 Oct 19 13:35 /dev/mapper/mpatha-part1 -> ../dm-4
lrwxrwxrwx 1 root root 7 Oct 19 13:35 /dev/mapper/mpathb -> ../dm-3
lrwxrwxrwx 1 root root 7 Oct 19 13:35 /dev/mapper/mpathb-part1 -> ../dm-5
root@ubusrv:~#

********
root@ubusrv:~# dd if=file.tar of=/dev/mapper/mpatha-part1 bs=10M
240+1 records in
240+1 records out
2526904320 bytes (2.5 GB, 2.4 GiB) copied, 5.65064 s, 447 MB/s <<<<<<<<
root@ubusrv:~#
********
----------------------


Is it a Device Mapper feature?


--
Regards,
Mirek

ProblemType: Bug
DistroRelease: Ubuntu 24.04
Package: linux-image-6.8.0-47-generic 6.8.0-47.47
ProcVersionSignature: Ubuntu 6.8.0-47.47-generic 6.8.12
Uname: Linux 6.8.0-47-generic x86_64
AlsaVersion: Advanced Linux Sound Architecture Driver Version k6.8.0-47-generic.
AplayDevices: Error: [Errno 2] No such file or directory: 'aplay'
ApportVersion: 2.28.1-0ubuntu3.1
Architecture: amd64
ArecordDevices: Error: [Errno 2] No such file or directory: 'arecord'
AudioDevicesInUse: Error: command ['fuser', '-v', '/dev/snd/timer', '/dev/snd/seq', '/dev/snd/pcmC0D0p', '/dev/snd/pcmC0D0c', '/dev/snd/hwC0D0', '/dev/snd/controlC0', '/dev/snd/by-path'] failed with exit code 1:
CRDA: N/A
Card0.Amixer.info: Error: [Errno 2] No such file or directory: 'amixer'
Card0.Amixer.values: Error: [Errno 2] No such file or directory: 'amixer'
CasperMD5CheckResult: pass
Date: Sat Oct 19 15:39:00 2024
InstallationDate: Installed on 2024-09-03 (46 days ago)
InstallationMedia: Ubuntu-Server 24.04 LTS "Noble Numbat" - Release amd64 (20240423)
IwConfig: Error: [Errno 2] No such file or directory: 'iwconfig'
Lsusb:
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 001 Device 002: ID 0627:0001 Adomax Technology Co., Ltd QEMU Tablet
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Lsusb-t:
/: Bus 001.Port 001: Dev 001, Class=root_hub, Driver=xhci_hcd/15p, 480M
|__ Port 001: Dev 002, If 0, Class=Human Interface Device, Driver=usbhid, 480M
/: Bus 002.Port 001: Dev 001, Class=root_hub, Driver=xhci_hcd/15p, 5000M
MachineType: QEMU Standard PC (Q35 + ICH9, 2009)
ProcEnviron:
LANG=en_US.UTF-8
PATH=(custom, no user)
SHELL=/bin/bash
TERM=xterm-256color
ProcFB: 0 qxldrmfb
ProcKernelCmdLine: BOOT_IMAGE=/vmlinuz-6.8.0-47-generic root=/dev/mapper/ubuntu--vg-ubuntu--lv ro
RelatedPackageVersions:
linux-restricted-modules-6.8.0-47-generic N/A
linux-backports-modules-6.8.0-47-generic N/A
linux-firmware 20240318.git3b128b60-0ubuntu2.4
RfKill: Error: [Errno 2] No such file or directory: 'rfkill'
SourcePackage: linux
UpgradeStatus: No upgrade log present (probably fresh install)
dmi.bios.date: 04/01/2014
dmi.bios.release: 0.0
dmi.bios.vendor: SeaBIOS
dmi.bios.version: 1.16.3-debian-1.16.3-2
dmi.chassis.type: 1
dmi.chassis.vendor: QEMU
dmi.chassis.version: pc-q35-8.2
dmi.modalias: dmi:bvnSeaBIOS:bvr1.16.3-debian-1.16.3-2:bd04/01/2014:br0.0:svnQEMU:pnStandardPC(Q35+ICH9,2009):pvrpc-q35-8.2:cvnQEMU:ct1:cvrpc-q35-8.2:sku:
dmi.product.name: Standard PC (Q35 + ICH9, 2009)
dmi.product.version: pc-q35-8.2
dmi.sys.vendor: QEMU

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2084979/+subscriptions

Комментариев нет:

Отправить комментарий