[Bug 2063322] Re: ite-cir driver failed to load due to a regression in linux kernel 6.5+ serial driver change

** Tags added: noble

** Tags added: mantic

--
You received this bug notification because you are subscribed to linux
in Ubuntu.
Matching subscriptions: Bgg, Bmail, Nb
https://bugs.launchpad.net/bugs/2063322

Title:
ite-cir driver failed to load due to a regression in linux kernel 6.5+
serial driver change

Status in linux package in Ubuntu:
New

Bug description:
[Impact]

From Kernel 6.5 and above, the serial driver now claims multiple UART
ports by default, when the previous behaviour was it would only claim
1. Due to this the ite-cir driver can no longer load because the port
it would assign itself is now being used in memory by the 16550/8250
driver. We have found a workaround by disabling the serial driver
from consuming more than 1 port. Below is the steps we went to
troubleshoot, investigate and the solution we've found. This is a
regression from 6.4 behaviour.

When installing 24.04 I noticed that the /dev/lirc0 device was not
loaded on the system which is what is created when the driver is
loaded. After reviewing the driver we use on 20.04 I then searched
the kernel logs and found that the ite-cir driver failed to load due
to error -16.

```
2024-04-19T13:44:51.873060+00:00 host kernel: rc rc0: ITE8708 CIR transceiver as /devices/pnp0/00:03/rc/rc0
2024-04-19T13:44:51.873062+00:00 host kernel: rc rc0: lirc_dev: driver ite-cir registered at minor = 0, raw IR receiver, raw IR transmitter
2024-04-19T13:44:51.873070+00:00 host kernel: input: ITE8708 CIR transceiver as /devices/pnp0/00:03/rc/rc0/input6
2024-04-19T13:44:51.873072+00:00 host kernel: i2c i2c-2: 2/2 memory slots populated (from DMI)
2024-04-19T13:44:51.873076+00:00 host kernel: i2c i2c-2: Successfully instantiated SPD at 0x50
2024-04-19T13:44:51.873078+00:00 host kernel: ite-cir: probe of 00:03 failed with error -16
```

I reviewed kernel error message and this error is due to the
device/resource is busy or in use. My colleague and I reviewed the
ite-cir driver source code and noticed the driver initialisation
succeeds up until L1404
https://github.com/torvalds/linux/blob/master/drivers/media/rc/ite-
cir.c#L1404 where it attempts to request_region, if it fails to do
this it will unregister the device.

The request region allocates a place in memory for the device to use,
we believe it's using the ioport.h's request region function
https://github.com/torvalds/linux/blob/master/include/linux/ioport.h#L278

When we added some debug lines to ite-cir we noticed that the memory
reference it is trying to allocate is 02f8-02ff. We checked the
ioports on 24.04 and noticed that the serial driver was taking up this
reference in memory when in 20.04 (The OS we currently use) and 23.04
it would not.

Ubuntu 24.04 ioports

```
user@host:~$ sudo cat /proc/ioports
0000-0cf7 : PCI Bus 0000:00
  02f8-02ff : serial
  03f8-03ff : serial
```

Ubuntu 20.04 ioports memory allocation

```
user@host:~$ sudo cat /proc/ioports
0000-0cf7 : PCI Bus 0000:00
  02f8-02ff : ite-cir
  03f8-03ff : serial
```

When we investigated any changes that might cause this, we came with
two potential commits that were released in 6.5 that might have caused
this issue:

* https://github.com/torvalds/linux/commit/9d86719f8769244dc99b8cb6091c41eae3fd684f
* https://github.com/torvalds/linux/commit/84a9582fd203063cd4d301204971ff2cd8327f1a

The serial changes allow 16550/8250 driver to consume as many serial
ports as they require initializing the driver, originally they only
used 1. Due to this we noticed that when we checked the serial ports
in use by the device that 8250 was consuming the place in memory that
ite-cir should.

Ubuntu 24.04

```
user@host:~$ sudo setserial -g /dev/ttyS*
/dev/ttyS0, UART: 16550A, Port: 0x03f8, IRQ: 4
/dev/ttyS1, UART: 8250, Port: 0x02f8, IRQ: 3
```

Ubuntu 20.04

```
user@host:~$ sudo setserial -g /dev/ttyS*
/dev/ttyS0, UART: 16550A, Port: 0x03f8, IRQ: 4
/dev/ttyS1, UART: unknown, Port: 0x02f8, IRQ: 3
```
[Regression]
We believe that the regression started from 6.5 kernel onwards based on the commit changes that were done to the serial driver that shipped to those kernels. 23.04 that uses 6.2 works as expected and shows no issues with using the serial port for IR while 23.10 that uses 6.5 and 24.04 that uses 6.8 does not load the ite-cir driver.

[Reproducible steps]

* Use Ubuntu 23.10 + or use Kernel 6.5+
* Use IR hardware that requires the ite-cir driver to be loaded
* Attempt to load ite-cir driver on boot of OS
* Check /var/log/kern.log for ite-cir driver failure

[Fix]

The requested solution would be to either:

* Set the serial driver to no longer consume more than 1 serial port
* Have the serial driver check what memory reference ite-cir driver needs to load, ignore that reference and then consume another point in memory instead.

We have found a workaround where we set the /etc/default/grub config
to make the 8250 use only 1 UART port, this allows ite-cir to consume
the port in memory like it use to, enabling us to use the driver again

/etc/default/grub

```
GRUB_CMDLINE_LINUX_DEFAULT="8250.nr_uarts=1"
```
The alternative fix is to have a startup script that will remove 8250 from using /dev/ttyS1 and then remove and re-add the ite-cir driver

```
#!/bin/bash

setserial /dev/ttyS1 uart none
modprobe -r ite-cir

```

[Diagnostic information]

version

```
Linux version 6.8.0-22-generic (buildd@lcy02-amd64-070) (x86_64-linux-gnu-gcc-13 (Ubuntu 13.2.0-23ubuntu3) 13.2.0, GNU ld (GNU Binutils for Ubuntu) 2.42) #22-Ubuntu SMP PREEMPT_DYNAMIC Thu Apr 4 22:30:32 UTC 2024
```

lsb_release
```
No LSB modules are available.
Description: Ubuntu Noble Numbat (development branch)
Release: 24.04
```

Kernel version

```
linux-image-6.8.0-22-generic:
  Installed: 6.8.0-22.22
  Candidate: 6.8.0-22.22
  Version table:
 *** 6.8.0-22.22 100
        100 /var/lib/dpkg/status
```

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/2063322/+subscriptions