Mouse pointer freezes frequently - possibly caused by ACPI. 20.04.4

I have a rather unusual problem that in the recent month or so, my mouse pointer has started to freeze completely several times a day. I believe ACPI might be involved as the usual, slight white noise coming from my headphones disappears concurrently when it happens. However, the audio works just fine but I have to unplug the mouse (USB) and plug it back in so it works once more. Keyboard is unaffected.

Now, I don't want to go all "acpi=off" here because fan control is important. But I remember from years back when I had a sound problem (crackling) that was caused by ACPI, where I could selectively shut off ACPI just for the audio system. Is there a way to do that for the mouse as well?

BTW, this is a desktop PC, no touchpad or other competing devices. Googling the problem results in tons of people asking how to prevent their hibernating system from waking up when the mouse is nudged, so kinda the opposite of that I want here...

Cheers!