Can't delete folder with ? in permissions

I have a mount point where a USB drive is usually mounted. The USB drive is currently disconnected. If I try to attach it, it won't mount, and a bit of investigation discovered there's nothing wrong with the USB drive, but the mount point.

alan@hal:~$ ls -l /media/alan/9098a06f-f509-452a-b2b8-3f00aee808a2 
ls: cannot access '/media/alan/9098a06f-f509-452a-b2b8-3f00aee808a2': Input/output error

alan@hal:~$ ls -l /media/alan/
ls: cannot access '/media/alan/9098a06f-f509-452a-b2b8-3f00aee808a2': Input/output error
total 0
d????????? ? ? ? ?            ? 9098a06f-f509-452a-b2b8-3f00aee808a2

alan@hal:~$ sudo rm -rf /media/alan/9098a06f-f509-452a-b2b8-3f00aee808a2 
rm: cannot remove '/media/alan/9098a06f-f509-452a-b2b8-3f00aee808a2': Is a directory

alan@hal:~$ sudo chmod 600 /media/alan/9098a06f-f509-452a-b2b8-3f00aee808a2 
chmod: cannot access '/media/alan/9098a06f-f509-452a-b2b8-3f00aee808a2': Input/output error

How can I remove this directory?

Edit: Surprisingly sudo umount /media/alan/9098a06f-f509-452a-b2b8-3f00aee808a2 removed it!? However, every time I re-attach the disk, I still get IO errors.