DISCLAIMER: I am not a sysadmin
A couple of my DigitalOcean droplets turned five earlier this year and were
still running Ubuntu 14.04 (happy belated birthday guys!). I noticed my apt upgrades had been rather quiet lately so I thought I would check the Ubuntu
support schedule. Oops.
Ignoring the conventional advice to spin up a new droplet and migrate my
applications, I logged into my droplets, ran do-release-upgrade, ignored
another warning not to do that, and hit ‘y’ [1]. Which stands for yolo, I
think.
After a reboot I logged into my droplet and my filesystem was read-only. Google told me to remount my drive:
$ mount -o remount,rw /
mount: can't find UUID=b96601ba-7d51-4c5f-bfe2-63815708aabd
Well son of a gun.
$ cat /etc/fstab
proc /proc proc nodev,noexec,nosuid 0 0
UUID=b96601ba-7d51-4c5f-bfe2-63815708aabd / ext4 noatime,errors=remount-ro 0 1
$ blkid
/dev/vda1: LABEL="DOROOT" UUID="f5849f94-8e81-404d-9470-0de7f7cd3d41" TYPE="ext4" PTTYPE="dos" PARTLABEL="primary" PARTUUID="81702cfe-3395-47a2-b909-d06d4f08641a"
So my UUID changed, and I could not fix my fstab because my filesystem was read-only. Additional Googling was fruitless, with a couple solutions involving DigitalOcean support and recovery ISOs. No thanks. I eventually figured out that I could override the UUID when remounting:
mount UUID=$(blkid /dev/vda1 -sUUID -ovalue) -o remount,rw /
I updated the UUID in my fstab, did a test reboot, and I was back in business.
I now recall having this same issue when 16.04 came out. At the time I assumed everything was borked so I restored a snapshot. Eighteen months from now I may run into this issue again, hopefully I won’t have to Google too hard.
[1] Dear future employers: I took snapshots first. And had backups.