Open Heart Partition Surgery
Sometimes in this life, you aren't sure that something will work, but you have to try it anyway.
The other day, I ran some experiments that I would describe—in my usual overdramatic way—as "open heart partition surgery."
A word to the wise: this post is really just a braindump for me to refer back to later, and is even less coherent than usual. It is certainly not a how-to guide… but if you are looking to mutilate your hard drive, welcome!
Backstory
A couple of months ago, I took advantage of some holiday sales and finally put together a desktop PC. I knew that I wanted to run Linux, but which distro? I've been using Ubuntu on my laptop for years, and while it's treated me well, I've been longing to explore new lands and gain a hands-on appreciation for the differences between distros. What better excuse to go distro-hopping than a fresh build?
I settled on testing two distros: Pop!_OS and Arch. Pop!_OS obviously felt familiar (it's based on Ubuntu), but with a souped-up desktop environment full of keyboard shortcuts and a tiling window manager option that appealed to me. Conversely, Arch's appeal lay in its challenge: it has a bit of a reputation for being a real "hacker's" distro—no picture book illustrations in my installer, thank-you-very-much; a shell (zsh, btw) is more than enough for me. This stereotype isn't quite true—the documentation on the Arch Wiki is usually very helpful and to-the-point—but it there is a hint of verity to it.
I won't belabour my comparison of the two here, but suffice it to say that it didn't take long before Arch + KDE Plasma won me over entirely. Two months later, I was ready to thank Pop!_OS and let it go. (Although I do miss some part of its desktop environment!)
However, I was now facing a problem. When setting up my PC, I installed Pop!_OS first. Being relatively inexperienced with this sort of thing, I used my best judgement with the partitioning strategy and created the following partitions (via the Pop!_OS installer): one for boot, one for Pop!_OS itself (followed by some empty space), then a separate partition to store games (so that I only needed to download games once, while still being able to play them on either distro—this worked great!) and finally a swap partition.
Model: Samsung SSD 980 PRO 1TB (nvme)
Disk /dev/nvme0n1: 1000GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:
Number Start End Size File system Name Flags
1 1049kB 1075MB 1074MB fat32 boot, esp
4 1075MB 250GB 249GB ext4 popos
5 250GB 446GB 196GB ext4 arch
2 446GB 983GB 537GB ext4 games
3 983GB 1000GB 17.2GB linux-swap(v1) swap
(Note: I was insufficiently diligent and didn't take complete notes as I worked through this; the above is me
approximating what the actual parted
output would've looked like at the
time.)
This worked fine when I was just evaluating Pop!_OS and Arch, but now I find myself wanting some unpartitioned disk space to play around with Linux From Scratch. What I would really like to do is relocate the existing Arch partition to where the Pop!_OS partition currently resides and extend it a bit (while leaving a little space unallocated). Can we make that happen?
Thought process
Based on some previous knowledge and a bit of general research, I found myself ruminating on the following pile of thoughts:
- I don't think we can (typically) grow partitions "backwards", because important filesystem metadata lives at the beginning of the partition.
- I know each partition gets assigned a
/dev/
file, and those/dev/
files aren't regular files, they're "block devices." But since everything is a file, I should be able to clone my Arch partition by just copying the corresponding/dev/
file… right? (For some reason, this feels crazy to me, but I don't see why it shouldn't work.) - Both OS partitions use ext4; ext4 filesystems can be resized to fill a partition that has been embiggened.
- From my Arch installation experience, I happen to know I'm using systemd-boot as my boot manager; furthermore, I know
the boot entries live under
/boot/loader/entries/
, and I've monkeyed with them a bit before, so that shouldn't be an obstacle. - I don't think there's anything "position-dependent" about my current system; i.e. as long as the boot entries point
to the right places, everything should work out. (For instance, in my initial setup, I arbitrarily chose to put my
games
partition first on the disk, but I'm sure everything would've worked the same if I had placed it at the end instead.)
Chapter 14 of Michael Kerrisk's The Linux Programming Interface was quite useful for confirming some of my uncertainties here.
The plan
Taken together, those thoughts suggest a plan: presumably, we should indeed be able to just reformat the Pop!_OS partition, copy over everything from the old Arch partition, then delete the old Arch partition, and extend the new Arch partition to use some of the old partition's space. In checklist form:
- Back up everything important on the old Arch partition! I give it at least a 30% chance I end up needing to do a
fresh install, so let's not lose anything should that happen.
- Come to think of it, I should've done the same for the Pop!_OS partition too… welp, it probably didn't have anything important anyways… 😬
- Delete the Pop!_OS partition
- Create a new partition in the place of the old Pop!_OS partition
- Create a new ext4 filesystem on the new partition
- Copy everything from the old Arch partition to the new partition
- Add boot entries for the new Arch partition
- Pause: reboot; confirm that we can still boot to both the new and old Arch partitions
- Delete the old Arch partition, and extend the new one to fill (some of) that space
- Resize the filesystem on the new partition to use the newly-expanded space; update boot entries
How did it go?
Surprisingly well! I'm writing this from my "relocated" Arch partition, and as best I can tell, the procedure was a
complete success. As a general note, I used parted
for all the partition-related tasks, and performed most of these
commands from the Arch installation image (because I can't imagine messing with mounted partitions ends well). Below
are my play-by-play notes from the operation; I haven't explicitly linked them to the checklist steps above, but I hope
it's obvious enough from the context.
-
D'oh, I guess deleting the Pop!_OS partition and then creating a new one in the same place is superfluous—the partition table describes the layout of the disk; editing the partition table shouldn't affect the actual data on disk. (I thought this might create a new UUID for the partition, but as we'll see later, it appears that isn't the case.)
-
Ooh,
mkfs.ext4
has a nice safety check—trying to run it on the Pop!_OS partition gives a warning message telling me that it contains a file system, and it even gives the time and location it was last mounted! -
Okay, time to do the actual copying… I assume we could just use
cp
for this, butdd
feels cooler:dd if=/dev/nvme0n1p5 of=/dev/nvme0n1p4
. Oh, and uh,bs=1M
, because… doing fewer, larger writes will be more efficient, maybe?- I was prepared to wait a while here (as I understand it, this should involve copying ~100Gb of data?), but it finished in two minutes. NVME is fast I guess?
- Now that I think about it more, I probably didn't need to run
mkfs.ext4
before this step: that initialized an empty ext4 filesystem, but presumably we immediately overwrote it here with ourdd
command.
-
The first unexpected wrinkle has arisen: after copying, the new partition has the exact same UUID as the old partition! (I checked with
lsblk --fs
.) I assumed this would be determined from the partition table somehow, but I guess it's stored somewhere in the partition itself? Anyways, a StackOverflow post shows how to usetune2fs
anduuidgen
to create a new one.tune2fs
insists that I start by runninge2fsck
(to check the file system integrity? …yep, looks like it); thankfully,e2fsck
doesn't report any problems. -
Let's just copy the existing boot loader entries I have for Arch (which I previously copied from the Pop!_OS default…) and change the UUID to match that of the new partition.
-
Time for the first moment of truth: at this point I should be able to boot back into both the new and old Arch partitions… and indeed, I can! We just might get out of here alive after all.
-
The point of no return: old Arch partition is deleted and the new partition is grown to absorb some of its space.
resize2fs
seemingly grows the filesystem without complaint. I suppose we can axe the old bootloader entries now too. -
We only have one (bootable) partition now, does it work? Ah… no. It seems to get stuck after printing an error
WARNING: The root device is not configured to be mounted read-write!
.- It can't be too borked though, because I'm able to pull up a separate virtual console and successfully log in there.
- After reading some Arch forum
wiki
posts, there's two files
I want to play with:
/etc/fstab
and the boot loader entry.- In my boot loader entry, I currently include
ro
, which presumably means read-only—I just copied this blindly from the old Pop!_OS entry, so maybe that's not appropriate for my Arch setup - D'oh—more importantly, I forgot to update
/etc/fstab
, which is still looking for the old partition UUID. What I think is happening here is: early on while booting, the correct partition is mounted (since the boot entry has the correct UUID), but read-only (also per my boot entry config). At some point,/etc/fstab
is supposed to take over (and remount the partition?), but it's misconfigured as-is, so it fails and we get stuck with a read-only mount—it's not surprising that that would cause issues.- I'm not sure why
ro
was the default for Pop!_OS though; maybe to prevent any inadvertent modifications early in the boot process?
- I'm not sure why
- In my boot loader entry, I currently include
- Sure enough, after fixing
/etc/fstab
, I can boot normally. I still see the same warning message as before, but it disappears after an instant, and after changingro
torw
in the boot entry, I don't seem to observe it at all.
After all of that, I have a working system with the following partitions:
Model: Samsung SSD 980 PRO 1TB (nvme)
Disk /dev/nvme0n1: 1000GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:
Number Start End Size File system Name Flags
1 1049kB 1075MB 1074MB fat32 boot, esp
4 1075MB 350GB 349GB ext4 arch
2 446GB 983GB 537GB ext4 games
3 983GB 1000GB 17.2GB linux-swap(v1) swap
Huzzah!
Lessons Learned
Here's what I took away from going through this:
-
Even during my initial PC setup, trying to pick an appropriate size for the gaming partition felt like a guessing game, and resizing it now would be a bit of a pain. While researching this subject, I came across some software that might alleviate that pain, namely Logical Volume Manager (aka LVM). As I understand it, LVM basically introduces an extra layer of abstraction—I could "assign" LVM one of the partitions on my hard drive, then use LVM to create any number of "logical volumes" that act just like regular partitions. The advantage of using LVM is that since it manages those logical volumes for you, you can resize them on the fly (and let LVM handle whatever actual changes are required on disk). (LVM offers functionality far beyond this toy example, but even that alone seems like it would be quite useful for my needs.)
-
I am often loath to take notes during this kind of thing, but afterwards, I'm always glad to have done so! ("Remember, kids, the only difference between screwing around and science is writing it down!")
-
I relied on the fact that the Arch partition was already smaller than the Pop!_OS partition I wanted it to replace. If that wasn't the case, I'm not sure how I would adapt this procedure; presumably the easiest way would be to have a separate drive to use as intermediate space to facilitate copying the larger partition? (I.e., copy the larger partition to intermediate space, edit partitions on the primary drive as desired, then copy the larger partition from the intermediate space back to the main drive.)