luksipc is a tool to convert (unencrypted) block devices to (encrypted) LUKS devices in-place (therefore it's name LUKS in-place conversion). This means the conversion is performed without the need of copying all data somewhere, recreating the whole disk (i.e. create a LUKS device, create a new filesystem on the mapped LUKS device, copy all data back). Instead, the process is reduced to:
- Unmounting the filesystem
- Resizing the filesystem to shrink about 10 megabytes (2048 kB is the current LUKS header size -- but do not trust this value, it has changed in the past!)
- Performing luksipc
- Adding custom keys to the LUKS keyring
3 Getting started
3.1 Theory of operation
3.1.1 Rough explanation
Image that you have a non-encrypted hard disk. The file system has been created right on the device (/dev/sda6 in this case), which means it fits really snugly:
Now first in order to use luksipc you need to resize the file system itself. This is done, for example, with "resize_reiserfs" for a ReiserFS filesystem. This only decreases the amount of space the file system occupies within the partition and is usually very fast. Resizing has to be at least the size of the LUKS header. In the following case I use 10 MiB to be on the safe side:
After this, luksipc comes into play. It performs an in-place encryption of the data and prepends the partition with a LUKS header:
3.1.2 In-depth explanation
Now what's the big deal about all this, anyways? The jumping point is that the LUKS header must be prepended to the partition. This means it has to be created where part of the file system resides. All data is then offset by this constant amount in a LUKS scenario (this is about 2048 kB). Imagine you have a disk which consists of five "chunks". One chunk is a block of space which is exactly the space the LUKS header takes (and also the amount that the encrypted disk is smaller than the unencrypted disk):
Then, you will have to resize your filesystem at least one chunk (which means the last chunk will not be occupied anymore). The disk space is reduced to 4 chunks:
Then luksipc is started: What it now does is it first loads chunk 0 into memory, since this is the chunk which will be overwritten by the LUKS header in the next step:
Then, the disk is LUKS-formatted, which creates a LUKS header. The data of chunk 0 is not lost, it's still kept in memory. Creating the LUKS header creates a second, "virtual" LUKS disk (i.e. /dev/mapper/luksipc), which is exactly one chunk shorter than the original disk. Every chunk on the virtual disk maps to one chunk on the physical disk, only offset by one chunk (i.e. virtual chunk 0 maps to physical chunk 1, virtual 1 maps to physical 2 and so on). The data that can be read from the virtual device at that point is garbage, because when you read from the LUKS device at that point, dm-crypt will decrypt the unencrypted data of the physical partition.
Then, a second chunk, chunk 1, is read into memory:
This is the time when the first chunk can be written to the encrypted disk. chunk 0 is written to the virtual chunk position 0 (which maps to physical chunk 1):
Then, chunk 2 is read in again. Since we have written chunk 0 to the virtual disk, we can reuse the memory space of chunk 0, since that doesn't need to be kept in memory anymore.
Again a chunk can now be written, this time chunk 1:
And a chunk can be read again, chunk 3 (overwriting the position of chunk 1 in memory):
Now that we've read all chunks, the remaining two chunks in memory can be written to the LUKS disk. First chunk 2:
Afterwards, chunk 3 is written:
Now we're all finished and luksipc is done converting the partition to LUKS!
3.2 Aborting the process
Obviously, when you abort the process in the middle of the conversion (meaning your disk is only half-converted), this is a bad situation. While actually the disk is almost completely readable (one half on the unencrypted part, the other on the LUKS device), you will have a very hard time, since you don't know where exactly that cut is. Furthermore, one chunk needs to be kept in memory at all times. This chunk is lost. Therefore, luksipc offers a solution. Should you, for whatever reason, need to abort the process, luksipc will create a "resume.bin" file for you. In this file it will write the current block device position and one chunk of data. Later on, by just calling luksipc with the "--resume" option, the process can be resumed from that point on.
Before you attempt to do anything with the device, be warned: You may lose some of your data, even all of it (for example if you put the LUKS device key in /dev/shm and reboot your system before you add another key to the keyring). Power failures are also bad (since you will not have a resume file in that case). Furthermore, luksipc may have bugs that wreak havoc on your data. Also keep in mind that luksipc relies completely on a perfect disk. If your disk has read-problems, it likely will abort somewhere within the middle of the process (most likely also without creating a resume file). If your disk is faulty, get a new one instead of trying to crypt it. And, most importantly: Always have a backup. Now, let's be honest here: You probably don't have a backup. If you had the disk space, you wouldn't have the need to convert data in-place. Or maybe you have one and it's really old. Or the data is not really that important. In any case, please please please do not assume that everything will run smoothly. It may not. You have been warned. I will not be held responsible for any of your actions.
That said let me point out that I trusted my software (after thourough testing) enough to let it convert a 1 TB partition without having a backup. This worked nicely. However, your milage may vary.
3.4 Running it
In the package, there is a regression/ subdirectory which performs quite a lot of tests. If you're unsure about how reliable it is or how it works, play around with those first to get a feeling of how it works. If you have any special options that you're passing to luksFormat and you feel that you might run into trouble - for god's sake, please first try it on some loopback block device first to be on the safe side.
Apart from that, running it is really straightforward (here I'm testing on a 800 MB loopback device where the underlying file is on /dev/shm in case you're wondering -- a hard disk will be much slower). Do not ever forget to resize your filesystem. I'm demonstrating how this is done for ReiserFS here:
joequad [~/luksipc-0.04]: resize_reiserfs -s -10M /dev/loop0 resize_reiserfs 3.6.21 (2009 www.namesys.com) You are running BETA version of reiserfs shrinker. This version is only for testing or VERY CAREFUL use. Backup of you data is recommended. Do you want to continue? [y/N]:y Processing the tree: 0%....20%....40%....60%....80%....100% left 0, 0 /sec nodes processed (moved): int 0 (0), leaves 1 (0), unfm 0 (0), total 1 (0). check for used blocks in truncated region ReiserFS report: blocksize 4096 block count 204288 (204800) free blocks 196070 (196582) bitmap block count 7 (7) Syncing..done resize_reiserfs: Resizing finished successfully.
Afterwards, just to confirm that the device has really been resized, we can double-check:
joequad [~/luksipc-0.04]: debugreiserfs /dev/loop0 debugreiserfs 3.6.21 (2009 www.namesys.com) Filesystem state: consistent Reiserfs super block in block 16 on 0x700 of format 3.6 with standard journal Count of blocks on the device: 202240 Number of bitmaps: 7 Blocksize: 4096 [...]
This means, the filesystem occupies 202240 blocks of 4096 bytes each. This is 828375040 bytes. The whole disk is 800 MB or 800 * 1024 * 1024 bytes = 838860800. The difference of these two is 10485760, which means we gained exactly the ten megabytes that we needed. Then, you can run luksipc:
joequad [~/luksipc-0.04]: ./luksipc -d /dev/loop0 WARNING! All data on /dev/loop0 is to be LUKSified! Ensure that: 1. You have resized the contained filesystem appropriately 2. You have ensured secure storage of the keyfile 3. Power conditions are satisfied (i.e. your Laptop is not running off battery) 4. You have a backup of all data on that device /dev/loop0: 800 MB = 0.8 GB Keyfile: /root/initial_keyfile.bin LUKS format parameters: None given Are all these conditions satisfied, then answer uppercase yes: YES [I]: Size of /dev/loop0 is 838860800 bytes (800 MB + 0 bytes) [I]: Performing dm-crypt status lookup [I]: Performing luksFormat [I]: Performing luksOpen [I]: Size of cryptodisk is 837808128 bytes (798 MB + 1044480 bytes) [I]: 1052672 bytes occupied by LUKS header (1028 kB + 0 bytes) [I]: Starting copying of data... [I]: 0:00: 13.1% 105 MB / 798 MB 116.3 MB/s 693 MB left [I]: 0:00: 25.9% 207 MB / 798 MB 102.4 MB/s 591 MB left [I]: 0:00: 38.7% 309 MB / 798 MB 100.3 MB/s 489 MB left [I]: 0:00: 51.4% 411 MB / 798 MB 97.3 MB/s 387 MB left [I]: 0:00: 64.2% 513 MB / 798 MB 93.7 MB/s 285 MB left [I]: 0:00: 77.0% 615 MB / 798 MB 91.3 MB/s 183 MB left [I]: 0:00: 89.7% 717 MB / 798 MB 88.9 MB/s 81 MB left [I]: Disk copy completed successfully. [I]: Performing luksClose
luksipc will have created a key file /root/initial_keyfile.bin that you can use to gain access to the newly created LUKS device:
joequad [~/luksipc-0.04]: cryptsetup luksOpen --key-file /root/initial_keyfile.bin /dev/loop0 mydisk
One thing that you should definitely do is add the key that you want to use for your device, maybe afterwards removing the initial keyfile:
joequad [~/luksipc-0.04]: cryptsetup luksAddKey --key-file /root/initial_keyfile.bin /dev/loop0 Enter new passphrase for key slot: Verify passphrase: joequad [~/luksipc-0.04]: cryptsetup luksKillSlot /dev/loop0 0 Enter any remaining LUKS passphrase:
If you really want to get the last out of your hard disk, you can then resize it again to occupy the whole LUKS device:
joequad [~/luksipc-0.04]: resize_reiserfs /dev/mapper/mydisk resize_reiserfs 3.6.21 (2009 www.namesys.com) ReiserFS report: blocksize 4096 block count 204528 (204288) free blocks 196310 (196070) bitmap block count 7 (7) Syncing..done resize_reiserfs: Resizing finished successfully.
3.5 Testing the software
If you do not trust luksipc, you can first try it out using a loop device. For this, within the regression package there's a tool that can create pseudo-random test data incredibly fast. It's called prng and takes only one parameter: The amount of bytes that it should generate. So for generating a test device, let's use 800 Megs:
joequad [~/luksipc-0.04]: ./prng $((800*1024*1024)) >/dev/shm/disk
800 MB is 819200 kB. Remember that the LUKS header is 1028 kB, so only the first 818172 kB will be available on the final encrypted device. Let's first make a MD5 hash over the original data (only first 818172 kB obviously):
joequad [~/luksipc-0.04]: dd if=/dev/shm/disk bs=1k count=818172 | md5sum 818172+0 records in 818172+0 records out 837808128 bytes (838 MB) copied, 2.29805 s, 365 MB/s 1c603e62ea6da631bc0dc3a6ed2f61f0 -
Now let's luksipc the device, first create a loop device that maps to the file:
joequad [~/luksipc-0.04]: losetup /dev/loop0 /dev/shm/disk
Then luksipc it -- and abort right in the middle!
joequad [~/luksipc-0.04]: ./luksipc -d /dev/loop0 [...] [I]: Size of /dev/loop0 is 838860800 bytes (800 MB + 0 bytes) [I]: Performing dm-crypt status lookup [I]: Performing luksFormat [I]: Performing luksOpen [I]: Size of cryptodisk is 837808128 bytes (798 MB + 1044480 bytes) [I]: 1052672 bytes occupied by LUKS header (1028 kB + 0 bytes) [I]: Starting copying of data... [I]: 0:00: 13.1% 105 MB / 798 MB 140.1 MB/s 693 MB left [I]: 0:00: 25.9% 207 MB / 798 MB 113.7 MB/s 591 MB left ^C[C]: Shutdown requested by user interrupt, please be patient... [I]: Gracefully shutting down.
Then, resume from the resume file (resume.bin):
joequad [~/luksipc-0.04]: ./luksipc -d /dev/loop0 --resume resume.bin WARNING! Resume LUKSification of /dev/loop0 requested. 1. The resume file really belongs to the correct disk 2. Power conditions are satisfied (i.e. your Laptop is not running off battery) /dev/loop0: 800 MB = 0.8 GB Resume file: resume.bin Are all these conditions satisfied, then answer uppercase yes: YES [I]: Size of /dev/loop0 is 838860800 bytes (800 MB + 0 bytes) [I]: Starting copying of data... [I]: 0:00: 50.3% 402 MB / 798 MB 120.7 MB/s 396 MB left [I]: 0:00: 63.1% 504 MB / 798 MB 95.8 MB/s 294 MB left [I]: 0:00: 75.8% 606 MB / 798 MB 88.3 MB/s 192 MB left [I]: 0:00: 88.6% 708 MB / 798 MB 85.6 MB/s 90 MB left [I]: Disk copy completed successfully. [I]: Performing luksClose
Now let's open the device:
joequad [~/luksipc-0.04]: cryptsetup luksOpen --key-file /root/initial_keyfile.bin /dev/loop0 newdisk
And check if the MD5 sum matches:
joequad [~/luksipc-0.04]: cat /dev/mapper/newdisk | md5sum 1c603e62ea6da631bc0dc3a6ed2f61f0 -
Indeed, it does! Now let's use dd for this again to measure the crypt overhead just for fun:
joequad [~/luksipc-0.04]: dd if=/dev/mapper/newdisk bs=1k | md5sum 818172+0 records in 818172+0 records out 837808128 bytes (838 MB) copied, 9.18947 s, 91.2 MB/s 1c603e62ea6da631bc0dc3a6ed2f61f0 -
You can see that the performance has decreased a lot (91 MB/s compared to 365 MB/s -- still it is a lot more than most hard disks will be able to utilize).
4.1 Development Snapshot
The luksipc HEAD is now on GitHub. I do not use GitHub to track internal development (this I do on my private repository to hide the countless really, really embarassing development bugs), but I will keep it in sync with my private repo once I do a release. So if you have patches, you can submit pull requests via GitHub and I'll try to incorporate them there (whenever I get around to it, which sometimes is embarassingly long to be honest).
4.2 Released Versions