Hits : 4451

Btrfs with Multiple Devices on LUKS


!! this is a mini howto – be careful !!


Below are my steps for creating an encrypted btrfs raid1 partition.


Identify the disks


# dmesg | grep disk 

sd 2:0:0:0: [sdd] 488397168 512-byte logical blocks: (250 GB/232 GiB)
sd 3:0:0:0: [sde] 488397168 512-byte logical blocks: (250 GB/232 GiB)


verify the disks by model


# find /sys/devices -type f -name model -exec cat {} \;

VB0250EAVER
VB0250EAVER


top


Create Random Encrypted keys


by running dd against urandom


dd if=/dev/urandom of=/etc/crypttab.keys/sdd bs=4096 count=1

dd if=/dev/urandom of=/etc/crypttab.keys/sde bs=4096 count=1


(folder /etc/crypttab.keys must exist – otherwise created it first)


and encrypt both disks with the above generated keys


# cryptsetup luksFormat /dev/sdd --key-file /etc/crypttab.keys/sdd

WARNING!
========
This will overwrite data on /dev/sdd irrevocably.

Are you sure? (Type uppercase yes): YES
YES


# cryptsetup luksFormat /dev/sde --key-file /etc/crypttab.keys/sde

WARNING!
========
This will overwrite data on /dev/sde irrevocably.

Are you sure? (Type uppercase yes): YES
YES


top


Verify Encrypted disks


as root you have to see two new unique identifiers for the encrypted disks:


# blkid | tail -2 

/dev/sdd: UUID="40258c46-4e6b-47b9-81fc-7247d9cb4968" TYPE="crypto_LUKS" 
/dev/sde: UUID="e2d6b4ad-9b9c-4a07-bf92-af2597bb1c4a" TYPE="crypto_LUKS"


Try opening and mapping the encrypted disk with the encrypted keys – using the identifiers:


# cryptsetup luksOpen /dev/disk/by-uuid/40258c46-4e6b-47b9-81fc-7247d9cb4968 sdd -d /etc/crypttab.keys/sdd

# cryptsetup luksOpen /dev/disk/by-uuid/e2d6b4ad-9b9c-4a07-bf92-af2597bb1c4a sde -d /etc/crypttab.keys/sde


verify the status of the mapped encrypted drives:


# cryptsetup status /dev/mapper/sdd
/dev/mapper/sdd is active.
  type:    LUKS1
  cipher:  aes-xts-plain64
  keysize: 256 bits
  device:  /dev/sdd
  offset:  4096 sectors
  size:    488393072 sectors
  mode:    read/write

# cryptsetup status /dev/mapper/sde
/dev/mapper/sde is active.
  type:    LUKS1
  cipher:  aes-xts-plain64
  keysize: 256 bits
  device:  /dev/sde
  offset:  4096 sectors
  size:    488393072 sectors
  mode:    read/write


top


Format


Now it's the time to format the new encrypted map drives under btrfs into a raid-1 setup:


# mkfs.btrfs -L VB0250EAVER -m raid1 -d raid1 /dev/mapper/sdd /dev/mapper/sde 

WARNING! - Btrfs v3.14.2 IS EXPERIMENTAL
WARNING! - see http://btrfs.wiki.kernel.org before using

Turning ON incompat feature 'extref': increased hardlink limit per file to 65536
adding device /dev/mapper/sde id 2
fs created label VB0250EAVER on /dev/mapper/sdd
	nodesize 16384 leafsize 16384 sectorsize 4096 size 465.77GiB
Btrfs v3.14.2


Verify that the encrypted map drives have created a new btrfs unique identifier:


# blkid | tail -2

/dev/mapper/sdd: LABEL="VB0250EAVER" UUID="e1408bb9-ad10-48f8-af7e-b86bf2866ef7" UUID_SUB="27fa4119-ea66-4dea-a3d7-0a43b7adc24d" TYPE="btrfs" 
/dev/mapper/sde: LABEL="VB0250EAVER" UUID="e1408bb9-ad10-48f8-af7e-b86bf2866ef7" UUID_SUB="8ea7256b-4264-4d32-811c-1e35039f006d" TYPE="btrfs"


as you can see, both encrypted map drives have the same UUID – but different subvolumes UUID.


top


Mount the new disk


Create a new directory


# mkdir -pv /mnt/VB0250EAVER


and add a new line (like the below) at the end of your fstab file:


# echo "UUID=e1408bb9-ad10-48f8-af7e-b86bf2866ef7 /mnt/VB0250EAVER auto defaults,noauto,user,exec 0 0" >> /etc/fstab


Now you can mount the encrypted btrfs raid1 new disk !


# mount /mnt/VB0250EAVER/


top


Disk Usage


at first let's find out how storage is being used:


# df -h /mnt/VB0250EAVER/
Filesystem       Size  Used Avail Use% Mounted on
/dev/mapper/sdd  466G  1,3M  464G   1% /mnt/VB0250EAVER


btrfs df shows:


# btrfs filesystem df /mnt/VB0250EAVER 
Data,	  RAID1 : total=1.00GiB, used=512.00KiB
Data,	  single: total=8.00MiB, used=0.00
System,	  RAID1 : total=8.00MiB, used=16.00KiB
System,	  single: total=4.00MiB, used=0.00
Metadata, RAID1 : total=1.00GiB, used=112.00KiB
Metadata, single: total=8.00MiB, used=0.00


and 


# btrfs filesystem show /mnt/VB0250EAVER 
Label: 'VB0250EAVER'  uuid: e1408bb9-ad10-48f8-af7e-b86bf2866ef7
	Total devices 2 FS bytes used 640.00KiB
	devid    1 size 232.88GiB used 2.03GiB path /dev/mapper/sdd
	devid    2 size 232.88GiB used 2.01GiB path /dev/dm-4

Btrfs v3.14.2


all seems nice !


Be aware on df, you will see the entire disk storage and not the raid1.
This is confusing, i know !


With btrfs you have to ignore df .


top



Make some noise


Let's create a 100Mb test file under our new encrypted btrfs raid1 disk:


# dd if=/dev/zero of=/mnt/VB0250EAVER/test bs=1024 count=102400
102400+0 records in
102400+0 records out
104857600 bytes (105 MB) copied, 0,290163 s, 361 MB/s


df will NOT show immediately the disk usage:


# df -h /mnt/VB0250EAVER/
Filesystem       Size  Used Avail Use% Mounted on
/dev/mapper/sdd  466G  1,3M  464G   1% /mnt/VB0250EAVER


Sync the disk and run it again:


# sync

# df -h /mnt/VB0250EAVER/
Filesystem       Size  Used Avail Use% Mounted on
/dev/mapper/sdd  466G  202M  464G   1% /mnt/VB0250EAVER


top


Some last info


btrfs df 


# btrfs filesystem df /mnt/VB0250EAVER
Data, 	  RAID1 : total=1.00GiB, used=100.50MiB
Data, 	  single: total=8.00MiB, used=0.00
System,   RAID1 : total=8.00MiB, used=16.00KiB
System,   single: total=4.00MiB, used=0.00
Metadata, RAID1 : total=1.00GiB, used=224.00KiB
Metadata, single: total=8.00MiB, used=0.00


btrfs show:


# btrfs filesystem show 

Label: 'VB0250EAVER'  uuid: e76cefe1-7ce3-43fa-953a-31602616d9ca
	Total devices 2 FS bytes used 100.77MiB
	devid    1 size 232.88GiB used 2.03GiB path /dev/mapper/sdd
	devid    2 size 232.88GiB used 2.01GiB path /dev/dm-4

Btrfs v3.14.2


top