Hits : 5844

Btrfs with Multiple Devices on LUKS

!! this is a mini howto – be careful !!

Below are my steps for creating an encrypted btrfs^[link1] raid1 partition.

Contents

Identify the disks
Create Random Encrypted keys
Verify Encrypted disks
Format
Mount the new disk
Disk Usage
Make some noise
Some last info

Identify the disks

# dmesg | grep disk 

sd 2:0:0:0: [sdd] 488397168 512-byte logical blocks: (250 GB/232 GiB)
sd 3:0:0:0: [sde] 488397168 512-byte logical blocks: (250 GB/232 GiB)

verify the disks by model

# find /sys/devices -type f -name model -exec cat {} \;

VB0250EAVER
VB0250EAVER

top

Create Random Encrypted keys

by running dd against urandom

dd if=/dev/urandom of=/etc/crypttab.keys/sdd bs=4096 count=1

dd if=/dev/urandom of=/etc/crypttab.keys/sde bs=4096 count=1

(folder /etc/crypttab.keys must exist – otherwise created it first)

and encrypt both disks with the above generated keys

# cryptsetup luksFormat /dev/sdd --key-file /etc/crypttab.keys/sdd

WARNING!
========
This will overwrite data on /dev/sdd irrevocably.

Are you sure? (Type uppercase yes): YES
YES


# cryptsetup luksFormat /dev/sde --key-file /etc/crypttab.keys/sde

WARNING!
========
This will overwrite data on /dev/sde irrevocably.

Are you sure? (Type uppercase yes): YES
YES

top

Verify Encrypted disks

as root you have to see two new unique identifiers for the encrypted disks:

# blkid | tail -2 

/dev/sdd: UUID="40258c46-4e6b-47b9-81fc-7247d9cb4968" TYPE="crypto_LUKS" 
/dev/sde: UUID="e2d6b4ad-9b9c-4a07-bf92-af2597bb1c4a" TYPE="crypto_LUKS"

Try opening and mapping the encrypted disk with the encrypted keys – using the identifiers:

# cryptsetup luksOpen /dev/disk/by-uuid/40258c46-4e6b-47b9-81fc-7247d9cb4968 sdd -d /etc/crypttab.keys/sdd

# cryptsetup luksOpen /dev/disk/by-uuid/e2d6b4ad-9b9c-4a07-bf92-af2597bb1c4a sde -d /etc/crypttab.keys/sde

verify the status of the mapped encrypted drives:

# cryptsetup status /dev/mapper/sdd
/dev/mapper/sdd is active.
  type:    LUKS1
  cipher:  aes-xts-plain64
  keysize: 256 bits
  device:  /dev/sdd
  offset:  4096 sectors
  size:    488393072 sectors
  mode:    read/write

# cryptsetup status /dev/mapper/sde
/dev/mapper/sde is active.
  type:    LUKS1
  cipher:  aes-xts-plain64
  keysize: 256 bits
  device:  /dev/sde
  offset:  4096 sectors
  size:    488393072 sectors
  mode:    read/write

top

Format

Now it's the time to format the new encrypted map drives under btrfs into a raid-1 setup:

# mkfs.btrfs -L VB0250EAVER -m raid1 -d raid1 /dev/mapper/sdd /dev/mapper/sde 

WARNING! - Btrfs v3.14.2 IS EXPERIMENTAL
WARNING! - see http://btrfs.wiki.kernel.org before using

Turning ON incompat feature 'extref': increased hardlink limit per file to 65536
adding device /dev/mapper/sde id 2
fs created label VB0250EAVER on /dev/mapper/sdd
	nodesize 16384 leafsize 16384 sectorsize 4096 size 465.77GiB
Btrfs v3.14.2

Verify that the encrypted map drives have created a new btrfs unique identifier:

# blkid | tail -2

/dev/mapper/sdd: LABEL="VB0250EAVER" UUID="e1408bb9-ad10-48f8-af7e-b86bf2866ef7" UUID_SUB="27fa4119-ea66-4dea-a3d7-0a43b7adc24d" TYPE="btrfs" 
/dev/mapper/sde: LABEL="VB0250EAVER" UUID="e1408bb9-ad10-48f8-af7e-b86bf2866ef7" UUID_SUB="8ea7256b-4264-4d32-811c-1e35039f006d" TYPE="btrfs"

as you can see, both encrypted map drives have the same UUID – but different subvolumes UUID.

top

Mount the new disk

Create a new directory

# mkdir -pv /mnt/VB0250EAVER

and add a new line (like the below) at the end of your fstab file:

# echo "UUID=e1408bb9-ad10-48f8-af7e-b86bf2866ef7 /mnt/VB0250EAVER auto defaults,noauto,user,exec 0 0" >> /etc/fstab

Now you can mount the encrypted btrfs raid1 new disk !

# mount /mnt/VB0250EAVER/

top

Disk Usage

at first let's find out how storage is being used:

# df -h /mnt/VB0250EAVER/
Filesystem       Size  Used Avail Use% Mounted on
/dev/mapper/sdd  466G  1,3M  464G   1% /mnt/VB0250EAVER

btrfs df shows:

# btrfs filesystem df /mnt/VB0250EAVER 
Data,	  RAID1 : total=1.00GiB, used=512.00KiB
Data,	  single: total=8.00MiB, used=0.00
System,	  RAID1 : total=8.00MiB, used=16.00KiB
System,	  single: total=4.00MiB, used=0.00
Metadata, RAID1 : total=1.00GiB, used=112.00KiB
Metadata, single: total=8.00MiB, used=0.00

and

# btrfs filesystem show /mnt/VB0250EAVER 
Label: 'VB0250EAVER'  uuid: e1408bb9-ad10-48f8-af7e-b86bf2866ef7
	Total devices 2 FS bytes used 640.00KiB
	devid    1 size 232.88GiB used 2.03GiB path /dev/mapper/sdd
	devid    2 size 232.88GiB used 2.01GiB path /dev/dm-4

Btrfs v3.14.2

all seems nice !

Be aware on df, you will see the entire disk storage and not the raid1.
This is confusing, i know !

With btrfs you have to ignore df .

top

Make some noise

Let's create a 100Mb test file under our new encrypted btrfs raid1 disk:

# dd if=/dev/zero of=/mnt/VB0250EAVER/test bs=1024 count=102400
102400+0 records in
102400+0 records out
104857600 bytes (105 MB) copied, 0,290163 s, 361 MB/s

df will NOT show immediately the disk usage:

# df -h /mnt/VB0250EAVER/
Filesystem       Size  Used Avail Use% Mounted on
/dev/mapper/sdd  466G  1,3M  464G   1% /mnt/VB0250EAVER

Sync the disk and run it again:

# sync

# df -h /mnt/VB0250EAVER/
Filesystem       Size  Used Avail Use% Mounted on
/dev/mapper/sdd  466G  202M  464G   1% /mnt/VB0250EAVER

top

Some last info

btrfs df

# btrfs filesystem df /mnt/VB0250EAVER
Data, 	  RAID1 : total=1.00GiB, used=100.50MiB
Data, 	  single: total=8.00MiB, used=0.00
System,   RAID1 : total=8.00MiB, used=16.00KiB
System,   single: total=4.00MiB, used=0.00
Metadata, RAID1 : total=1.00GiB, used=224.00KiB
Metadata, single: total=8.00MiB, used=0.00

btrfs show:

# btrfs filesystem show 

Label: 'VB0250EAVER'  uuid: e76cefe1-7ce3-43fa-953a-31602616d9ca
	Total devices 2 FS bytes used 100.77MiB
	devid    1 size 232.88GiB used 2.03GiB path /dev/mapper/sdd
	devid    2 size 232.88GiB used 2.01GiB path /dev/dm-4

Btrfs v3.14.2

top

Links

^[link1] https://btrfs.wiki.kernel.org/

MyWackoSite: LUKS