Configuring CAS Devices

HDD-based Ceph Cluster with Open Cache Acceleration Software (Open CAS)

ID	Date	Version	Classification
778514	05/10/2023	1.0	Public

Details in Open CAS Linux Admin Guide.

On each machine, use one SSD as the cache device and three HDD as core devices, combine them into three CAS device via casadm.

Note that the term cache device refers to the SSD/NVMe device or RAM disk that is used for caching data from a slower device. While the term core device refers to the slower device to be cached.

Find out all the available disks:

lsblk -o name,type,rota,wwn,mountpoint

Given the following output as an example:

The devices with ROTA equals 1 are HDDs.

NAME                    TYPE ROTA WWN                                MOUNTPOINT
sda                     disk    0 0x55cd2e41536be6f6
├─sdb1                  part    0 0x55cd2e41536be6f6                 /boot/efi
└─sdb2                  part    0 0x55cd2e41536be6f6
  └─system--vg-root--lv lvm     0                                    /
sdb                     disk    0 0x55cd2e415378f598
sdc                     disk    0 0x55cd2e415379ee47
sdd                     disk    1 0x5000c500e01cb406
sde                     disk    1 0x5000c500df7bcba8
sdf                     disk    1 0x5000c500ed9e462e

Here we use sdb (SSD) as cache device, and sdd, sde, sdf (HDD) as core devices.

WARNING: Do not use the disk where the operating system resides!

# double check the disk ids
ls -l /dev/disk/by-id/ | grep wwn

# create a new cache device using sdb
sudo casadm -S -d /dev/disk/by-id/wwn-0x55cd2e415378f598 -c wb --force

# add core devices to the cache device and merge into a new CAS device
for id in 0x5000c500e01cb406 0x5000c500df7bcba8 0x5000c500ed9e462e; do
  sudo casadm -A -i 1 -d /dev/disk/by-id/wwn-$id
done

# create configuration file for io classification
# only cache with request_size <=128K
cat <<EOF > ioclass-config.csv
IO class id,IO class name,Eviction priority,Allocation
0,unclassified,22,0
1,request_size:le:131072,1,1
EOF

# apply optimized configuration for better performance
for cache in 1; do
  sudo casadm -C --load-config -i $cache -f ioclass-config.csv
  # clean policy alru with activity threashold to 1s
  sudo casadm -X -n cleaning-alru -i $cache -t 1000
  for core in 1 2 3; do
    # seq-cutoff always and threshold 16KB
    sudo casadm -X -n seq-cutoff -i $cache -j $core -p always -t 16
  done
done

# check the result
sudo casadm -L

You should get output like this:

type    id   disk       status    write policy   device
cache   1    /dev/sdb   Running   wb             -
├core   1    /dev/sdd   Active    -              /dev/cas1-1
├core   2    /dev/sde   Active    -              /dev/cas1-2
└core   3    /dev/sdf   Active    -              /dev/cas1-3

Three CAS devices are created: /dev/cas1-1, /dev/cas1-2, /dev/cas1-3.