| SCST Performance |
| ================ |
| |
| A question that is asked often is how to tune performance. This means how to |
| improve the IOPS and/or bandwidth measured at the initiator side. In this |
| README it is explained how to optimize storage performance. |
| |
| Local storage |
| ------------- |
| Start with measuring the performance of the local block device(s). IOPS can be |
| measured e.g. as follows: |
| |
| fio --ioengine=libaio --rw=randread --ioscheduler=none --numjobs=$(nproc) \ |
| --runtime=60 --group_reporting=1 --gtod_reduce=1 --norandommap \ |
| --thread --buffered=0 --iodepth=256 --iodepth_batch=128 --bs=4k \ |
| --name=bdev --filename=/dev/... |
| |
| The bandwidth supported by a block device can be measured by increasing the |
| block size to a larger value in the above command, e.g. --bs=1M. |
| |
| Storage network |
| --------------- |
| Start with measuring the network bandwidth using your favorite tool, e.g. |
| netperf for non-RDMA networks or ib_write_bw for RDMA networks. |
| |
| Next, add a nullio LUN to SCST, e.g. by adding the following in /etc/scst.conf: |
| |
| HANDLER vdisk_nullio { |
| DEVICE disk09 { |
| blocksize 4096 |
| size_mb 256 |
| } |
| } |
| |
| TARGET_DRIVER ... { |
| TARGET ... { |
| LUN ... disk09 |
| } |
| } |
| |
| After a nullio LUN has been added, verify that this LUN is visible at the |
| initiator side. If it is not visible at the initiator side, consider |
| rescanning LUNs or disconnecting and reconnecting the initiator system to the |
| SCST server. |
| |
| Once the nullio LUN is visible at the initiator side, measure IOPS and |
| bandwidth. The bandwidth should be close to the network bandwidth. When using |
| the Linux iSCSI initiator this may require configuring multiple iSCSI sessions |
| and activating multipathd on top of the multiple iSCSI sessions. The sequence |
| for logging in with iSCSI and activating multiple iSCSI sessions is as follows: |
| |
| iscsiadm -m iface -I iface2 -o new |
| iscsiadm -m iface -I iface2 -o update -n iface.initiatorname -v ${iqn2} |
| iscsiadm -m discovery -t st -p ${scst_ip_address} |
| iscsiadm -m discovery -t st -p ${scst_ip_address} -I iface2 |
| iscsiadm -m node -p ${scst_ip_address} -l |
| iscsiadm -m node -p ${scst_ip_address} -I iface2 -l |
| |
| SCST Configuration |
| ------------------ |
| If the number of IOPS measured at the initiator side is significantly lower |
| than the minimum of the IOPS supported by the local storage and the storage |
| network, further tuning is required. Look up in /proc/interrupts which CPU |
| cores process the most network and storage interrupts and configure the SCST |
| kernel threads such that these run on other CPU cores than those that process |
| the most interrupts by configuring the cpu_mask attribute. More information |
| about the SCST cpu_mask sysfs attribute is available in the SCST README. |