| SCST Local ... | |
| Richard Sharpe, 30-Nov-2008 | |
| This is the SCST Local driver. Its function is to allow you to access devices | |
| that are exported via SCST directly on the same Linux system that they are | |
| exported from. | |
| No assumptions are made in the code about the device types on the target, so | |
| any device handlers that you load in SCST should be visible, including tapes | |
| and so forth. | |
| You can freely use any sg, sd, st, etc. devices imported from target, | |
| except the following: you can't mount file systems or put swap on them | |
| for all dev handlers, except BLOCKIO and pass-through, because it can | |
| lead to recursive memory allocation deadlock. This is a limitation of | |
| Linux memory/cache manager. See SCST README file for details. For | |
| BLOCKIO and pass-through dev handlers there's no such limitation, so you | |
| can freely mount file systems over them. | |
| To build, simply issue 'make' in the scst_local directory. | |
| Try 'modinfo scst_local' for a listing of module parameters so far. | |
| Here is how I have used it so far: | |
| 1. Load up scst: | |
| modprobe scst | |
| modprobe scst_vdisk | |
| 2. Create a virtual disk (or your own device handler): | |
| dd if=/dev/zero of=/some/path/vdisk1.img bs=16384 count=1000000 | |
| echo "add_device vm_disk1 filename=/some/path/vdisk1.img" >/sys/kernel/scst_tgt/handlers/vdisk_fileio/mgmt | |
| 3. Load the scst_local driver: | |
| insmod scst_local | |
| echo "add vm_disk1 0" >/sys/kernel/scst_tgt/targets/scst_local/scst_local_tgt/luns/mgmt | |
| 4. Check what you have | |
| cat /proc/scsi/scsi | |
| Attached devices: | |
| Host: scsi0 Channel: 00 Id: 00 Lun: 00 | |
| Vendor: ATA Model: ST9320320AS Rev: 0303 | |
| Type: Direct-Access ANSI SCSI revision: 05 | |
| Host: scsi4 Channel: 00 Id: 00 Lun: 00 | |
| Vendor: TSSTcorp Model: CD/DVDW TS-L632D Rev: TO04 | |
| Type: CD-ROM ANSI SCSI revision: 05 | |
| Host: scsi7 Channel: 00 Id: 00 Lun: 00 | |
| Vendor: SCST_FIO Model: vm_disk1 Rev: 200 | |
| Type: Direct-Access ANSI SCSI revision: 04 | |
| Or instead of manually "add_device" in (2) and step (3) write a | |
| scstadmin config: | |
| HANDLER vdisk_fileio { | |
| DEVICE vm_disk1 { | |
| filename /some/path/vdisk1.img | |
| } | |
| } | |
| TARGET_DRIVER scst_local { | |
| TARGET scst_local_tgt { | |
| LUN 0 vm_disk1 | |
| } | |
| } | |
| then: | |
| insmod scst_local | |
| scstadmin -config conf_file.cfg | |
| More advanced examples: | |
| For (3) you can: | |
| insmod scst_local add_default_tgt=0 | |
| echo "add_target scst_local_tgt session_name=scst_local_host" >/sys/kernel/scst_tgt/targets/scst_local//mgmt | |
| echo "add vm_disk1 0" >/sys/kernel/scst_tgt/targets/scst_local/scst_local_tgt/luns/mgmt | |
| Scst_local module's parameter add_default_tgt disables creation of | |
| default target "scst_local_tgt" and session "scst_local_host", so you | |
| needed to create it manually. | |
| There can be any number of targets and sessions created. Each SCST | |
| session corresponds to SCSI host. You can change which LUNs assigned to | |
| each session by using SCST access control. This mode is intended for | |
| user space target drivers (see below). | |
| Alternatively, you can write an scstadmin's config file conf_file.cfg: | |
| HANDLER vdisk_fileio { | |
| DEVICE vm_disk1 { | |
| filename /some/path/vdisk1.img | |
| } | |
| } | |
| TARGET_DRIVER scst_local { | |
| TARGET scst_local_tgt { | |
| session_name scst_local_host | |
| LUN 0 vm_disk1 | |
| } | |
| } | |
| then: | |
| insmod scst_local add_default_tgt=0 | |
| scstadmin -config conf_file.cfg | |
| NOTE! Although scstadmin allows to create scst_local's sessions using | |
| "session_name" expression, it doesn't save existing sessions during | |
| writing config file by "write_config" command. If you need this | |
| functionality, feel free to send a request for it in SCST development | |
| mailing list. | |
| 5. Have fun. | |
| Some of this was coded while in Santa Clara, some in Bangalore, and some in | |
| Hyderabad. Noe doubt some will be coded on the way back to Santa Clara. | |
| The code still has bugs, so if you encounter any, email me the fixes at: | |
| realrichardsharpe@gmail.com | |
| I am thinking of renaming this to something more interesting. | |
| Sysfs interface | |
| =============== | |
| See SCST's README for a common SCST sysfs description. | |
| Root of this driver is /sys/kernel/scst_tgt/targets/scst_local. It has | |
| the following additional entry: | |
| - stats - read-only attribute with some statistical information. | |
| Each target subdirectory contains the following additional entries: | |
| - phys_transport_version - contains and allows to change physical | |
| transport version descriptor. It determines by which physical | |
| interface this target will look like. See SPC for more details. By | |
| default, it is not defined (0). | |
| - scsi_transport_version - contains and allows to change SCSI | |
| transport version descriptor. It determines by which SCSI | |
| transport this target will look like. See SPC for more details. By | |
| default, it is SAS. | |
| Each session subdirectory contains the following additional entries: | |
| - transport_id - contains this host's TransportID. This TransportID | |
| used to identify initiator in Persisten Reservation commands. If you | |
| change scsi_transport_version for a target, make sure you set for all | |
| its sessions correct TransportID. See SPC for more details. | |
| - host - links to the corresponding SCSI host. Using it you can find | |
| local sg/bsg/sd/etc. devices of this session. For instance, this | |
| links points out to host12, so you can find your sg devices by: | |
| $ lsscsi -g|grep "\[12:" | |
| [12:0:0:0] disk SCST_FIO rd1 200 /dev/sdc /dev/sg2 | |
| [12:0:0:1] disk SCST_FIO nullio 200 /dev/sdd /dev/sg3 | |
| They are /dev/sg2 and /dev/sg3. | |
| The following management commands available via /sys/kernel/scst_tgt/targets/scst_local/mgmt: | |
| - add_target target_name [session_name=sess_name; [session_name=sess_name1;] [...]] - | |
| creates a target with optionally one or more sessions. | |
| - del_target target_name - deletes a target. | |
| - add_session target_name session_name - adds to target target_name | |
| session (SCSI host) with name session_name. | |
| - del_session target_name session_name - deletes session session_name | |
| from target target_name. | |
| Note on performance | |
| =================== | |
| Although this driver implemented in the most performance effective way, | |
| including zero-copy passing data between SCSI/block subsystems and SCST, | |
| in many cases it is NOT suited to measure performance as a NULL link. | |
| For example, it is not suited for max IOPS measurements. This is because | |
| for such cases not performance of the link between the target and | |
| initiator is the bottleneck, but CPU or memory speed on the target or | |
| initiator. For scst_local you have both initiator and target on the same | |
| system, which means each your initiator and target are much less | |
| CPU/memory powerful. | |
| User space target drivers | |
| ========================= | |
| Scst_local can be used to write full featured SCST target drivers in | |
| user space: | |
| 1. For each SCSI target a user space target driver should create an | |
| scst_local's target using "add_target" command. | |
| 2. Then the user space target driver should, if needed, set its SCSI and | |
| physical transport version descriptors using attributes | |
| scsi_transport_version and phys_transport_version correspondingly in | |
| /sys/kernel/scst_tgt/targets/scst_local/target_name directory. | |
| 3. For incoming session (I_T nexus) from an initiator the user space | |
| target driver should create scst_local's session using "add_session" | |
| command. | |
| 4. Then, if needed, the user space target driver should set TransportID | |
| for this session (I_T nexus) using attribute | |
| /sys/kernel/scst_tgt/targets/scst_local/target_name/sessions/session_name/transport_id | |
| 5. Then the user space target driver should find out sg/bsg devices for | |
| the LUNs the created session has using link | |
| /sys/kernel/scst_tgt/targets/scst_local/target_name/sessions/session_name/host | |
| as described above. | |
| 6. Then the user space target driver can start serving the initiator using | |
| found sg/bsg devices. | |
| For other connected initiators steps 3-6 should be repeated. | |
| Change log | |
| ========== | |
| V0.1 24-Sep-2008 (Hyderabad) Initial coding, pretty chatty and messy, | |
| but worked. | |
| V0.2 25-Sep-2008 (Hong Kong) Cleaned up the code a lot, reduced the log | |
| chatter, fixed a bug where multiple LUNs did not | |
| work. Also, added logging control. Tested with | |
| five virtual disks. They all came up as /dev/sdb | |
| through /dev/sdf and I could dd to them. Also | |
| fixed a bug preventing multiple adapters. | |
| V0.3 26-Sep-2008 (Santa Clara) Added back a copyright plus cleaned up some | |
| unused functions and structures. | |
| V0.4 5-Oct-2008 (Santa Clara) Changed name to scst_local as suggested, cleaned | |
| up some unused variables (made them used) and | |
| change allocation to a kmem_cache pool. | |
| V0.5 5-Oct-2008 (Santa Clara) Added mgmt commands to handle dev reset and | |
| aborts. Not sure if aborts works. Also corrected | |
| the version info and renamed readme to README. | |
| V0.6 7-Oct-2008 (Santa Clara) Removed some redundant code and made some | |
| changes suggested by Vladislav. | |
| V0.7 11-Oct-2008 (Santa Clara) Moved into the scst tree. Cleaned up some | |
| unused functions, used TRACE macros etc. | |
| V0.9 30-Nov-2008 (Mtn View) Cleaned up an additional problem with symbols not | |
| being defined in older version of the kernel. Also | |
| fixed some English and cleaned up this doc. | |
| V1.0 10-Sep-2010 (Moscow) Sysfs management added. Reviewed and cleaned up. | |
| V2.1 Update for kernels up to 3.0. Cleanups. | |