Target driver for QLogic 2[2-6]xx/8[1-3]xx Fibre Channel cards
==============================================================

Version 3.5.0, 21 December 2020
----------------------------

This is target driver for QLogic 2[2-6]xx/8[1-3]xx Fibre Channel cards.
Particularly, this driver supports 16G Hilda QLogic chip based adapters
(however, post-Hilda QLogic chips not supported). This driver should
also support FCoE, but that has never been verified. It has passed
intensive internal SanDisk tests. It is stable and production ready.
This driver is in stable maintenance mode in favor of the QLogic git
driver, which has support for all the latest QLogic adapters.

This driver consists from two parts: the target mode driver itself and
the changed initiator driver from Linux kernel, which is, particularly,
intended to perform all the initialization and shutdown tasks. The
initiator driver was changed to provide the target mode support and all
necessary callbacks, but it's still capable to work as initiator only.
Mode, when a host acts as the initiator and the target simultaneously,
is supported as well.

This version is compatible with SCST core version 3.2.0 and higher and
Linux kernel 2.6.26 and higher. Sorry, kernels below 2.6.26 are not
supported, because it's too hard to backport used initiator driver to
older kernels.

See also "ToDo" file for list of known issues and unimplemented
features.


Installation
------------

Only vanilla kernels from kernel.org and RHEL/CentOS 5.2 kernels are
supported, but SCST should work on other (vendors') kernels, if you
manage to successfully compile it on them. The main problem with
vendors' kernels is that they often contain patches, which will appear
only in the next version of the vanilla kernel, therefore it's quite
hard to track such changes. Thus, if during compilation for some vendor
kernel your compiler complains about redefinition of some symbol, you
should either switch to vanilla kernel, or add or change as necessary
the corresponding to that symbol "#if LINUX_VERSION_CODE" statement.

Before installation make sure that the link
"/lib/modules/`you_kernel_version`/build" points to the source code for
your currently running kernel.

If your kernel version is <2.6.28, then you should consider applying
kernel patch scst_fc_vport_create.patch from the "kernel" subdirectory.
Without it, creating and removing NPIV targets using SCST sysfs
interface will be disabled. NOTE: you will still be able to create and
remove NPIV targets using the standard Linux interface (i.e. echoing
wwpn:wwnn into /sys/class/fc_host/hostX/vport_create and
/sys/class/fc_host/hostX/vport_delete).

Then you should replace (or link) by the initiator driver from this
package "qla2xxx" subdirectory in kernel_source/drivers/scsi/ of the
currently running kernel and using your favorite kernel configuration
tool enable in the QLogic QLA2XXX Fibre Channel driver target mode
support (CONFIG_SCSI_QLA2XXX_TARGET). Then rebuild the kernel and its
modules. During this step you will compile the initiator driver. To
install it, install the built kernel and its modules.

Then edit qla2x00-target/Makefile and set SCST_INC_DIR variable to point
to the directory, where SCST's public include files are located. If you
install QLA2x00 target driver's source code in the SCST's directory,
then SCST_INC_DIR will be set correctly for you.

Also you can set SCST_DIR variable to the directory, where SCST was
built, but this is optional. If you don't set it or set incorrectly,
during the compilation you will get a bunch of harmless warnings like
"WARNING: "scst_rx_data" [/XXX/qla2x00tgt.ko] undefined!"

To compile the target driver, type 'make' in qla2x00-target/
subdirectory. It will build qla2x00tgt.ko module.

To install the target driver, type 'make install' in qla2x00-target/
subdirectory. The target driver will be installed in
/lib/modules/`you_kernel_version`/extra. To uninstall it, type 'make
uninstall'.


Usage
-----

After the drivers are loaded and adapters successfully initialized by
the initiator driver, including firmware image load, you should
configure exported devices using the corresponding interface of SCST
core. It is highly recommended to use scstadmin utility for that
purpose.

Then target mode should be enabled via a sysfs interface on a per card
basis, like:

echo "1" >/sys/kernel/scst_tgt/targets/qla2x00t/target/enabled

See below for full description of the driver's sysfs interface.

With the obsolete proc interface you should instead use
target_mode_enabled under the appropriate scsi_host entry, like:

echo "1" >/sys/class/scsi_host/host0/target_mode_enabled

You can find some installation and configuration HOWTOs in
http://scst.sourceforge.net/qla2x00t-howto.html and
https://forums.openfiler.com/viewtopic.php?id=3422.


IMPORTANT USAGE NOTES
---------------------

1. It is strongly recommended to use firmware version 5.x or higher for
24xx/25xx adapters. See
http://sourceforge.net/mailarchive/forum.php?thread_name=4B4CD39F.6020401%40vlnb.net&forum_name=scst-devel
for more details why. Recent report also suggest that openSUSE ships FW
8.6.0 that leads to severe "scst: TM fn ABORT_TASK/0" errors on the
target side, but using the latest 8.7.0 firmware from QLogic works well.

2. If you reload qla2x00tgt module, you should also reload qla2xxx
module, otherwise your initiators could not see the target, when it is
enabled after qla2x00tgt module load.

3. You need to issue LIP after you enabled a target, if you enabled it
after one or more its initiators already started.


Initiator and target modes
--------------------------

When qla2xxx compiled with CONFIG_SCSI_QLA2XXX_TARGET enabled, it has
parameter "qlini_mode", which determines when initiator mode will be
enabled. Possible values:

 - "exclusive" (default) - initiator mode will be enabled on load,
disabled on enabling target mode and then on disabling target mode
enabled back.

 - "disabled" - initiator mode will never be enabled.

 - "enabled" - initiator mode will always stay enabled.

Usage of mode "disabled" is recommended, if you have incorrectly
functioning your target's initiators, which if once seen a port in
initiator mode, later refuse to see it as a target.

Use mode "enabled" if you need your QLA adapters to work in both
initiator and target modes at the same time.

You can always see which modes are currently active in active_mode sysfs
attribute.

In all the modes you can at any time use sysfs attribute
ini_mode_force_reverse to force enable or disable initiator mode on any
particular port. Setting this attribute to 1 will reverse current status
of the initiator mode from enabled to disabled and vice versa.


Explicit confirmation
---------------------

This option should (actually, almost always must) be enabled by echoing
"1" in /sys/kernel/scst_tgt/targets/qla2x00t/target/host/explicit_conform_enabled,
if a target card exports at least one stateful SCSI device, like tape,
and class 2 isn't used, otherwise link-level errors could lead to loss
of the target/initiator state synchronization. Also check if initiator
supports this feature, it is reported in the kernel logs ("confirmed
completion supported" or not). No major performance degradation was
noticed, if it is enabled. Supported only for 23xx+. Disabled by
default.

For NPIV targets this option is set on their parent targets and shared
among all NPIV targets on this hardware target.


Class 2
-------

Class 2 is the close equivalent of TCP in the network world. If you
enable it, all the Fibre Channel packets will be acknowledged. By
default, class 3 is used, which is UDP-like. Enable class 2 by echoing
"1" in /sys/kernel/scst_tgt/targets/qla2x00t/target/host/class2_enabled.
This option needs a special firmware with class 2 support. Disabled by
default.


N_Port ID Virtualization
------------------------

Unfortunately, due to severe problems in the original qla2xxx driver,
NPIV in this version is not supported. If you need NPIV, you can use
previous version 2.1 of this driver.


N_Port ID Virtualization (NPIV) is a Fibre Channel facility allowing
multiple N_Port IDs to share a single physical N_Port. NPIV is fully
supported by this driver. You must have 24xx+ ISPs with NPIV-supporting
and NPIV-switches switch(es) to use this facility.

You can add NPIV targets by echoing:

add_target target_name node_name=node_name_value; parent_host=parent_host_value

in /sys/kernel/scst_tgt/targets/qla2x00t/mgmt.

Removing NPIV targets is done by echoing:

del_target target_name

in/sys/kernel/scst_tgt/targets/qla2x00t/mgmt.

Also, you can create and remove NPIV targets using the standard Linux
interface (i.e. echoing wwpn:wwnn into /sys/class/fc_host/hostX/vport_create
and /sys/class/fc_host/hostX/vport_delete).

It is recommended to use scstadmin utility and its config file to
configure virtual NPIV targets instead of the above direct interface.


Compilation options
-------------------

There are the following compilation options, that could be commented
in/out in Makefile:

 - CONFIG_SCST_DEBUG - turns on some debugging code, including some logging.
   Makes the driver considerably bigger and slower, producing large amount of
   log data.

 - CONFIG_SCST_TRACING - turns on ability to log events. Makes the driver
   considerably bigger and leads to some performance loss.

 - CONFIG_QLA_TGT_DEBUG_WORK_IN_THREAD - makes SCST process incoming
   commands from the qla2x00t target driver and call the driver's
   callbacks in internal SCST threads context instead of SIRQ context,
   where those commands were received. Useful for debugging and lead to
   some performance loss.

 - CONFIG_QLA_TGT_DEBUG_SRR - turns on retransmitting packets (SRR)
   debugging. In this mode some CTIOs will be "broken" to force the
   initiator to issue a retransmit request.


Sysfs interface
---------------

Starting from 2.0.0 this driver has sysfs interface.

Root of SCST sysfs interface is /sys/kernel/scst_tgt. Root of this
driver is /sys/kernel/scst_tgt/targets/qla2x00t. It has the following
entries:

 - None, one or more subdirectories for targets with name equal to port
   names of the corresponding targets.

 - trace_level - allows to enable and disable various tracing
   facilities. See content of this file for help how to use it.

 - version - read-only attribute, which allows to see version of
   this driver and enabled optional features.

 - mgmt - main management entry, which allows to configure NPIV targets.
   See content of this file for help how to use it.

 - hw_target (hardware target only) - read-only attribute with value 1.
   It allows to distinguish hardware and virtual targets.

Each target subdirectory contains the following entries:

 - host - link pointing on the corresponding scsi_host of the initiator
   driver

 - ini_groups - subdirectory defining initiator groups for this target,
   used to define per-initiator access control. See SCST core README for
   more details.

 - luns - subdirectory defining LUNs of this target. See SCST core
   README for more details.

 - sessions - subdirectory containing connected to this target sessions.

 - enabled - using this attribute you can enable or disable target mode
   of this FC port. It allows to finish configuring it before it starts
   accepting new connections. 0 by default.

 - explicit_confirmation - allows to enable explicit confirmations, see
   above.

 - rel_tgt_id - allows to read or write SCSI Relative Target Port
   Identifier attribute. This identifier is used to identify SCSI Target
   Ports by some SCSI commands, mainly by Persistent Reservations
   commands. This identifier must be unique among all SCST targets, but
   for convenience SCST allows disabled targets to have not unique
   rel_tgt_id. In this case SCST will not allow to enable this target
   until rel_tgt_id becomes unique. This attribute initialized unique by
   SCST by default.

 - node_name (NPIV targets only) - read-only attribute, which allows to see
   the target World Wide Node Name.

 - parent_host (NPIV target only) - read-only attribute, which allows to see
   the parent HBA World Wide Port Name (WWPN).

Subdirectory "sessions" contains one subdirectory for each connected
session with name equal to port name of the connected initiator.

Each session subdirectory contains the following entries:

 - initiator_name - contains initiator's port name

 - active_commands - contains number of active, i.e. not yet or being
   executed, SCSI commands in this session.

 - commands - contains overall number of SCSI commands in this session.

Below is a sample script, which configures 2 virtual disk "disk1" using
/disk1 image for usage with 25:00:00:f0:98:87:92:f3 hardware target, and
"disk2" using /disk2 image for usage with 50:50:00:00:00:00:00:11 NPIV
target. All initiators connected to this targets will see those devices.

#!/bin/bash

modprobe scst
modprobe scst_vdisk

echo "add_device disk1 filename=/disk1; nv_cache=1" >/sys/kernel/scst_tgt/handlers/vdisk_fileio/mgmt
echo "add_device disk2 filename=/disk2; nv_cache=1" >/sys/kernel/scst_tgt/handlers/vdisk_fileio/mgmt

modprobe qla2x00tgt

echo "add_target 50:50:00:00:00:00:00:11 node_name=50:50:00:00:00:00:00:00;parent_host=25:00:00:f0:98:87:92:f3" >\
/sys/kernel/scst_tgt/targets/qla2x00t/mgmt

echo "add disk1 0" >/sys/kernel/scst_tgt/targets/qla2x00t/25:00:00:f0:98:87:92:f3/luns/mgmt
echo "add disk2 0" >/sys/kernel/scst_tgt/targets/qla2x00t/50:50:00:00:00:00:00:11/luns/mgmt
echo 1 >/sys/kernel/scst_tgt/targets/qla2x00t/25:00:00:f0:98:87:92:f3/enabled
echo 1 >/sys/kernel/scst_tgt/targets/qla2x00t/50:50:00:00:00:00:00:11/enabled

Below is another sample script, which configures 1 real local SCSI disk
0:0:1:0 for usage with 25:00:00:f0:98:87:92:f3 target:

#!/bin/bash

modprobe scst
modprobe scst_disk

echo "add_device 0:0:1:0" >/sys/kernel/scst_tgt/handlers/dev_disk/mgmt

modprobe qla2x00tgt

echo "add 0:0:1:0 0" >/sys/kernel/scst_tgt/targets/qla2x00t/25:00:00:f0:98:87:92:f3/luns/mgmt
echo 1 >/sys/kernel/scst_tgt/targets/qla2x00t/25:00:00:f0:98:87:92:f3/enabled

Below is an advanced sample script, which configures more virtual
devices of various types, including virtual CDROM. In this script
initiator 25:00:00:f0:99:87:94:a3 will see disk1 and disk2 devices, all
other initiators will see read only blockio, nullio and cdrom devices.

#!/bin/bash

modprobe scst
modprobe scst_vdisk

echo "add_device disk1 filename=/disk1; nv_cache=1" >/sys/kernel/scst_tgt/handlers/vdisk_fileio/mgmt
echo "add_device disk2 filename=/disk2; blocksize=4096; nv_cache=1" >/sys/kernel/scst_tgt/handlers/vdisk_fileio/mgmt
echo "add_device blockio filename=/dev/sda5" >/sys/kernel/scst_tgt/handlers/vdisk_blockio/mgmt
echo "add_device nullio" >/sys/kernel/scst_tgt/handlers/vdisk_nullio/mgmt
echo "add_device cdrom" >/sys/kernel/scst_tgt/handlers/vcdrom/mgmt

modprobe qla2x00tgt

echo "add blockio 0 read_only=1" >/sys/kernel/scst_tgt/targets/qla2x00t/25:00:00:f0:98:87:92:f3/luns/mgmt
echo "add nullio 1" >/sys/kernel/scst_tgt/targets/qla2x00t/25:00:00:f0:98:87:92:f3/luns/mgmt
echo "add cdrom 2" >/sys/kernel/scst_tgt/targets/qla2x00t/25:00:00:f0:98:87:92:f3/luns/mgmt

echo "create 25:00:00:f0:99:87:94:a3" >/sys/kernel/scst_tgt/targets/qla2x00t/25:00:00:f0:98:87:92:f3/ini_groups/mgmt
echo "add disk1 0" >/sys/kernel/scst_tgt/targets/qla2x00t/25:00:00:f0:98:87:92:f3/ini_groups/25:00:00:f0:99:87:94:a3/luns/mgmt
echo "add disk2 1" >/sys/kernel/scst_tgt/targets/qla2x00t/25:00:00:f0:98:87:92:f3/ini_groups/25:00:00:f0:99:87:94:a3/luns/mgmt
echo "add 25:00:00:f0:99:87:94:a3" >/sys/kernel/scst_tgt/targets/qla2x00t/25:00:00:f0:98:87:92:f3/ini_groups/25:00:00:f0:99:87:94:a3/initiators/mgmt

echo 1 >/sys/kernel/scst_tgt/targets/qla2x00t/25:00:00:f0:98:87:92:f3/enabled

The resulting overall SCST sysfs hierarchy with initiator
25:00:00:f0:99:87:94:a3 connected will look like:

/sys/kernel/scst_tgt
|-- devices
|   |-- blockio
|   |   |-- blocksize
|   |   |-- exported
|   |   |   `-- export0 -> ../../../targets/qla2x00t/25:00:00:f0:98:87:92:f3/luns/0
|   |   |-- filename
|   |   |-- handler -> ../../handlers/vdisk_blockio
|   |   |-- nv_cache
|   |   |-- read_only
|   |   |-- removable
|   |   |-- resync_size
|   |   |-- size_mb
|   |   |-- t10_dev_id
|   |   |-- threads_num
|   |   |-- threads_pool_type
|   |   |-- type
|   |   `-- usn
|   |-- cdrom
|   |   |-- exported
|   |   |   `-- export0 -> ../../../targets/qla2x00t/25:00:00:f0:98:87:92:f3/luns/2
|   |   |-- filename
|   |   |-- handler -> ../../handlers/vcdrom
|   |   |-- size_mb
|   |   |-- t10_dev_id
|   |   |-- threads_num
|   |   |-- threads_pool_type
|   |   |-- type
|   |   `-- usn
|   |-- disk1
|   |   |-- blocksize
|   |   |-- exported
|   |   |   `-- export0 -> ../../../targets/qla2x00t/25:00:00:f0:98:87:92:f3/ini_groups/25:00:00:f0:99:87:94:a3/luns/0
|   |   |-- filename
|   |   |-- handler -> ../../handlers/vdisk_fileio
|   |   |-- nv_cache
|   |   |-- o_direct
|   |   |-- read_only
|   |   |-- removable
|   |   |-- resync_size
|   |   |-- size_mb
|   |   |-- t10_dev_id
|   |   |-- threads_num
|   |   |-- threads_pool_type
|   |   |-- type
|   |   |-- usn
|   |   `-- write_through
|   |-- disk2
|   |   |-- blocksize
|   |   |-- exported
|   |   |   `-- export0 -> ../../../targets/qla2x00t/25:00:00:f0:98:87:92:f3/ini_groups/25:00:00:f0:99:87:94:a3/luns/1
|   |   |-- filename
|   |   |-- handler -> ../../handlers/vdisk_fileio
|   |   |-- nv_cache
|   |   |-- o_direct
|   |   |-- read_only
|   |   |-- removable
|   |   |-- resync_size
|   |   |-- size_mb
|   |   |-- t10_dev_id
|   |   |-- threads_num
|   |   |-- threads_pool_type
|   |   |-- type
|   |   |-- usn
|   |   `-- write_through
|   `-- nullio
|       |-- blocksize
|       |-- exported
|       |   `-- export0 -> ../../../targets/qla2x00t/25:00:00:f0:98:87:92:f3/luns/1
|       |-- handler -> ../../handlers/vdisk_nullio
|       |-- read_only
|       |-- removable
|       |-- size_mb
|       |-- t10_dev_id
|       |-- threads_num
|       |-- threads_pool_type
|       |-- type
|       `-- usn
|-- handlers
|   |-- vcdrom
|   |   |-- cdrom -> ../../devices/cdrom
|   |   |-- mgmt
|   |   |-- trace_level
|   |   `-- type
|   |-- vdisk_blockio
|   |   |-- blockio -> ../../devices/blockio
|   |   |-- mgmt
|   |   |-- trace_level
|   |   `-- type
|   |-- vdisk_fileio
|   |   |-- disk1 -> ../../devices/disk1
|   |   |-- disk2 -> ../../devices/disk2
|   |   |-- mgmt
|   |   |-- trace_level
|   |   `-- type
|   `-- vdisk_nullio
|       |-- mgmt
|       |-- nullio -> ../../devices/nullio
|       |-- trace_level
|       `-- type
|-- sgv
|   |-- global_stats
|   |-- sgv
|   |   `-- stats
|   |-- sgv-clust
|   |   `-- stats
|   `-- sgv-dma
|       `-- stats
|-- targets
|   `-- qla2x00t
|       |-- 25:00:00:f0:98:87:92:f3
|       |   |-- enabled
|       |   |-- explicit_confirmation
|       |   |-- host -> ../../../../../class/scsi_host/host4
|       |   |-- ini_groups
|       |   |   |-- 25:00:00:f0:99:87:94:a3
|       |   |   |   |-- initiators
|       |   |   |   |   |-- 25:00:00:f0:99:87:94:a3
|       |   |   |   |   `-- mgmt
|       |   |   |   `-- luns
|       |   |   |       |-- 0
|       |   |   |       |   |-- device -> ../../../../../../../devices/disk1
|       |   |   |       |   `-- read_only
|       |   |   |       |-- 1
|       |   |   |       |   |-- device -> ../../../../../../../devices/disk2
|       |   |   |       |   `-- read_only
|       |   |   |       `-- mgmt
|       |   |   `-- mgmt
|       |   |-- luns
|       |   |   |-- 0
|       |   |   |   |-- device -> ../../../../../devices/blockio
|       |   |   |   `-- read_only
|       |   |   |-- 1
|       |   |   |   |-- device -> ../../../../../devices/nullio
|       |   |   |   `-- read_only
|       |   |   |-- 2
|       |   |   |   |-- device -> ../../../../../devices/cdrom
|       |   |   |   `-- read_only
|       |   |   `-- mgmt
|       |   |-- rel_tgt_id
|	|   |-- hw_target
|       |   `-- sessions
|       |       `-- 25:00:00:f0:99:87:94:a3
|       |           |-- active_commands
|       |           |-- commands
|       |           |-- initiator_name
|       |           `-- luns -> ../../ini_groups/25:00:00:f0:99:87:94:a3/luns
|       |-- trace_level
|       |-- version
|	`-- mgmt
|-- threads
|-- trace_level
`-- version


Performance advices
-------------------

1. If you are going to use your target in an VM environment, for
instance as a shared storage with VMware, make sure all your VMs
connected to the target via *separate* sessions. You can check it using
SCST proc or sysfs interface. You should use available facilities, like
NPIV, to make separate sessions for each VM. If you miss it, you can
greatly loose performance of parallel access to your target from
different VMs. This isn't related to the case if your VMs are using the
same shared storage, like with VMFS, for instance. In this case all your
VM hosts will be connected to the target via separate sessions, which is
enough.

2. See SCST core's README for more advices. Especially pay attention to
have io_grouping_type option set correctly.


Credits
-------

Thanks to:

 * QLogic support for their invaluable help.

 * Nathaniel Clark <nate@misrule.us> for porting to new 2.6 kernel
initiator driver.

 * Mark Buechler <mark.buechler@gmail.com> for the original
WWN-based authentication, a lot of useful suggestions, bug reports and
help in debugging.

 * Ming Zhang <mingz@ele.uri.edu> for fixes.

 * Uri Yanai <Uri.Yanai@ngsoft.com> and Dorit Halsadi
<Dorit.Halsadi@dothill.com> for adding full NPIV support.

Vladislav Bolkhovitin <vst@vlnb.net>, http://scst.sourceforge.net