sg3_utils/doc/sg_dd.8

.TH SG_DD "8" "June 2013" "sg3_utils\-1.37" SG3_UTILS
.SH NAME
sg_dd \- copy data to and from files and devices, especially SCSI
devices
.SH SYNOPSIS
.B sg_dd
[\fIbs=BS\fR] [\fIconv=CONV\fR] [\fIcount=COUNT\fR] [\fIibs=BS\fR]
[\fIif=IFILE\fR] [\fIiflag=FLAGS\fR] [\fIobs=BS\fR] [\fIof=OFILE\fR]
[\fIoflag=FLAGS\fR] [\fIseek=SEEK\fR] [\fIskip=SKIP\fR] [\fI\-\-help\fR]
[\fI\-\-version\fR]
.PP
[\fIblk_sgio=\fR{0|1}] [\fIbpt=BPT\fR] [\fIcdbsz=\fR{6|10|12|16}]
[\fIcoe=\fR{0|1|2|3}] [\fIcoe_limit=CL\fR] [\fIdio=\fR{0|1}]
[\fIodir=\fR{0|1}] [\fIof2=OFILE2\fR] [\fIretries=RETR\fR] [\fIsync=\fR{0|1}]
[\fItime=\fR{0|1}] [\fIverbose=VERB\fR] [\fI\-V\fR]
.SH DESCRIPTION
.\" Add any additional description here
.PP
Copy data to and from any files. Specialized for "files" that are Linux SCSI
generic (sg) devices, raw devices or other devices that support the SG_IO
ioctl (which are only found in the lk 2.6 series). Similar syntax and
semantics to
.B dd(1)
command.
.PP
The first group in the synopsis above are "standard" Unix
.B dd(1)
operands. The second group are extra options added by this utility.
Both groups are defined below.
.PP
This utility is only supported on Linux whereas most other utilities in the
sg3_utils package have been ported to other operating systems. A utility
called "ddpt" has similar syntax and functionality to sg_dd. ddpt drops some
Linux specific features while adding some other generic features. This allows
ddpt to be ported to other operating systems.
.SH OPTIONS
.TP
\fBblk_sgio\fR={0|1}
when set to 0, block devices (e.g. /dev/sda) are treated like normal
files (i.e.
.B read(2)
and
.B write(2)
are used for IO). When set to 1, block devices are assumed to accept the
SG_IO ioctl and SCSI commands are issued for IO. This is only supported
for 2.6 series kernels. Note that ATAPI devices (e.g. cd/dvd players) use
the SCSI command set but ATA disks do not (unless there is a protocol
conversion as often occurs in the USB mass storage class). If the input
or output device is a block device partition (e.g. /dev/sda3) then setting
this option causes the partition information to be ignored (since access
is directly to the underlying device). Default is 0. See the 'sgio' flag.
.TP
\fBbpt\fR=\fIBPT\fR
each IO transaction will be made using \fIBPT\fR blocks (or less if near
the end of the copy). Default is 128 for block sizes less that 2048
bytes, otherwise the default is 32. So for bs=512 the reads and writes
will each convey 64 KiB of data by default (less if near the end of the
transfer or memory restrictions). When cd/dvd drives are accessed, the
block size is typically 2048 bytes and bpt defaults to 32 which again
implies 64 KiB transfers. The block layer when the blk_sgio=1 option
is used has relatively low upper limits for transfer sizes (compared
to sg device nodes, see /sys/block/<dev_name>/queue/max_sectors_kb ).
.TP
\fBbs\fR=\fIBS\fR
where \fIBS\fR
.B must
be the block size of the physical device (if either the input or output
files are accessed via SCSI commands). Note that this differs from
.B dd(1)
which permits \fIBS\fR to be an integral multiple. Default is 512 which
is usually correct for disks but incorrect for cdroms (which normally
have 2048 byte blocks). For this utility the maximum size of each individual
IO operation is \fIBS\fR * \fIBPT\fR bytes.
.TP
\fBcdbsz\fR={6|10|12|16}
size of SCSI READ and/or WRITE commands issued on sg device
names (or block devices when 'iflag=sgio' and/or 'oflag=sgio' is given).
Default is 10 byte SCSI command blocks (unless calculations indicate
that a 4 byte block number may be exceeded or \fIBPT\fR is greater than
16 bits (65535), in which case it defaults to 16 byte SCSI commands).
.TP
\fBcoe\fR={0|1|2|3}
set to 1 or more for continue on error. Only applies to errors on sg
devices or block devices with the 'sgio' flag set. Thus errors on other
files will stop sg_dd. Default is 0 which implies stop on any error. See
the 'coe' flag for more information.
.TP
\fBcoe_limit\fR=\fICL\fR
where \fICL\fR is the maximum number of consecutive bad blocks stepped
over (due to "coe>0") on reads before the copy terminates. This only
applies when \fIIFILE\fR is accessed via the SG_IO ioctl. The default
is 0 which is interpreted as no limit. This option is meant to stop
the copy soon after unrecorded media is detected while still
offering "continue on error" capability.
.TP
\fBconv\fR=\fBsparse\fR
see the CONVERSIONS section below.
.TP
\fBcount\fR=\fICOUNT\fR
copy \fICOUNT\fR blocks from \fIIFILE\fR to \fIOFILE\fR. Default is the
minimum (of \fIIFILE\fR and \fIOFILE\fR) number of blocks that sg devices
report from SCSI READ CAPACITY commands or that block devices (or their
partitions) report. Normal files are not probed for their size. If
\fIskip=SKIP\fR or \fIskip=SEEK\fR are given and the count is derived (i.e.
not explicitly given) then the derived count is scaled back so that the
copy will not overrun the device. If the file name is a block device
partition and \fICOUNT\fR is not given then the size of the partition
rather than the size of the whole device is used. If \fICOUNT\fR is not
given (or \fIcount=\-1\fR) and cannot be derived then an error message is
issued and no copy takes place.
.TP
\fBdio\fR={0|1}
default is 0 which selects indirect (buffered) IO on sg devices. Value of 1
attempts direct IO which, if not available, falls back to indirect IO and
notes this at completion. If direct IO is selected and /proc/scsi/sg/allow_dio
has the value of 0 then a warning is issued (and indirect IO is performed).
For finer grain control use 'iflag=dio' or 'oflag=dio'.
.TP
\fBibs\fR=\fIBS\fR
if given must be the same as \fIBS\fR given to 'bs=' option.
.TP
\fBif\fR=\fIIFILE\fR
read from \fIIFILE\fR instead of stdin. If \fIIFILE\fR is '\-' then stdin
is read. Starts reading at the beginning of \fIIFILE\fR unless \fISKIP\fR
is given.
.TP
\fBiflag\fR=\fIFLAGS\fR
where \fIFLAGS\fR is a comma separated list of one or more flags outlined
below.  These flags are associated with \fIIFILE\fR and are ignored when
\fIIFILE\fR is stdin.
.TP
\fBobs\fR=\fIBS\fR
if given must be the same as \fIBS\fR given to 'bs=' option.
.TP
\fBodir\fR={0|1}
when set to one opens block devices (e.g. /dev/sda) with the O_DIRECT
flag. User memory buffers are aligned to the page size when set. The
default is 0 (i.e. the O_DIRECT flag is not used). Has no effect on sg,
normal or raw files. If blk_sgio is also set then both are honoured:
block devices are opened with the O_DIRECT flag and SCSI commands are
issued via the SG_IO ioctl.
.TP
\fBof\fR=\fIOFILE\fR
write to \fIOFILE\fR instead of stdout. If \fIOFILE\fR is '\-' then writes
to stdout.  If \fIOFILE\fR is /dev/null then no actual writes are performed.
If \fIOFILE\fR is '.' (period) then it is treated the same way as
/dev/null (this is a shorthand notation). If \fIOFILE\fR exists then it
is _not_ truncated; it is overwritten from the start of \fIOFILE\fR
unless 'oflag=append' or \fISEEK\fR is given.
.TP
\fBof2\fR=\fIOFILE2\fR
write output to \fIOFILE2\fR. The default action is not to do this additional
write (i.e. when this option is not given). \fIOFILE2\fR is assumed to be
a normal file or a fifo (i.e. a named pipe). \fIOFILE2\fR is opened for writing,
created if necessary, and closed at the end of the transfer. If \fIOFILE2\fR
is a fifo (named pipe) then some other command should be consuming that
data (e.g. 'md5sum OFILE2'), otherwise this utility will block.
.TP
\fBoflag\fR=\fIFLAGS\fR
where \fIFLAGS\fR is a comma separated list of one or more flags outlined
below.  These flags are associated with \fIOFILE\fR and are ignored when
\fIOFILE\fR is /dev/null, '.' (period), or stdout.
.TP
\fBretries\fR=\fIRETR\fR
sometimes retries at the host are useful, for example when there is a
transport error. When \fIRETR\fR is greater than zero then SCSI READs and
WRITEs are retried on error, \fIRETR\fR times. Default value is zero.
.TP
\fBseek\fR=\fISEEK\fR
start writing \fISEEK\fR bs\-sized blocks from the start of \fIOFILE\fR.
Default is block 0 (i.e. start of file).
.TP
\fBskip\fR=\fISKIP\fR
start reading \fISKIP\fR bs\-sized blocks from the start of \fIIFILE\fR.
Default is block 0 (i.e. start of file).
.TP
\fBsync\fR={0|1}
when 1, does SYNCHRONIZE CACHE command on \fIOFILE\fR at the end of the
transfer. Only active when \fIOFILE\fR is a sg device file name or a block
device and 'blk_sgio=1' is given.
.TP
\fBtime\fR={0|1}
when 1, times transfer and does throughput calculation, outputting the
results (to stderr) at completion. When 0 (default) doesn't perform timing.
.TP
\fBverbose\fR=\fIVERB\fR
as \fIVERB\fR increases so does the amount of debug output sent to stderr.
Default value is zero which yields the minimum amount of debug output.
A value of 1 reports extra information that is not repetitive. A value
2 reports cdbs and responses for SCSI commands that are not repetitive
(i.e. other that READ and WRITE). Error processing is not considered
repetitive. Values of 3 and 4 yield output for all SCSI commands (and
Unix read() and write() calls) so there can be a lot of output.
This only occurs for scsi generic (sg) devices and block devices when
the 'blk_sgio=1' option is set.
.TP
\fB\-\-help\fR
outputs usage message and exits.
.TP
\fB\-\-version\fR
outputs version number information and exits.
.TP
\fB\-V\fR
outputs version number information and exits.
.SH CONVERSIONS
One or more conversions can be given to the "conv=" option. If more than
one is given, they should be comma separated. sg_dd does not perform the
traditional dd conversions (e.g. ASCII to EBCDIC). Recently added
conversions overlap somewhat with the flags so some conversions are
now supported by sg_dd.
.TP
noerror
this conversion is very close to "iflag=coe" and is treated as such. See
the "coe" flag. Note that an error on \fIOFILE\fR will stop the copy.
.TP
notrunc
this conversion is accepted for compatibilty with dd and ignored since
the default action of this utility is not to truncate \fIOFILE\fR.
.TP
null
has no affect, just a placeholder.
.TP
sparse
FreeBSD supports "conv=sparse" so the same syntax is supported in sg_dd.
See "sparse" in the FLAGS sections for more information.
.TP
sync
is ignored by sg_dd. With dd it means supply zero fill (rather than skip)
and is typically used like this "conv=noerror,sync" to have the same
functionality as sg_dd's "iflag=coe".
.SH FLAGS
Here is a list of flags and their meanings:
.TP
append
causes the O_APPEND flag to be added to the open of \fIOFILE\fR. For regular
files this will lead to data appended to the end of any existing data.
Cannot be used together with the \fIseek=SEEK\fR option as they conflict.
The default action of this utility is to overwrite any existing data
from the beginning of the file or, if \fISEEK\fR is given, starting at
block \fISEEK\fR. Note that attempting to 'append' to a device file (e.g.
a disk) will usually be ignored or may cause an error to be reported.
.TP
coe
continue on error. Only active for sg devices and block devices that
have the 'sgio' flag set. 'iflag=coe oflag=coe' and 'coe=1' are
equivalent. Use this flag twice (e.g. 'iflag=coe,coe') to have the
same action as the 'coe=2'. A medium, hardware or blank check error
while reading will re\-read blocks prior to the bad block, then try to
recover the bad block, supplying zeros if that fails, and finally reread
the blocks after the bad block. A medium, hardware or blank check error
while writing is noted and ignored. The recovery of the bad block when
reading uses the SCSI READ LONG command if 'coe' given twice or
more (also with the command line option 'coe=2'). Further, the READ LONG
will set its CORRCT bit if 'coe' given thrice. SCSI disks may automatically
try and remap faulty sectors (see the AWRE and ARRE in the read write
error recovery mode page (the sdparm utility can access and possibly change
these attributes)). Errors occurring on other files types will stop sg_dd.
Error messages are sent to stderr. This flag is similar
 o 'conv=noerror,sync' in the
.B dd(1)
utility. See note about READ LONG below.
.TP
dio
request the sg device node associated with this flag does direct IO.
If direct IO is not available, falls back to indirect IO and notes
this at completion. If direct IO is selected and /proc/scsi/sg/allow_dio
has the value of 0 then a warning is issued (and indirect IO is performed).
.TP
direct
causes the O_DIRECT flag to be added to the open of \fIIFILE\fR and/or
\fIOFILE\fR. This flag requires some memory alignment on IO. Hence user
memory buffers are aligned to the page size. Has no effect on sg, normal
or raw files. If 'iflag=sgio' and/or 'oflag=sgio' is also set then both
are honoured: block devices are opened with the O_DIRECT flag and SCSI
commands are issued via the SG_IO ioctl.
.TP
dpo
set the DPO bit (disable page out) in SCSI READ and WRITE commands. Not
supported for 6 byte cdb variants of READ and WRITE. Indicates that
data is unlikely to be required to stay in device (e.g. disk) cache.
May speed media copy and/or cause a media copy to have less impact
on other device users.
.TP
dsync
causes the O_SYNC flag to be added to the open of \fIIFILE\fR and/or
\fIOFILE\fR. The 'd' is prepended to lower confusion with the 'sync=0|1'
option which has another action (i.e. a synchronisation to media at the
end of the transfer).
.TP
excl
causes the O_EXCL flag to be added to the open of \fIIFILE\fR and/or
\fIOFILE\fR.
.TP
flock
after opening the associated file (i.e. \fIIFILE\fR and/or \fIOFILE\fR)
an attempt is made to get an advisory exclusive lock with the flock()
system call. The flock arguments are "FLOCK_EX | FLOCK_NB" which will
cause the lock to be taken if available else a "temporarily unavailable"
error is generated. An exit status of 90 is produced in the latter case
and no copy is done.
.TP
fua
causes the FUA (force unit access) bit to be set in SCSI READ and/or WRITE
commands. This only has an effect with sg devices or block devices
that have the 'sgio' flag set. The 6 byte variants of the SCSI READ and
WRITE commands do not support the FUA bit.
.TP
nocache
use posix_fadvise() to advise corresponding file there is no need to fill
the file buffer with recently read or written blocks.
.TP
null
has no affect, just a placeholder.
.TP
sgio
causes block devices to be accessed via the SG_IO ioctl rather than
standard UNIX read() and write() commands. When the SG_IO ioctl is
used the SCSI READ and WRITE commands are used directly to move
data. sg devices always use the SG_IO ioctl. This flag offers finer
grain control compared to the otherwise identical 'blk_sgio=1' option.
.TP
sparse
after each \fIBS\fR * \fIBPT\fR byte segment is read from the input,
it is checked for being all zeros. If so, nothing is written to the output
file unless this is the last segment of the transfer. This flag is only
active with the oflag option. It cannot be used when the output is not
seekable (e.g. stdout). It is ignored if the output file is /dev/null .
Note that this utility does not remove the \fIOFILE\fR prior to starting
to write to it. Hence it may be advantageous to manually remove the
\fIOFILE\fR if it is large prior to using oflag=sparse. The last segment
is always written so regular files will show the same length and so
programs like md5sum and sha1sum will generate the same value regardless
of whether oflag=sparse is given or not. This option may be used when the
\fIOFILE\fR is a raw device but is probably only useful if the device is
known to contain zeros (e.g. a SCSI disk after a FORMAT command).
.SH RETIRED OPTIONS
Here are some retired options that are still present:
.TP
append=0 | 1
when set, equivalent to 'oflag=append'. When clear the action is
to overwrite the existing file (if it exists); this is the default.
See the 'append' flag.
.TP
fua=0 | 1 | 2 | 3
force unit access bit. When 3, fua is set on both \fIIFILE\fR and
\fIOFILE\fR; when 2, fua is set on \fIIFILE\fR;, when 1, fua is set on
\fIOFILE\fR; when 0 (default), fua is cleared on both. See the 'fua' flag.
.SH NOTES
Block devices (e.g. /dev/sda and /dev/hda) can be given for \fIIFILE\fR.
If neither '\-iflag=direct', 'iflag=sgio' nor 'blk_sgio=1' is given then
normal block IO involving buffering and caching is performed. If
only '\-iflag=direct' is given then the buffering and caching is
bypassed (this is applicable to both SCSI devices and ATA disks).
If 'iflag=sgio' or 'blk_sgio=1' is given then the SG_IO ioctl is used on
the given file causing SCSI commands to be sent to the device and that also
bypasses most of the actions performed by the block layer (this is only
applicable to SCSI devices, not ATA disks). The same applies for block
devices given for \fIOFILE\fR.
.PP
Various numeric arguments (e.g. \fISKIP\fR) may include multiplicative
suffixes or be given in hexadecimal. See the "NUMERIC ARGUMENTS" section
in the sg3_utils(8) man page.
.PP
The \fICOUNT\fR, \fISKIP\fR and \fISEEK\fR arguments can take 64 bit
values (i.e. very big numbers). Other values are limited to what can fit in
a signed 32 bit number.
.PP
Data usually gets to the user space in a 2 stage process: first the
SCSI adapter DMAs into kernel buffers and then the sg driver copies
this data into user memory (write operations reverse this sequence).
This is called "indirect IO" and there is a 'dio' option to
select "direct IO" which will DMA directly into user memory. Due to some
issues "direct IO" is disabled in the sg driver and needs a
configuration change to activate it. This is typically done
with 'echo 1 > /proc/scsi/sg/allow_dio'.
.PP
All informative, warning and error output is sent to stderr so that
dd's output file can be stdout and remain unpolluted. If no options
are given, then the usage message is output and nothing else happens.
.PP
Even if READ LONG succeeds on a "bad" block when 'coe=2' (or 'coe=3')
is given, the recovered data may not be useful. There are no guarantees
that the user data will appear "as is" in the first 512 bytes.
.PP
A raw device must be bound to a block device prior to using sg_dd.
See
.B raw(8)
for more information about binding raw devices. To be safe, the sg device
mapping to SCSI block devices should be checked with 'cat /proc/scsi/scsi',
or sg_map before use.
.PP
Disk partition information can often be found with
.B fdisk(8)
[the "\-ul" argument is useful in this respect].
.PP
For sg devices (and block devices when blk_sgio=1 is given) this utility
issues SCSI READ and WRITE (SBC) commands which are appropriate for disks and
reading from CD/DVD/HD\-DVD/BD drives. Those commands
are not formatted correctly for tape devices so sg_dd should not be used on
tape devices. If the largest block address of the requested transfer
exceeds a 32 bit block number (i.e 0xffff) then a warning is issued and
the sg device is accessed via SCSI READ(16) and WRITE(16) commands.
.PP
The attributes of a block device (partition) are ignored when 'blk_sgio=1'
is used. Hence the whole device is read (rather than just the second
partition) by this invocation:
.PP
   sg_dd if=/dev/sdb2 blk_sgio=1 of=t bs=512
.SH EXAMPLES
.PP
Looks quite similar in usage to dd:
.PP
   sg_dd if=/dev/sg0 of=t bs=512 count=1MB
.PP
This will copy 1 million 512 byte blocks from the device associated with
/dev/sg0 (which should have 512 byte blocks) to a file called t.
Assuming /dev/sda and /dev/sg0 are the same device then the above is
equivalent to:
.PP
   dd if=/dev/sda iflag=direct of=t bs=512 count=1000000
.PP
although dd's speed may improve if bs was larger and count was suitably
reduced. The use of the 'iflag=direct' option bypasses the buffering and
caching that is usually done on a block device.
.PP
Using a raw device to do something similar on a ATA disk:
.PP
   raw /dev/raw/raw1 /dev/hda
.br
   sg_dd if=/dev/raw/raw1 of=t bs=512 count=1MB
.PP
To copy a SCSI disk partition to an ATA disk partition:
.PP
   raw /dev/raw/raw2 /dev/hda3
.br
   sg_dd if=/dev/sg0 skip=10123456 of=/dev/raw/raw2 bs=512
.PP
This assumes a valid partition is found on the SCSI disk at the given
skip block address (past the 5 GB point of that disk) and that
the partition goes to the end of the SCSI disk. An explicit count
is probably a safer option. The partition is copied to /dev/hda3 which
is an offset into the ATA disk /dev/hda . The exact number of blocks
read from /dev/sg0 are written to /dev/hda (i.e. no padding).
.PP
To time a streaming read of the first 1 GB (2 ** 30 bytes) on a disk
this utility could be used:
.PP
   sg_dd if=/dev/sg0 of=/dev/null bs=512 count=2m time=1
.PP
On completion this will output a line like:
"time to transfer data was 18.779506 secs, 57.18 MB/sec". The "MB/sec"
in this case is 1,000,000 bytes per second.
.PP
The 'of2=' option can be used to copy data and take a md5sum of it
without needing to re\-read the data:
.PP
  mkfifo fif
.br
  md5sum fif &
.br
  sg_dd if=/dev/sg3 iflag=coe of=sg3.img oflag=sparse of2=fif bs=512
.PP
This will image /dev/sg3 (e.g. an unmounted disk) and place the contents
in the (sparse) file sg3.img . Without re\-reading the data it will also
perform a md5sum calculation on the image.
.SH SIGNALS
The signal handling has been borrowed from dd: SIGINT, SIGQUIT and
SIGPIPE output the number of remaining blocks to be transferred and
the records in + out counts; then they have their default action.
SIGUSR1 causes the same information to be output yet the copy continues.
All output caused by signals is sent to stderr.
.SH EXIT STATUS
The exit status of sg_dd is 0 when it is successful. Otherwise see
the sg3_utils(8) man page. Since this utility works at a higher level
than individual commands, and there are 'coe' and 'retries' flags,
individual SCSI command failures do not necessary cause the process
to exit.
.PP
An additional exit status of 90 is generated if the flock flag is given
and some other process holds the advisory exclusive lock.
.SH AUTHORS
Written by Douglas Gilbert and Peter Allworth.
.SH "REPORTING BUGS"
Report bugs to <dgilbert at interlog dot com>.
.SH COPYRIGHT
Copyright \(co 2000\-2013 Douglas Gilbert
.br
This software is distributed under the GPL version 2. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
.SH "SEE ALSO"
There is a web page discussing sg_dd at http://sg.danny.cz/sg/sg_dd.html
.PP
A POSIX threads version of this utility called
.B sgp_dd
is in the sg3_utils package. Another version from that package is called
.B sgm_dd
and it uses memory mapped IO to speed transfers from sg devices.
.PP
The lmbench package contains
.B lmdd
which is also interesting. For moving data to and from tapes see
.B dt
which is found at http://www.scsifaq.org/RMiller_Tools/index.html
.PP
To change mode parameters that effect a SCSI device's caching and error
recovery see
.B sdparm(sdparm)
.PP
To verify the data on the media or to verify it against some other
copy of the data see
.B sg_verify(sg3_utils)
.PP
See also
.B raw(8), dd(1), ddrescue(GNU), ddpt