| .\" |
| .\" CDDL HEADER START |
| .\" |
| .\" The contents of this file are subject to the terms of the |
| .\" Common Development and Distribution License (the "License"). |
| .\" You may not use this file except in compliance with the License. |
| .\" |
| .\" You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE |
| .\" or http://www.opensolaris.org/os/licensing. |
| .\" See the License for the specific language governing permissions |
| .\" and limitations under the License. |
| .\" |
| .\" When distributing Covered Code, include this CDDL HEADER in each |
| .\" file and include the License file at usr/src/OPENSOLARIS.LICENSE. |
| .\" If applicable, add the following below this CDDL HEADER, with the |
| .\" fields enclosed by brackets "[]" replaced with your own identifying |
| .\" information: Portions Copyright [yyyy] [name of copyright owner] |
| .\" |
| .\" CDDL HEADER END |
| .\" |
| .\" Copyright (c) 2007, Sun Microsystems, Inc. All Rights Reserved. |
| .\" Copyright (c) 2012, 2018 by Delphix. All rights reserved. |
| .\" Copyright (c) 2012 Cyril Plisko. All Rights Reserved. |
| .\" Copyright (c) 2017 Datto Inc. |
| .\" Copyright (c) 2018 George Melikov. All Rights Reserved. |
| .\" Copyright 2017 Nexenta Systems, Inc. |
| .\" Copyright (c) 2017 Open-E, Inc. All Rights Reserved. |
| .\" |
| .Dd May 27, 2021 |
| .Dt ZPOOL-EVENTS 8 |
| .Os |
| . |
| .Sh NAME |
| .Nm zpool-events |
| .Nd list recent events generated by kernel |
| .Sh SYNOPSIS |
| .Nm zpool |
| .Cm events |
| .Op Fl vHf |
| .Op Ar pool |
| .Nm zpool |
| .Cm events |
| .Fl c |
| . |
| .Sh DESCRIPTION |
| Lists all recent events generated by the ZFS kernel modules. |
| These events are consumed by the |
| .Xr zed 8 |
| and used to automate administrative tasks such as replacing a failed device |
| with a hot spare. |
| For more information about the subclasses and event payloads |
| that can be generated see |
| .Sx EVENTS |
| and the following sections. |
| . |
| .Sh OPTIONS |
| .Bl -tag -compact -width Ds |
| .It Fl c |
| Clear all previous events. |
| .It Fl f |
| Follow mode. |
| .It Fl H |
| Scripted mode. |
| Do not display headers, and separate fields by a |
| single tab instead of arbitrary space. |
| .It Fl v |
| Print the entire payload for each event. |
| .El |
| . |
| .Sh EVENTS |
| These are the different event subclasses. |
| The full event name would be |
| .Sy ereport.fs.zfs.\& Ns Em SUBCLASS , |
| but only the last part is listed here. |
| .Pp |
| .Bl -tag -compact -width "vdev.bad_guid_sum" |
| .It Sy checksum |
| Issued when a checksum error has been detected. |
| .It Sy io |
| Issued when there is an I/O error in a vdev in the pool. |
| .It Sy data |
| Issued when there have been data errors in the pool. |
| .It Sy deadman |
| Issued when an I/O request is determined to be "hung", this can be caused |
| by lost completion events due to flaky hardware or drivers. |
| See |
| .Sy zfs_deadman_failmode |
| in |
| .Xr zfs 4 |
| for additional information regarding "hung" I/O detection and configuration. |
| .It Sy delay |
| Issued when a completed I/O request exceeds the maximum allowed time |
| specified by the |
| .Sy zio_slow_io_ms |
| module parameter. |
| This can be an indicator of problems with the underlying storage device. |
| The number of delay events is ratelimited by the |
| .Sy zfs_slow_io_events_per_second |
| module parameter. |
| .It Sy config |
| Issued every time a vdev change have been done to the pool. |
| .It Sy zpool |
| Issued when a pool cannot be imported. |
| .It Sy zpool.destroy |
| Issued when a pool is destroyed. |
| .It Sy zpool.export |
| Issued when a pool is exported. |
| .It Sy zpool.import |
| Issued when a pool is imported. |
| .It Sy zpool.reguid |
| Issued when a REGUID (new unique identifier for the pool have been regenerated) have been detected. |
| .It Sy vdev.unknown |
| Issued when the vdev is unknown. |
| Such as trying to clear device errors on a vdev that have failed/been kicked |
| from the system/pool and is no longer available. |
| .It Sy vdev.open_failed |
| Issued when a vdev could not be opened (because it didn't exist for example). |
| .It Sy vdev.corrupt_data |
| Issued when corrupt data have been detected on a vdev. |
| .It Sy vdev.no_replicas |
| Issued when there are no more replicas to sustain the pool. |
| This would lead to the pool being |
| .Em DEGRADED . |
| .It Sy vdev.bad_guid_sum |
| Issued when a missing device in the pool have been detected. |
| .It Sy vdev.too_small |
| Issued when the system (kernel) have removed a device, and ZFS |
| notices that the device isn't there any more. |
| This is usually followed by a |
| .Sy probe_failure |
| event. |
| .It Sy vdev.bad_label |
| Issued when the label is OK but invalid. |
| .It Sy vdev.bad_ashift |
| Issued when the ashift alignment requirement has increased. |
| .It Sy vdev.remove |
| Issued when a vdev is detached from a mirror (or a spare detached from a |
| vdev where it have been used to replace a failed drive - only works if |
| the original drive have been re-added). |
| .It Sy vdev.clear |
| Issued when clearing device errors in a pool. |
| Such as running |
| .Nm zpool Cm clear |
| on a device in the pool. |
| .It Sy vdev.check |
| Issued when a check to see if a given vdev could be opened is started. |
| .It Sy vdev.spare |
| Issued when a spare have kicked in to replace a failed device. |
| .It Sy vdev.autoexpand |
| Issued when a vdev can be automatically expanded. |
| .It Sy io_failure |
| Issued when there is an I/O failure in a vdev in the pool. |
| .It Sy probe_failure |
| Issued when a probe fails on a vdev. |
| This would occur if a vdev |
| have been kicked from the system outside of ZFS (such as the kernel |
| have removed the device). |
| .It Sy log_replay |
| Issued when the intent log cannot be replayed. |
| The can occur in the case of a missing or damaged log device. |
| .It Sy resilver.start |
| Issued when a resilver is started. |
| .It Sy resilver.finish |
| Issued when the running resilver have finished. |
| .It Sy scrub.start |
| Issued when a scrub is started on a pool. |
| .It Sy scrub.finish |
| Issued when a pool has finished scrubbing. |
| .It Sy scrub.abort |
| Issued when a scrub is aborted on a pool. |
| .It Sy scrub.resume |
| Issued when a scrub is resumed on a pool. |
| .It Sy scrub.paused |
| Issued when a scrub is paused on a pool. |
| .It Sy bootfs.vdev.attach |
| .El |
| . |
| .Sh PAYLOADS |
| This is the payload (data, information) that accompanies an |
| event. |
| .Pp |
| For |
| .Xr zed 8 , |
| these are set to uppercase and prefixed with |
| .Sy ZEVENT_ . |
| .Pp |
| .Bl -tag -compact -width "vdev_cksum_errors" |
| .It Sy pool |
| Pool name. |
| .It Sy pool_failmode |
| Failmode - |
| .Sy wait , |
| .Sy continue , |
| or |
| .Sy panic . |
| See the |
| .Sy failmode |
| property in |
| .Xr zpoolprops 7 |
| for more information. |
| .It Sy pool_guid |
| The GUID of the pool. |
| .It Sy pool_context |
| The load state for the pool (0=none, 1=open, 2=import, 3=tryimport, 4=recover |
| 5=error). |
| .It Sy vdev_guid |
| The GUID of the vdev in question (the vdev failing or operated upon with |
| .Nm zpool Cm clear , |
| etc.). |
| .It Sy vdev_type |
| Type of vdev - |
| .Sy disk , |
| .Sy file , |
| .Sy mirror , |
| etc. |
| See the |
| .Sy Virtual Devices |
| section of |
| .Xr zpoolconcepts 7 |
| for more information on possible values. |
| .It Sy vdev_path |
| Full path of the vdev, including any |
| .Em -partX . |
| .It Sy vdev_devid |
| ID of vdev (if any). |
| .It Sy vdev_fru |
| Physical FRU location. |
| .It Sy vdev_state |
| State of vdev (0=uninitialized, 1=closed, 2=offline, 3=removed, 4=failed to open, 5=faulted, 6=degraded, 7=healthy). |
| .It Sy vdev_ashift |
| The ashift value of the vdev. |
| .It Sy vdev_complete_ts |
| The time the last I/O request completed for the specified vdev. |
| .It Sy vdev_delta_ts |
| The time since the last I/O request completed for the specified vdev. |
| .It Sy vdev_spare_paths |
| List of spares, including full path and any |
| .Em -partX . |
| .It Sy vdev_spare_guids |
| GUID(s) of spares. |
| .It Sy vdev_read_errors |
| How many read errors that have been detected on the vdev. |
| .It Sy vdev_write_errors |
| How many write errors that have been detected on the vdev. |
| .It Sy vdev_cksum_errors |
| How many checksum errors that have been detected on the vdev. |
| .It Sy parent_guid |
| GUID of the vdev parent. |
| .It Sy parent_type |
| Type of parent. |
| See |
| .Sy vdev_type . |
| .It Sy parent_path |
| Path of the vdev parent (if any). |
| .It Sy parent_devid |
| ID of the vdev parent (if any). |
| .It Sy zio_objset |
| The object set number for a given I/O request. |
| .It Sy zio_object |
| The object number for a given I/O request. |
| .It Sy zio_level |
| The indirect level for the block. |
| Level 0 is the lowest level and includes data blocks. |
| Values > 0 indicate metadata blocks at the appropriate level. |
| .It Sy zio_blkid |
| The block ID for a given I/O request. |
| .It Sy zio_err |
| The error number for a failure when handling a given I/O request, |
| compatible with |
| .Xr errno 3 |
| with the value of |
| .Sy EBADE |
| used to indicate a ZFS checksum error. |
| .It Sy zio_offset |
| The offset in bytes of where to write the I/O request for the specified vdev. |
| .It Sy zio_size |
| The size in bytes of the I/O request. |
| .It Sy zio_flags |
| The current flags describing how the I/O request should be handled. |
| See the |
| .Sy I/O FLAGS |
| section for the full list of I/O flags. |
| .It Sy zio_stage |
| The current stage of the I/O in the pipeline. |
| See the |
| .Sy I/O STAGES |
| section for a full list of all the I/O stages. |
| .It Sy zio_pipeline |
| The valid pipeline stages for the I/O. |
| See the |
| .Sy I/O STAGES |
| section for a full list of all the I/O stages. |
| .It Sy zio_delay |
| The time elapsed (in nanoseconds) waiting for the block layer to complete the |
| I/O request. |
| Unlike |
| .Sy zio_delta , |
| this does not include any vdev queuing time and is |
| therefore solely a measure of the block layer performance. |
| .It Sy zio_timestamp |
| The time when a given I/O request was submitted. |
| .It Sy zio_delta |
| The time required to service a given I/O request. |
| .It Sy prev_state |
| The previous state of the vdev. |
| .It Sy cksum_expected |
| The expected checksum value for the block. |
| .It Sy cksum_actual |
| The actual checksum value for an errant block. |
| .It Sy cksum_algorithm |
| Checksum algorithm used. |
| See |
| .Xr zfsprops 7 |
| for more information on the available checksum algorithms. |
| .It Sy cksum_byteswap |
| Whether or not the data is byteswapped. |
| .It Sy bad_ranges |
| .No [\& Ns Ar start , end ) |
| pairs of corruption offsets. |
| Offsets are always aligned on a 64-bit boundary, |
| and can include some gaps of non-corruption. |
| (See |
| .Sy bad_ranges_min_gap ) |
| .It Sy bad_ranges_min_gap |
| In order to bound the size of the |
| .Sy bad_ranges |
| array, gaps of non-corruption |
| less than or equal to |
| .Sy bad_ranges_min_gap |
| bytes have been merged with |
| adjacent corruption. |
| Always at least 8 bytes, since corruption is detected on a 64-bit word basis. |
| .It Sy bad_range_sets |
| This array has one element per range in |
| .Sy bad_ranges . |
| Each element contains |
| the count of bits in that range which were clear in the good data and set |
| in the bad data. |
| .It Sy bad_range_clears |
| This array has one element per range in |
| .Sy bad_ranges . |
| Each element contains |
| the count of bits for that range which were set in the good data and clear in |
| the bad data. |
| .It Sy bad_set_bits |
| If this field exists, it is an array of |
| .Pq Ar bad data No & ~( Ns Ar good data ) ; |
| that is, the bits set in the bad data which are cleared in the good data. |
| Each element corresponds a byte whose offset is in a range in |
| .Sy bad_ranges , |
| and the array is ordered by offset. |
| Thus, the first element is the first byte in the first |
| .Sy bad_ranges |
| range, and the last element is the last byte in the last |
| .Sy bad_ranges |
| range. |
| .It Sy bad_cleared_bits |
| Like |
| .Sy bad_set_bits , |
| but contains |
| .Pq Ar good data No & ~( Ns Ar bad data ) ; |
| that is, the bits set in the good data which are cleared in the bad data. |
| .It Sy bad_set_histogram |
| If this field exists, it is an array of counters. |
| Each entry counts bits set in a particular bit of a big-endian uint64 type. |
| The first entry counts bits |
| set in the high-order bit of the first byte, the 9th byte, etc, and the last |
| entry counts bits set of the low-order bit of the 8th byte, the 16th byte, etc. |
| This information is useful for observing a stuck bit in a parallel data path, |
| such as IDE or parallel SCSI. |
| .It Sy bad_cleared_histogram |
| If this field exists, it is an array of counters. |
| Each entry counts bit clears in a particular bit of a big-endian uint64 type. |
| The first entry counts bits |
| clears of the high-order bit of the first byte, the 9th byte, etc, and the |
| last entry counts clears of the low-order bit of the 8th byte, the 16th byte, etc. |
| This information is useful for observing a stuck bit in a parallel data |
| path, such as IDE or parallel SCSI. |
| .El |
| . |
| .Sh I/O STAGES |
| The ZFS I/O pipeline is comprised of various stages which are defined below. |
| The individual stages are used to construct these basic I/O |
| operations: Read, Write, Free, Claim, and Ioctl. |
| These stages may be |
| set on an event to describe the life cycle of a given I/O request. |
| .Pp |
| .TS |
| tab(:); |
| l l l . |
| Stage:Bit Mask:Operations |
| _:_:_ |
| ZIO_STAGE_OPEN:0x00000001:RWFCI |
| |
| ZIO_STAGE_READ_BP_INIT:0x00000002:R---- |
| ZIO_STAGE_WRITE_BP_INIT:0x00000004:-W--- |
| ZIO_STAGE_FREE_BP_INIT:0x00000008:--F-- |
| ZIO_STAGE_ISSUE_ASYNC:0x00000010:RWF-- |
| ZIO_STAGE_WRITE_COMPRESS:0x00000020:-W--- |
| |
| ZIO_STAGE_ENCRYPT:0x00000040:-W--- |
| ZIO_STAGE_CHECKSUM_GENERATE:0x00000080:-W--- |
| |
| ZIO_STAGE_NOP_WRITE:0x00000100:-W--- |
| |
| ZIO_STAGE_DDT_READ_START:0x00000200:R---- |
| ZIO_STAGE_DDT_READ_DONE:0x00000400:R---- |
| ZIO_STAGE_DDT_WRITE:0x00000800:-W--- |
| ZIO_STAGE_DDT_FREE:0x00001000:--F-- |
| |
| ZIO_STAGE_GANG_ASSEMBLE:0x00002000:RWFC- |
| ZIO_STAGE_GANG_ISSUE:0x00004000:RWFC- |
| |
| ZIO_STAGE_DVA_THROTTLE:0x00008000:-W--- |
| ZIO_STAGE_DVA_ALLOCATE:0x00010000:-W--- |
| ZIO_STAGE_DVA_FREE:0x00020000:--F-- |
| ZIO_STAGE_DVA_CLAIM:0x00040000:---C- |
| |
| ZIO_STAGE_READY:0x00080000:RWFCI |
| |
| ZIO_STAGE_VDEV_IO_START:0x00100000:RW--I |
| ZIO_STAGE_VDEV_IO_DONE:0x00200000:RW--I |
| ZIO_STAGE_VDEV_IO_ASSESS:0x00400000:RW--I |
| |
| ZIO_STAGE_CHECKSUM_VERIFY:0x00800000:R---- |
| |
| ZIO_STAGE_DONE:0x01000000:RWFCI |
| .TE |
| . |
| .Sh I/O FLAGS |
| Every I/O request in the pipeline contains a set of flags which describe its |
| function and are used to govern its behavior. |
| These flags will be set in an event as a |
| .Sy zio_flags |
| payload entry. |
| .Pp |
| .TS |
| tab(:); |
| l l . |
| Flag:Bit Mask |
| _:_ |
| ZIO_FLAG_DONT_AGGREGATE:0x00000001 |
| ZIO_FLAG_IO_REPAIR:0x00000002 |
| ZIO_FLAG_SELF_HEAL:0x00000004 |
| ZIO_FLAG_RESILVER:0x00000008 |
| ZIO_FLAG_SCRUB:0x00000010 |
| ZIO_FLAG_SCAN_THREAD:0x00000020 |
| ZIO_FLAG_PHYSICAL:0x00000040 |
| |
| ZIO_FLAG_CANFAIL:0x00000080 |
| ZIO_FLAG_SPECULATIVE:0x00000100 |
| ZIO_FLAG_CONFIG_WRITER:0x00000200 |
| ZIO_FLAG_DONT_RETRY:0x00000400 |
| ZIO_FLAG_DONT_CACHE:0x00000800 |
| ZIO_FLAG_NODATA:0x00001000 |
| ZIO_FLAG_INDUCE_DAMAGE:0x00002000 |
| |
| ZIO_FLAG_IO_ALLOCATING:0x00004000 |
| ZIO_FLAG_IO_RETRY:0x00008000 |
| ZIO_FLAG_PROBE:0x00010000 |
| ZIO_FLAG_TRYHARD:0x00020000 |
| ZIO_FLAG_OPTIONAL:0x00040000 |
| |
| ZIO_FLAG_DONT_QUEUE:0x00080000 |
| ZIO_FLAG_DONT_PROPAGATE:0x00100000 |
| ZIO_FLAG_IO_BYPASS:0x00200000 |
| ZIO_FLAG_IO_REWRITE:0x00400000 |
| ZIO_FLAG_RAW_COMPRESS:0x00800000 |
| ZIO_FLAG_RAW_ENCRYPT:0x01000000 |
| |
| ZIO_FLAG_GANG_CHILD:0x02000000 |
| ZIO_FLAG_DDT_CHILD:0x04000000 |
| ZIO_FLAG_GODFATHER:0x08000000 |
| ZIO_FLAG_NOPWRITE:0x10000000 |
| ZIO_FLAG_REEXECUTED:0x20000000 |
| ZIO_FLAG_DELEGATED:0x40000000 |
| ZIO_FLAG_FASTWRITE:0x80000000 |
| .TE |
| . |
| .Sh SEE ALSO |
| .Xr zfs 4 , |
| .Xr zed 8 , |
| .Xr zpool-wait 8 |