libibverbs: Introduce Completion Counters verbs Extend verbs interface to support Completion Counters that can be seen as a light-weight alternative to polling CQ. A completion counter object separately counts successful and error completions, can be attached to multiple QPs and be configured to count completions of a subset of operation types. This is especially useful for batch or credit based workloads running on accelerators but can serve many other types of applications as well. Expose supported number of completion counters through query device extended verb. Reviewed-by: Daniel Kinsbursky <dkinsb@amazon.com> Reviewed-by: Yonatan Nachum <ynachum@amazon.com> Signed-off-by: Michael Margolin <mrgolin@amazon.com>
diff --git a/libibverbs/man/CMakeLists.txt b/libibverbs/man/CMakeLists.txt index f8eca86..0efe0b0 100644 --- a/libibverbs/man/CMakeLists.txt +++ b/libibverbs/man/CMakeLists.txt
@@ -15,6 +15,7 @@ ibv_create_ah.3 ibv_create_ah_from_wc.3 ibv_create_comp_channel.3 + ibv_create_comp_cntr.3.md ibv_create_counters.3.md ibv_create_cq.3 ibv_create_cq_ex.3 @@ -58,6 +59,8 @@ ibv_post_send.3 ibv_post_srq_ops.3 ibv_post_srq_recv.3 + ibv_qp_attach_comp_cntr.3.md + ibv_query_comp_cntr_caps.3.md ibv_query_device.3 ibv_query_device_ex.3 ibv_query_ece.3.md @@ -102,6 +105,13 @@ ibv_create_ah.3 ibv_destroy_ah.3 ibv_create_ah_from_wc.3 ibv_init_ah_from_wc.3 ibv_create_comp_channel.3 ibv_destroy_comp_channel.3 + ibv_create_comp_cntr.3 ibv_destroy_comp_cntr.3 + ibv_create_comp_cntr.3 ibv_set_comp_cntr.3 + ibv_create_comp_cntr.3 ibv_set_err_comp_cntr.3 + ibv_create_comp_cntr.3 ibv_inc_comp_cntr.3 + ibv_create_comp_cntr.3 ibv_inc_err_comp_cntr.3 + ibv_create_comp_cntr.3 ibv_read_comp_cntr.3 + ibv_create_comp_cntr.3 ibv_read_err_comp_cntr.3 ibv_create_counters.3 ibv_destroy_counters.3 ibv_create_cq.3 ibv_destroy_cq.3 ibv_create_flow.3 ibv_destroy_flow.3
diff --git a/libibverbs/man/ibv_create_comp_cntr.3.md b/libibverbs/man/ibv_create_comp_cntr.3.md new file mode 100644 index 0000000..aa5ce43 --- /dev/null +++ b/libibverbs/man/ibv_create_comp_cntr.3.md
@@ -0,0 +1,180 @@ +--- +date: 2026-02-09 +footer: libibverbs +header: "Libibverbs Programmer's Manual" +layout: page +license: 'Licensed under the OpenIB.org BSD license (FreeBSD Variant) - See COPYING.md' +section: 3 +title: ibv_create_comp_cntr +tagline: Verbs +--- + +# NAME + +**ibv_create_comp_cntr**, **ibv_destroy_comp_cntr** - Create or destroy a +completion counter + +**ibv_set_comp_cntr**, **ibv_set_err_comp_cntr** - Set the value of a +completion or error counter + +**ibv_inc_comp_cntr**, **ibv_inc_err_comp_cntr** - Increment a completion or +error counter + +**ibv_read_comp_cntr**, **ibv_read_err_comp_cntr** - Read the value of a +completion or error counter + +# SYNOPSIS + +```c +#include <infiniband/verbs.h> + +struct ibv_comp_cntr *ibv_create_comp_cntr(struct ibv_context *context, + struct ibv_comp_cntr_init_attr *cc_attr); + +int ibv_destroy_comp_cntr(struct ibv_comp_cntr *comp_cntr); + +int ibv_set_comp_cntr(struct ibv_comp_cntr *comp_cntr, uint64_t value); +int ibv_set_err_comp_cntr(struct ibv_comp_cntr *comp_cntr, uint64_t value); +int ibv_inc_comp_cntr(struct ibv_comp_cntr *comp_cntr, uint64_t amount); +int ibv_inc_err_comp_cntr(struct ibv_comp_cntr *comp_cntr, uint64_t amount); +int ibv_read_comp_cntr(struct ibv_comp_cntr *comp_cntr, uint64_t *value); +int ibv_read_err_comp_cntr(struct ibv_comp_cntr *comp_cntr, uint64_t *value); +``` + +# DESCRIPTION + +Completion counters provide a lightweight completion mechanism as an +alternative or extension to completion queues (CQs). Rather than generating +individual completion queue entries, a completion counter tracks the aggregate +number of completed operations. This makes them well suited for applications +that need to know how many requests have completed without requiring +per-request details, such as credit based flow control or tracking responses +from remote peers. + +Each completion counter maintains two distinct 64-bit values: a completion +count that is incremented on successful completions, and an error count that +is incremented when operations complete in error. + +**ibv_create_comp_cntr**() allocates a new completion counter for the RDMA +device context *context*. The properties of the counter are defined by +*cc_attr*. The maximum number of completion counters a device supports is +reported by **ibv_query_comp_cntr_caps**(3). + +**ibv_destroy_comp_cntr**() releases all resources associated with the +completion counter *comp_cntr*. The counter must not be attached to any QP +when destroyed. + +**ibv_set_comp_cntr**() sets the completion count of *comp_cntr* to *value*. + +**ibv_set_err_comp_cntr**() sets the error count of *comp_cntr* to *value*. + +**ibv_inc_comp_cntr**() increments the completion count of *comp_cntr* by +*amount*. + +**ibv_inc_err_comp_cntr**() increments the error count of *comp_cntr* by +*amount*. + +**ibv_read_comp_cntr**() reads the current completion count of *comp_cntr* +into *value*. + +**ibv_read_err_comp_cntr**() reads the current error count of *comp_cntr* +into *value*. + +# ARGUMENTS + +## ibv_comp_cntr + +```c +struct ibv_comp_cntr { + struct ibv_context *context; + uint32_t handle; +}; +``` + +*context* +: Device context associated with the completion counter. + +*handle* +: Kernel object handle for the completion counter. + +## ibv_comp_cntr_init_attr + +```c +enum ibv_comp_cntr_type { + IBV_COMP_CNTR_TYPE_WRS, + IBV_COMP_CNTR_TYPE_BYTES, +}; + +struct ibv_comp_cntr_init_attr { + uint32_t comp_mask; + enum ibv_comp_cntr_type type; + uint32_t flags; +}; +``` + +*comp_mask* +: Bitmask specifying what fields in the structure are valid. + +*type* +: The counting mode for the completion counter. Not all devices support + all modes. + **IBV_COMP_CNTR_TYPE_WRS** counts completed work requests (default). + **IBV_COMP_CNTR_TYPE_BYTES** counts completed bytes. + +*flags* +: Reserved for future use, for now must be 0. + +# RETURN VALUE + +**ibv_create_comp_cntr**() returns a pointer to the allocated ibv_comp_cntr +object, or NULL if the request fails (and sets errno to indicate the failure +reason). + +**ibv_destroy_comp_cntr**(), **ibv_set_comp_cntr**(), +**ibv_set_err_comp_cntr**(), **ibv_inc_comp_cntr**(), +**ibv_inc_err_comp_cntr**(), **ibv_read_comp_cntr**(), and +**ibv_read_err_comp_cntr**() return 0 on success, or the value of errno on +failure (which indicates the failure reason). + +# ERRORS + +ENOTSUP +: Completion counters are not supported on this device, or the + requested operation is not supported for the given counter + configuration. + +ENOMEM +: Not enough resources to create the completion counter. + +EINVAL +: Invalid argument(s) passed. + +EBUSY +: The completion counter is still attached to a QP + (**ibv_destroy_comp_cntr**() only). + +# NOTES + +Counter values must only be updated using **ibv_set_comp_cntr**(), +**ibv_set_err_comp_cntr**(), **ibv_inc_comp_cntr**(), or +**ibv_inc_err_comp_cntr**(). + +Updates made to counter values (e.g. via **ibv_set_comp_cntr**() or +**ibv_inc_comp_cntr**()) may not be immediately visible when reading the +counter via **ibv_read_comp_cntr**() or **ibv_read_err_comp_cntr**(). A small +delay may occur between the update and the observed value. However, the final +updated value will eventually be reflected. + +Applications should ensure that the counter value is stable before calling +**ibv_set_comp_cntr**() or **ibv_set_err_comp_cntr**(). Otherwise, concurrent +updates may be lost. + +# SEE ALSO + +**ibv_query_comp_cntr_caps**(3), **ibv_qp_attach_comp_cntr**(3), +**efadv_create_comp_cntr**(3), **ibv_create_cq**(3), **ibv_create_cq_ex**(3), +**ibv_create_qp**(3) + +# AUTHORS + +Michael Margolin <mrgolin@amazon.com>
diff --git a/libibverbs/man/ibv_qp_attach_comp_cntr.3.md b/libibverbs/man/ibv_qp_attach_comp_cntr.3.md new file mode 100644 index 0000000..e7f2eef --- /dev/null +++ b/libibverbs/man/ibv_qp_attach_comp_cntr.3.md
@@ -0,0 +1,123 @@ +--- +date: 2026-02-09 +footer: libibverbs +header: "Libibverbs Programmer's Manual" +layout: page +license: 'Licensed under the OpenIB.org BSD license (FreeBSD Variant) - See COPYING.md' +section: 3 +title: ibv_qp_attach_comp_cntr +tagline: Verbs +--- + +# NAME + +**ibv_qp_attach_comp_cntr** - Attach a completion counter to a QP + +# SYNOPSIS + +```c +#include <infiniband/verbs.h> + +int ibv_qp_attach_comp_cntr(struct ibv_qp *qp, + struct ibv_comp_cntr *comp_cntr, + struct ibv_qp_attach_comp_cntr_attr *attr); +``` + +# DESCRIPTION + +**ibv_qp_attach_comp_cntr**() attaches the completion counter *comp_cntr* to +the queue pair *qp*. The *attr* argument specifies which operation types +should update the counter. + +The QP must be in **IBV_QPS_RESET** or **IBV_QPS_INIT** state when attaching +a completion counter. Attempting to attach a counter to a QP in any other +state will fail with EINVAL. + +The completion counter starts collecting values for the specified QP once +attached. Attaching the same completion counter to multiple QPs will +accumulate values from all attached QPs into the same counter. + +The *op_mask* field controls which operation completions are counted. Local +operations (**IBV_QP_ATTACH_COMP_CNTR_OP_SEND**, **IBV_QP_ATTACH_COMP_CNTR_OP_RECV**, +**IBV_QP_ATTACH_COMP_CNTR_OP_RDMA_READ**, **IBV_QP_ATTACH_COMP_CNTR_OP_RDMA_WRITE**) +count completions initiated by the local QP. Remote operations +(**IBV_QP_ATTACH_COMP_CNTR_OP_REMOTE_RDMA_READ**, +**IBV_QP_ATTACH_COMP_CNTR_OP_REMOTE_RDMA_WRITE**) count completions of incoming +RDMA operations initiated by the remote side. Supported *op_mask* values may +vary by device and can be queried using **ibv_query_comp_cntr_caps**(3); +unsupported values will result in an ENOTSUP error. + +Multiple completion counters can be attached to the same QP, provided their +*op_mask* values do not overlap. Each QP and operation type pair can be +associated with at most one completion counter. Attempting to attach a +counter with an *op_mask* that conflicts with an already attached counter +will fail. + +There is no explicit detach operation. A completion counter is implicitly +detached when the QP it is attached to is destroyed. A completion counter +cannot be destroyed while it is still attached to any QP; the QP must be +destroyed first. + +# ARGUMENTS + +*qp* +: The queue pair to attach the completion counter to. + +*comp_cntr* +: The completion counter to attach, previously created with + **ibv_create_comp_cntr**(). + +*attr* +: Attach attributes specifying which operation types update the counter. + +## ibv_qp_attach_comp_cntr_attr + +```c +enum ibv_qp_attach_comp_cntr_op { + IBV_QP_ATTACH_COMP_CNTR_OP_SEND = 1 << 0, + IBV_QP_ATTACH_COMP_CNTR_OP_RECV = 1 << 1, + IBV_QP_ATTACH_COMP_CNTR_OP_RDMA_READ = 1 << 2, + IBV_QP_ATTACH_COMP_CNTR_OP_REMOTE_RDMA_READ = 1 << 3, + IBV_QP_ATTACH_COMP_CNTR_OP_RDMA_WRITE = 1 << 4, + IBV_QP_ATTACH_COMP_CNTR_OP_REMOTE_RDMA_WRITE = 1 << 5, +}; + +struct ibv_qp_attach_comp_cntr_attr { + uint32_t comp_mask; + uint32_t op_mask; +}; +``` + +*comp_mask* +: Bitmask specifying what fields in the structure are valid. Used + for extensibility, should be set to 0. + +*op_mask* +: Bitmask of **ibv_qp_attach_comp_cntr_op** values specifying which + operation types should update the counter. + +# RETURN VALUE + +**ibv_qp_attach_comp_cntr**() returns 0 on success, or the value of errno on +failure (which indicates the failure reason). + +# ERRORS + +EINVAL +: Invalid argument(s) passed. + +ENOTSUP +: Requested operation is not supported on this device. + +EBUSY +: The *op_mask* overlaps with a completion counter already attached + to this QP. + +# SEE ALSO + +**ibv_create_comp_cntr**(3), **ibv_query_comp_cntr_caps**(3), +**ibv_create_qp**(3) + +# AUTHORS + +Michael Margolin <mrgolin@amazon.com>
diff --git a/libibverbs/man/ibv_query_comp_cntr_caps.3.md b/libibverbs/man/ibv_query_comp_cntr_caps.3.md new file mode 100644 index 0000000..c7a367d --- /dev/null +++ b/libibverbs/man/ibv_query_comp_cntr_caps.3.md
@@ -0,0 +1,71 @@ +--- +date: 2026-07-17 +footer: libibverbs +header: "Libibverbs Programmer's Manual" +layout: page +license: 'Licensed under the OpenIB.org BSD license (FreeBSD Variant) - See COPYING.md' +section: 3 +title: ibv_query_comp_cntr_caps +tagline: Verbs +--- + +# NAME + +**ibv_query_comp_cntr_caps** - Query completion counter capabilities + +# SYNOPSIS + +```c +#include <infiniband/verbs.h> + +int ibv_query_comp_cntr_caps(struct ibv_context *context, + struct ibv_comp_cntr_caps *caps); +``` + +# DESCRIPTION + +**ibv_query_comp_cntr_caps**() queries the completion counter capabilities +of the RDMA device associated with *context*. + +# ARGUMENTS + +## ibv_comp_cntr_caps + +```c +struct ibv_comp_cntr_caps { + uint64_t max_value; + uint32_t max_counters; + uint32_t supported_qp_attach_ops; +}; +``` + +*max_value* +: The maximum value a completion counter can hold. A subsequent + increment that would exceed this value wraps the counter to zero. + +*max_counters* +: The maximum number of completion counters that can be created on + this device. + +*supported_qp_attach_ops* +: A bitmask of **ibv_qp_attach_comp_cntr_op** values indicating which + attach operations are supported by the device. See + **ibv_qp_attach_comp_cntr**(3) for the list of operations. + +# RETURN VALUE + +**ibv_query_comp_cntr_caps**() returns 0 on success, or the value of errno +on failure (which indicates the failure reason). + +# ERRORS + +ENOTSUP +: Completion counters are not supported on this device. + +# SEE ALSO + +**ibv_create_comp_cntr**(3), **ibv_qp_attach_comp_cntr**(3) + +# AUTHORS + +Michael Margolin <mrgolin@amazon.com>
diff --git a/libibverbs/verbs.h b/libibverbs/verbs.h index 9ed55d4..83c9c90 100644 --- a/libibverbs/verbs.h +++ b/libibverbs/verbs.h
@@ -2117,6 +2117,42 @@ struct ibv_pd *parent_domain; }; +enum ibv_qp_attach_comp_cntr_op { + IBV_QP_ATTACH_COMP_CNTR_OP_SEND = 1 << 0, + IBV_QP_ATTACH_COMP_CNTR_OP_RECV = 1 << 1, + IBV_QP_ATTACH_COMP_CNTR_OP_RDMA_READ = 1 << 2, + IBV_QP_ATTACH_COMP_CNTR_OP_REMOTE_RDMA_READ = 1 << 3, + IBV_QP_ATTACH_COMP_CNTR_OP_RDMA_WRITE = 1 << 4, + IBV_QP_ATTACH_COMP_CNTR_OP_REMOTE_RDMA_WRITE = 1 << 5, +}; + +struct ibv_comp_cntr_caps { + uint64_t max_value; + uint32_t max_counters; + uint32_t supported_qp_attach_ops; /* Bitmask of ibv_qp_attach_comp_cntr_op */ +}; + +struct ibv_comp_cntr { + struct ibv_context *context; + uint32_t handle; +}; + +enum ibv_comp_cntr_type { + IBV_COMP_CNTR_TYPE_WRS, + IBV_COMP_CNTR_TYPE_BYTES, +}; + +struct ibv_comp_cntr_init_attr { + uint32_t comp_mask; /* Compatibility mask */ + enum ibv_comp_cntr_type type; + uint32_t flags; +}; + +struct ibv_qp_attach_comp_cntr_attr { + uint32_t comp_mask; /* Compatibility mask */ + uint32_t op_mask; /* Bitmask of ibv_qp_attach_comp_cntr_op */ +}; + enum ibv_parent_domain_init_attr_mask { IBV_PARENT_DOMAIN_INIT_ATTR_ALLOCATORS = 1 << 0, IBV_PARENT_DOMAIN_INIT_ATTR_PD_CONTEXT = 1 << 1, @@ -3027,6 +3063,71 @@ return vctx->modify_cq(cq, attr); } + +/** + * ibv_query_comp_cntr_caps - Query completion counter capabilities + * @context: Device context. + * @caps: Output capabilities struct. + */ +int ibv_query_comp_cntr_caps(struct ibv_context *context, + struct ibv_comp_cntr_caps *caps); + +/** + * ibv_create_comp_cntr - Create a completion counter + * @context: Device context to create the counter on. + * @cc_attr: Attributes for the completion counter. + */ +struct ibv_comp_cntr *ibv_create_comp_cntr(struct ibv_context *context, + struct ibv_comp_cntr_init_attr *cc_attr); + +/** + * ibv_destroy_comp_cntr - Destroy a completion counter + * @comp_cntr: The completion counter to destroy. + */ +int ibv_destroy_comp_cntr(struct ibv_comp_cntr *comp_cntr); + +/** + * ibv_set_comp_cntr - Set the completion count value + * @comp_cntr: The completion counter to update. + * @value: The value to set. + */ +int ibv_set_comp_cntr(struct ibv_comp_cntr *comp_cntr, uint64_t value); + +/** + * ibv_set_err_comp_cntr - Set the error count value + * @comp_cntr: The completion counter to update. + * @value: The value to set. + */ +int ibv_set_err_comp_cntr(struct ibv_comp_cntr *comp_cntr, uint64_t value); + +/** + * ibv_inc_comp_cntr - Increment the completion count + * @comp_cntr: The completion counter to increment. + * @amount: The amount to increment by. + */ +int ibv_inc_comp_cntr(struct ibv_comp_cntr *comp_cntr, uint64_t amount); + +/** + * ibv_inc_err_comp_cntr - Increment the error count + * @comp_cntr: The completion counter to increment. + * @amount: The amount to increment by. + */ +int ibv_inc_err_comp_cntr(struct ibv_comp_cntr *comp_cntr, uint64_t amount); + +/** + * ibv_read_comp_cntr - Read the completion count value + * @comp_cntr: The completion counter to read. + * @value: Output pointer to store the current completion count. + */ +int ibv_read_comp_cntr(struct ibv_comp_cntr *comp_cntr, uint64_t *value); + +/** + * ibv_read_err_comp_cntr - Read the error count value + * @comp_cntr: The completion counter to read. + * @value: Output pointer to store the current error count. + */ +int ibv_read_err_comp_cntr(struct ibv_comp_cntr *comp_cntr, uint64_t *value); + /** * ibv_create_srq - Creates a SRQ associated with the specified protection * domain. @@ -3347,6 +3448,15 @@ } /** + * ibv_qp_attach_comp_cntr - Attach a completion counter to a QP + * @qp: The queue pair to attach the counter to. + * @comp_cntr: The completion counter to attach. + * @attr: Attach attributes. + */ +int ibv_qp_attach_comp_cntr(struct ibv_qp *qp, struct ibv_comp_cntr *comp_cntr, + struct ibv_qp_attach_comp_cntr_attr *attr); + +/** * ibv_query_qp_data_in_order - Checks whether the data is guaranteed to be * written in-order. * @qp: The QP to query.