mlx5: Fix clock update EBUSY error flow
[ Upstream commit 0819419e791dd9cf9e145353fd97a09c43632f9f ]
mlx5dv_get_clock_info() reads clock data published by the kernel, and is
inherently racy with kernel clock updates. This race cannot be
eliminated; EBUSY will always be possible at some rate when wallclock
timestamping is used with extended CQ polling.
Add documentation for the EBUSY errno so callers know the expected
behavior: EBUSY indicates a CQE is available but the wallclock read
failed. Callers are expected to retry polling the CQ. Providers must not
consume the CQE on EBUSY and must leave the CQ in a valid state for the
next poll attempt.
Prior to this fix, the error was returned after the CQE was parsed, the
consumer index incremented, and WQ/SRQ modified. This left the CQ in an
inconsistent state: the CQE was silently lost and the CQ doorbell was
never updated, making recovery impossible regardless of retries.
Fix by moving the clock update before CQE parsing. At that point only
the consumer index has been advanced, so on failure it can be safely
reverted and EBUSY returned with the CQ intact.
Since EBUSY was previously undocumented and the error path was broken,
no existing user could have relied on this behavior. This fix is
therefore not a behavior change in practice.
Fixes: 4745c8079a86 ("mlx5: Implement read_completion_wallclock_ns")
Signed-off-by: Michael Gur <michaelgur@nvidia.com>
Signed-off-by: Yishai Hadas <yishaih@nvidia.com>
Signed-off-by: Nicolas Morey <nmorey@suse.com>
This is the userspace components for the Linux Kernel's drivers/infiniband subsystem. Specifically this contains the userspace libraries for the following device nodes:
The userspace component of the libibverbs RDMA kernel drivers are included under the providers/ directory. Support for the following Kernel RDMA drivers is included:
Additional service daemons are provided for:
This project uses a cmake based build system. Quick start:
$ bash build.sh
build/bin will contain the sample programs and build/lib will contain the shared libraries. The build is configured to run all the programs ‘in-place’ and cannot be installed.
$ apt-get install build-essential cmake gcc libudev-dev libnl-3-dev libnl-route-3-dev ninja-build pkg-config valgrind python3-dev cython3 python3-docutils pandoc
Supported releases:
$ dnf builddep redhat/rdma-core.spec
NOTE: Fedora Core uses the name ‘ninja-build’ for the ‘ninja’ command.
$ zypper install cmake gcc libnl3-devel libudev-devel ninja pkg-config valgrind-devel python3-devel python3-Cython python3-docutils pandoc
Install required packages:
$ yum install cmake gcc libnl3-devel libudev-devel make pkgconfig valgrind-devel
Developers on CentOS 7 or Amazon Linux 2 are suggested to install more modern tooling for the best experience.
CentOS 7:
$ yum install epel-release $ yum install cmake3 ninja-build pandoc
Amazon Linux 2:
$ amazon-linux-extras install epel $ yum install cmake3 ninja-build pandoc
NOTE: EPEL uses the name ‘ninja-build’ for the ‘ninja’ command, and ‘cmake3’ for the ‘cmake’ command.
To set up software RDMA on an existing interface with either of the available drivers, use the following commands, substituting <DRIVER> with the name of the driver of your choice (rdma_rxe or siw) and <TYPE> with the type corresponding to the driver (rxe or siw).
# modprobe <DRIVER> # rdma link add <NAME> type <TYPE> netdev <DEVICE>
Please note that you need version of iproute2 recent enough is required for the command above to work.
You can use either ibv_devices or rdma link to verify that the device was successfully added.
Bugs should be reported to the linux-rdma@vger.kernel.org mailing list In your bug report, please include:
Information about your system:
How to reproduce the bug.
If the bug is a crash, the exact output printed out when the crash occurred, including any kernel messages produced.