libibnetdisc: keep cache loadable after port timeouts
[ Upstream commit de83d1c208f9550ec1024a132ad982dcba9d8c06 ]
ibnetdiscover --cache writes each node's port references and each
port's remote-port reference from the in-memory fabric. If a PortInfo
query times out during discovery, the port can remain as a shell in
node->ports[] or as a remoteport without ever being inserted into the
fabric port table. Such a port has no cache port record, but references
to it were still written. ibnd_load_fabric() then rejected the cache with
"Cache invalid: cannot find port".
Make the cache writer emit only references to ports that are actually
serialized. A port is serialized only when it is reachable from its
owning node, and the same predicate is used for node port references,
remote-port references, and port records. References are matched against
a sorted set of serialized ports, avoiding repeated port-table walks on
large fabrics. Keep nodes with no ports array as zero-port node records
so fabric->from_node remains resolvable.
Also tolerate old incomplete caches when reading: missing node port keys
and missing remote ports are skipped instead of aborting the load. While
doing that, keep malformed caches from creating unsafe fabric state by
bounds-checking cached port numbers before indexing node->ports[] (an
out-of-bounds write) and by skipping unclaimed port records before they
are linked into the fabric and then freed (a use-after-free).
Fixes: 01b8045f28f6 ("add libibnetdisc caching to libibnetdiscover")
Signed-off-by: Jonathan Süssemilch Poulain <jpoulain@coreweave.com>
Signed-off-by: Nicolas Morey <nmorey@suse.com>
This is the userspace components for the Linux Kernel's drivers/infiniband subsystem. Specifically this contains the userspace libraries for the following device nodes:
The userspace component of the libibverbs RDMA kernel drivers are included under the providers/ directory. Support for the following Kernel RDMA drivers is included:
Additional service daemons are provided for:
This project uses a cmake based build system. Quick start:
$ bash build.sh
build/bin will contain the sample programs and build/lib will contain the shared libraries. The build is configured to run all the programs ‘in-place’ and cannot be installed.
$ apt-get install build-essential cmake gcc libudev-dev libnl-3-dev libnl-route-3-dev ninja-build pkg-config valgrind python3-dev cython3 python3-docutils pandoc
Supported releases:
$ dnf builddep redhat/rdma-core.spec
NOTE: Fedora Core uses the name ‘ninja-build’ for the ‘ninja’ command.
$ zypper install cmake gcc libnl3-devel libudev-devel ninja pkg-config valgrind-devel python3-devel python3-Cython python3-docutils pandoc
Install required packages:
$ yum install cmake gcc libnl3-devel libudev-devel make pkgconfig valgrind-devel
Developers on CentOS 7 or Amazon Linux 2 are suggested to install more modern tooling for the best experience.
CentOS 7:
$ yum install epel-release $ yum install cmake3 ninja-build pandoc
Amazon Linux 2:
$ amazon-linux-extras install epel $ yum install cmake3 ninja-build pandoc
NOTE: EPEL uses the name ‘ninja-build’ for the ‘ninja’ command, and ‘cmake3’ for the ‘cmake’ command.
To set up software RDMA on an existing interface with either of the available drivers, use the following commands, substituting <DRIVER> with the name of the driver of your choice (rdma_rxe or siw) and <TYPE> with the type corresponding to the driver (rxe or siw).
# modprobe <DRIVER> # rdma link add <NAME> type <TYPE> netdev <DEVICE>
Please note that you need version of iproute2 recent enough is required for the command above to work.
You can use either ibv_devices or rdma link to verify that the device was successfully added.
Bugs should be reported to the linux-rdma@vger.kernel.org mailing list In your bug report, please include:
Information about your system:
How to reproduce the bug.
If the bug is a crash, the exact output printed out when the crash occurred, including any kernel messages produced.
See Contributing to rdma-core.
Stable versions are released regularly with backported fixes (see Documentation/stable.md) The current minimum version still maintained is ‘v33.X’