srp_daemon: run port services without PrivateNetwork
Since kernel v7.3, commit e9153a267119 ("RDMA/srp: Make the SRP sysfs
class net namespace aware") tags the infiniband_srp sysfs class with
the net namespace of the underlying RDMA device. The
srp_daemon_port@.service unit runs srp_daemon with PrivateNetwork=yes,
so the daemon now sees an empty /sys/class/infiniband_srp/ inside its
private network namespace and fails to log in to any target, endlessly
logging:
srp_daemon: unable to open /sys/class/infiniband_srp/srp-mlx5_0-1/add_target, maybe ib_srp is not loaded
even though ib_srp is loaded and writing add_target from a root shell
in the init netns works. This makes the SRP initiator unusable through
the standard systemd service on any v7.3+ kernel.
srp_daemon fundamentally needs to run in the net namespace of the RDMA
device it manages, both for the umad device and now for the sysfs
class, so network namespace isolation is not applicable to this
service. Set PrivateNetwork=no.
Verified on Ubuntu 26.10 (kernel 7.3.0-9-generic) against an SCST
target: with PrivateNetwork=yes no login ever happens; with this change
both ports log in within seconds.
Signed-off-by: Aurelien Aptel <aaptel@nvidia.com>
This is the userspace components for the Linux Kernel's drivers/infiniband subsystem. Specifically this contains the userspace libraries for the following device nodes:
The userspace component of the libibverbs RDMA kernel drivers are included under the providers/ directory. Support for the following Kernel RDMA drivers is included:
Additional service daemons are provided for:
This project uses a cmake based build system. Quick start:
$ bash build.sh
build/bin will contain the sample programs and build/lib will contain the shared libraries. The build is configured to run all the programs ‘in-place’ and cannot be installed.
$ apt-get install build-essential cmake gcc libudev-dev libnl-3-dev libnl-route-3-dev ninja-build pkg-config valgrind python3-dev cython3 python3-docutils pandoc
Supported releases:
$ dnf builddep redhat/rdma-core.spec
NOTE: Fedora Core uses the name ‘ninja-build’ for the ‘ninja’ command.
$ zypper install cmake gcc libnl3-devel libudev-devel ninja pkg-config valgrind-devel python3-devel python3-Cython python3-docutils pandoc
Install required packages:
$ yum install cmake gcc libnl3-devel libudev-devel make pkgconfig valgrind-devel
Developers on CentOS 7 or Amazon Linux 2 are suggested to install more modern tooling for the best experience.
CentOS 7:
$ yum install epel-release $ yum install cmake3 ninja-build pandoc
Amazon Linux 2:
$ amazon-linux-extras install epel $ yum install cmake3 ninja-build pandoc
NOTE: EPEL uses the name ‘ninja-build’ for the ‘ninja’ command, and ‘cmake3’ for the ‘cmake’ command.
To set up software RDMA on an existing interface with either of the available drivers, use the following commands, substituting <DRIVER> with the name of the driver of your choice (rdma_rxe or siw) and <TYPE> with the type corresponding to the driver (rxe or siw).
# modprobe <DRIVER> # rdma link add <NAME> type <TYPE> netdev <DEVICE>
Please note that you need version of iproute2 recent enough is required for the command above to work.
You can use either ibv_devices or rdma link to verify that the device was successfully added.
Bugs should be reported to the linux-rdma@vger.kernel.org mailing list In your bug report, please include:
Information about your system:
How to reproduce the bug.
If the bug is a crash, the exact output printed out when the crash occurred, including any kernel messages produced.
See Contributing to rdma-core.
Stable versions are released regularly with backported fixes (see Documentation/stable.md) The current minimum version still maintained is ‘v33.X’