---

license: Apache-2.0 name: doca-gpunetio-ib-write-bw description: > Use this skill when the user is building, running, or interpreting the doca/tools/gpunetio_ib_write_bw client+server benchmark — a CUDA kernel on the client posts RDMA WRITE work requests through the doca-gpunetio device-side surface to measure sustained GPU-driven WRITE bandwidth on a GPU+IB-device pair. Trigger even when the user does not explicitly mention "doca-gpunetio-ib-write-bw" or "GPUNetIO" — typical implicit phrasings include "measure WRITE BW when the GPU posts the WRs", "BW swings between runs on the same flags", "is the NIC saturated or am I CPU-bound on the CUDA kernel", "meson compile fails for the GPUNetIO bw tool", "nvidia_peermem isn't picking up my GPU buffer", or "GPU-initiated WRITE throughput vs CPU-initiated perftest". Refuse and route elsewhere for general doca-gpunetio library work, DOCA install, the GPU-initiated WRITE latency analog, the CPU-initiated upstream perftest, or application-level end-to-end throughput — those belong to other skills. metadata: kind: tool compatibility: > Requires DOCA SDK on Linux with a BlueField DPU or ConnectX NIC, NVIDIA GPU, CUDA toolkit and nvcc, loaded nvidia_peermem, and an InfiniBand RNIC paired with the GPU. Uses pkg-config for doca-gpunetio, doca-rdma, and doca-common, plus the installed gpunetio_ib_write_bw sources. Run only on a trusted, non-shared IB fabric during the benchmark window.

DOCA GPUNetIO ib_write_bw

Where to start: This is a tool skill for the GPUNetIO- flavored ib_write_bw benchmark shipped under doca/tools/gpunetio_ib_write_bw/ (a client + server pair, built from source against the installed DOCA via meson). It measures sustained RDMA WRITE bandwidth when the WRs are posted from a CUDA kernel through the doca-gpunetio device-side surface, with the GPU on the data path. Open TASKS.md and start at ## configure for the GPU-NIC pairing precondition and the build pattern; jump to ## run for the smoke-before-bulk flow. Open CAPABILITIES.md when the question is what this tool actually measures, how the result decomposes (GPU occupancy vs NIC issue rate vs link saturation), or how the result reads against the GPI sister tool and the upstream CPU-initiated perftest ib_write_bw. If DOCA is not installed yet, route to doca-setup first; if the user is still deciding between the GPI and GPUNetIO programming surfaces, the picture in ../../libs/doca-gpunetio/CAPABILITIES.md#capabilities-and-modes and ../../libs/doca-gpi/CAPABILITIES.md#capabilities-and-modes is the first stop.

Example questions this skill answers well

The CLASSES of doca-gpunetio-ib-write-bw questions this skill is built to answer, each with one worked example. The class is the load-bearing piece; the worked example is one instance.

• "What sustained RDMA-WRITE bandwidth can the GPUNetIO path deliver on this GPU-NIC pair?" — worked example: "measure sustained WRITE BW between two hosts with an H100 + ConnectX-7 on each side". Answered by the GPU-NIC pairing precondition in CAPABILITIES.md ## Capabilities and modes
  • the bring-up flow in TASKS.md ## configure + TASKS.md ## run. The same shape answers "measure GPUNetIO-driven WRITE BW between a host GPU and a BlueField DPU".
• "Where is the bottleneck — GPU compute occupancy, NIC issue rate, or link saturation?" — worked example: "I see 120 Gbit/s on a 200 Gbit/s link; is the NIC saturated, am I CPU-bound on the client, or is the CUDA kernel not driving enough WRs in flight?". Answered by the throughput-decomposition rules in CAPABILITIES.md ## Observability
  • the eval-loop overlay in TASKS.md ## test.
• "How does the result differ from the classic CPU- initiated perftest ib_write_bw?" — worked example: "my team has a CPU-initiated WRITE BW number on this same NIC; should I expect the GPUNetIO number to match or be different?". Answered by the "GPU-initiated path adds (or removes) overhead vs the CPU-initiated path" rule in CAPABILITIES.md ## Capabilities and modes.
• "Is the doca-gpunetio path the right surface for my sustained-throughput workload class?" — worked example: "my application streams sensor data from GPU memory at line rate to a remote consumer". Answered by the "when GPUNetIO is the right surface vs GPI vs CPU- initiated" rule in CAPABILITIES.md ## Capabilities and modes
  • the use-side decision in TASKS.md ## use.
• "My BW number swings between runs. What do I check before quoting it?" — worked example: "three runs at the same flags gave 145, 187, and 160 Gbit/s; is the benchmark noisy or is my platform inconsistent?". Answered by the measurement-soundness rules in CAPABILITIES.md ## Error taxonomy layer 5 + the steady-state guidance in TASKS.md ## test.
• "What version of DOCA + CUDA Toolkit do I need for this binary to build and run?" — worked example: "my install has DOCA at one semver and CUDA at another; will the ToT-shipped gpunetio_ib_write_bw even link?". Answered by the version overlay in CAPABILITIES.md ## Version compatibility which cross-links the canonical detection chain in doca-version.

Audience

This skill serves external developers and performance engineers who need a reproducible measurement of sustained RDMA WRITE bandwidth when the WRs are posted from a CUDA kernel through doca-gpunetio, on the user's actual install and GPU-NIC pair. Concretely:

• A developer comparing the GPUNetIO path against the GPI path or the host-initiated perftest-style path before committing an application design to one of them.
• A platform operator validating a tuning change (NUMA pinning, GPU PCIe placement, IB device choice, GID index, NIC firmware burn) by re-running this benchmark against the new state.
• An SRE / performance engineer producing a "this is the GPUNetIO-driven WRITE BW on this GPU-NIC pair today" artifact downstream consumers can cite.
• An AI agent answering "is the doca-gpunetio path a win for my sustained-throughput workload class" honestly — with a measured number, the build + invocation that produced it, and the GPU + NIC + DOCA version that scopes it — rather than guessing from datasheet headlines.

It is not for users debugging the doca-gpunetio library itself (route to ../../libs/doca-gpunetio/SKILL.md), and not a substitute for the perftest upstream ib_write_bw (which measures CPU-initiated WRITE BW).

Language scope

The doca-gpunetio-ib-write-bw tool is shipped as C plus a CUDA .cu translation unit under doca/tools/gpunetio_ib_write_bw/, split into a client/ subtree and a server/ subtree. The verified surface (per client/{main.c,common.h,common.c,kernel.cu,perftest.c} and server/{main.c,common.h,common.c,perftest.c}): host-side build via meson against the installed DOCA pkg-config modules (doca-gpunetio, doca-rdma, doca-common); the device-side build via nvcc against the DOCA GPU NetIO device-side header set; the OOB descriptor exchange via a TCP socket between client and server. There is no Python / Rust / Go binding — the tool is a pair of CLI binaries. The skill's job is to keep the operator-side workflow language-neutral; the device-side CUDA surface is not wrappable in another language.

When to load this skill

Load this skill when the user is — or the agent needs to — build and run the gpunetio_ib_write_bw client + server on real hosts with DOCA installed plus a CUDA Toolkit matched to the DOCA install, and a GPU + IB device pair on the host's PCIe topology. Concretely:

• Measuring sustained kernel-initiated RDMA WRITE bandwidth between two hosts (or a host and a BlueField DPU) with the GPUNetIO surface.
• Deciding whether the GPUNetIO path is the right runtime surface for a class of workload vs the GPI programming surface (the doca-gpi library — doca/tools/ ships no GPI benchmark binary) or the classic CPU-initiated perftest path.
• Capturing a documented baseline (build + invocation + DOCA version + GPU + NIC + as-deployed environment + numbers) for later regression hunts.
• Diagnosing a build / link / run failure that surfaces the GPUNetIO + RDMA bring-up sequence under this tool's shipped scaffolding.

Do not load this skill for general DOCA orientation, library API work, or installation. For those, use doca-public-knowledge-map, ../../libs/doca-gpunetio/SKILL.md, or doca-setup. Do not load it for application-level end-to-end throughput either — this benchmark measures the WR-submission path through GPUNetIO, not the user's full pipeline.

What this skill provides

This is a thin loader. Substantive material lives in two companion files:

• CAPABILITIES.md — what the tool measures (the sustained-WRITE-BW primitive driven by a client-side CUDA kernel through doca-gpunetio), the runtime-surface selection rule (GPUNetIO vs GPI vs CPU-initiated), the GPU-NIC pairing precondition, the throughput-decomposition guide (GPU compute occupancy vs NIC issue rate vs link saturation), the version overlay (DOCA .pc PLUS CUDA Toolkit), the layered error taxonomy (config-syntax / build-time / GPU-NIC- pairing / GPUNetIO-lifecycle / RDMA-connection / measurement-soundness / version / cross-cutting), the observability surface (stdout report, DOCA log levels, OOB-socket exchange), and the safety overlay (the "GPU-side handle is a credential" rule from doca-gpunetio; the cross-cutting hardware-safety meta-policy).
• TASKS.md — step-by-step workflows for the in-scope task verbs: install (preconditions — DOCA install, CUDA Toolkit, GPU + NIC pair, OOB connectivity), configure (build-tree under doca/tools/gpunetio_ib_write_bw/ and the meson build wrapping the shipped DOCA), build (the meson setup + meson compile pattern from the public DOCA build documentation), modify (do not patch the shipped tool source; modify the invocation and the surrounding environment instead), run (smoke- before-bulk; client + server bring-up order; reading the per-iteration report), test (the eval loop — steady-state, NUMA placement, NIC saturation cross- check), debug (walk the error taxonomy layer by layer), use (how a BW result feeds a class-of- workload decision), plus a Deferred task verbs block routing out-of-scope questions.

The skill assumes a host where DOCA is already installed, a CUDA Toolkit matched to the install is present, and the operator has whatever privileges the public install profile expects for binding a doca_dev, a doca_gpu, and an OOB TCP socket.

What this skill deliberately does not ship

This skill is agent guidance, not a samples or scripts bundle. To keep the boundary clean, it deliberately does not contain — and pull requests should not add:

• Specific flag strings or expected throughput numbers beyond what the tool's shipped --help and main.c ARGP registration establish. The flag surface is small (device name, GPU PCIe