Intel Arc B70 system RAM OOM: why VRAM is not enough

Two 32-GB B70s and 64 GB of system RAM can still run out of host memory while a model fits in VRAM. We have encountered this repeatedly in our B70 work. The useful question is where memory grows: during checkpoint loading, device allocation, quantization, or actual inference.

There is a documented Intel multi-device allocation problem behind some of these cases. Other cases involve ordinary CPU copies or offload. Identify the path before changing settings: a GPU-memory workaround can increase demand on system RAM.

Which memory ran out?

An exit code of 137 alone does not establish an OOM: it indicates a SIGKILL-style termination. Check the runtime’s termination record and kernel or cgroup evidence. On cgroup v2, memory.events includes oom and oom_kill; memory.max identifies the configured hard limit. A service can hit that limit without exhausting the entire machine. Linux cgroup documentation

Why a model that fits in VRAM can exhaust RAM

The weight file describes one artifact. The loading path may temporarily hold source weights, a destination model and conversion buffers. Transformers documents these loading costs and provides memory-conscious initialization and placement mechanisms. CPU offload deliberately keeps work or weights in host memory; automatic placement can include CPU or disk, so inspect the resulting device map. Transformers loading documentation

There is also a more surprising path. A multi-device SYCL context can produce host-backed GTT allocations alongside device allocations. An upstream B70 report describes approximately one host-memory copy per GPU allocation. GTT counters describe driver-managed memory; they are not another VRAM capacity figure, and process RSS does not account for every driver allocation. Intel compute-runtime issue #980

With as much installed VRAM as host RAM, that behavior can consume the host’s working margin before allowing for the operating system and application. Treat matching RAM and VRAM capacities as a reason to measure, not a sizing rule that guarantees success.

Fixes by software stack

llama.cpp SYCL: check the allocation path

Our PR #21597, merged May 14, 2026, introduced direct Level Zero device allocation and transfer changes for this failure mode. The original dual-B70 report compared a Qwen3.5-27B Q4_K_M workload that exhausted roughly 60,034 MiB of host RAM with a changed path using about 6.7 GiB. Those are historical measurements from the reported environment, not a current universal requirement.

Check whether your exact build includes the fix and uses the intended path. The implementation includes fallback behavior, so a commit date alone cannot prove which allocator executed. Keep the build and loaded runtime versions with your evidence. Applying an old patch again to a current build is not the next step.

PyTorch XPU: separate processes need separate device visibility

In our retained Wan video-generation investigation, dividing sequential stages into one process per card reduced GTT pressure and allowed a previously failing workload to run. Each child selected its card before initializing the GPU runtime; saved intermediate state connected the stages. This is a historical application-specific workaround, not a drop-in replacement for every multi-GPU algorithm. That retained local report has not been rerun as a full video workflow. The fresh allocation probe below tests the visibility mechanism separately.

The key distinction is visibility. Starting two processes is insufficient if both still initialize a context containing both cards. For independent jobs, select one physical device before each process imports or initializes its XPU runtime, and verify the physical mapping. An isolated worker may call its selected card device zero even when it is the second physical GPU.

vLLM XPU: per-worker isolation is an active upstream lead

vLLM issue #53822 reports excessive host-backed memory under tensor parallelism even though workers are separate processes. The proposed remedy sets each child’s device visibility before it starts and handles the resulting local device numbering. The report also identifies compile-cache and speculative-decoding complications.

PR #56012, opened September 9, remained open when reviewed September 11. Its author reports B60 tests, including tensor and data parallel execution. Credit belongs to those upstream contributors; their results do not establish compatibility with every B70 image or our DFlash setup.

Do not paste a single global device mask into a two-card server and assume it implements per-worker isolation. Use a revision or recipe that explicitly handles worker startup, ordinal mapping and cache state, then retest outputs as well as memory.

Loading and quantization: check what moves onto the CPU

For vLLM loading peaks, inspect whether your selected version and executor support --max-parallel-loading-workers; the documented purpose is limiting simultaneous loading work to avoid host OOM with large tensor-parallel models. It addresses loading pressure, not every driver allocation problem. vLLM 0.21 parallel configuration

Intel’s versioned vLLM container notes describe VLLM_OFFLOAD_WEIGHTS_BEFORE_QUANT=1 as moving weights to the CPU before quantization to relieve device-memory pressure. That consumes host capacity; it is not a general system-RAM fix. The cited repository is now archived, so use it to understand that specific option, then inspect your actual image’s implementation. Intel vLLM 0.14.1 XPU notes

For Transformers, use its supported memory-conscious loading path and compatible pre-quantized checkpoints where appropriate. Record actual dtype and device placement. A smaller final model does not prove its conversion can fit the machine. Transformers loading documentation

Fresh B70 test: the same allocation, different visibility

On September 11, we allocated and filled a 4-GiB tensor on each B70: 8 GiB total in every layout. We repeated each layout three times in fresh containers. Each repetition returned the same resident-GTT totals shown here.

Environment: Ubuntu 26.04.1, kernel 7.0.0-31-generic, compute-runtime 26.27.39122.11-0, host Level Zero loader 1.28.2-2, and PyTorch 2.11.0+xpu in the pinned container recorded below. Isolation used a separate ZE_AFFINITY_MASK for each child before importing PyTorch.

These are driver counters summed across unique device/client identities, with runtime overhead included. They are not a measurement of unique physical host RAM. Each child sampled after allocation and held its tensors for eight seconds; snapshots were asynchronous. This small allocation test does not validate model inference, distributed communication, DFlash, Windows, or other Arc cards.

Download the results and exact environment and allocation probe. The probe performs real GPU allocations when run: use it only in a prepared PyTorch XPU environment with sufficient headroom. It accepts --devices 0,1 for the shared-process case and --devices 0 for a child whose visibility has already been restricted. Reproducing a distributed server requires its own worker-startup integration.

Collect a useful memory record

Capture a baseline, loading peak, first request, warm requests and post-shutdown recovery. Start with these read-only Linux checks:

free -h
vmstat 1 5

Use MemAvailable to assess host headroom. Low MemFree by itself can reflect reclaimable caches. Keep swap activity and process memory alongside it; resident process memory, virtual address space and total machine usage are different quantities. Linux /proc documentation

For the actual GPU worker, inspect its DRM file-descriptor records. Replace the example PID with your worker’s process ID:

GPU_WORKER_PID=12345
ps -p "$GPU_WORKER_PID" -o pid,comm,rss,vsz
rg -H '^drm-(driver|pdev|client-id|total-|resident-|shared-)' \
  /proc/"$GPU_WORKER_PID"/fdinfo/*

Permissions may restrict these records. Match the physical device and DRM client identity, preserve units, and deduplicate repeated descriptors for the same client. Shared buffers can appear in multiple clients: do not add every counter to RSS and label the result physical RAM. Missing fields mean unavailable telemetry. DRM client accounting documentation

If GTT grows with device allocations, test a small, bounded allocation before attempting another full model load. Change one factor at a time. A useful result includes the runtime version, selected devices, the same allocation or workload, before/after measurements and correct outputs.

What not to treat as a universal fix

Lower context and concurrency can restore device headroom, but do not repair a loader that duplicates weights. CPU offload can make host pressure worse. Swap may delay a failure while increasing latency. Clearing caches is not evidence that an allocator defect is resolved.

Measure the highest-demand stage of the workload before deciding that more RAM is required. Leave operating headroom and stop a diagnostic before deliberately exhausting the workstation. For a reproducible report, include the software-stack record and which memory counter actually failed.

If you have a matching case or a verified fix, a useful contribution is a small reproduction with versions, device mapping and memory snapshots. Keep private prompts and identifying process details out of public logs. This guide will grow through dated evidence and corrections.