Conal Mullan 99d9296d96 FIX: image-edit Modal app OOMs on A100-40GB — request A100-80GB ...

gpu="A100" resolves to the 40GB variant. QwenImageEditPlusPipeline loads in
bfloat16 and goes fully resident via .to("cuda") at ~38.8GiB, leaving ~72MiB
free, so the app raised torch.OutOfMemoryError on the first allocation after
load and returned HTTP 500 on every request.

Not fragmentation: the failure reported only 198MiB reserved-but-unallocated,
so the PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True hint in torch's error
text does not apply here — the model genuinely fills the card.

A100-80GB matches how the same model is provisioned elsewhere: docker/modal-ltx2
already requests A100-80GB, and the RunPod deployment of Qwen-Image-Edit runs on
AMPERE_80.

Verified end-to-end after redeploy: 1024x1024 edit returned in 92.9s total /
16.5s inference, no OOM.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019s6SvNiz9a8XpD8r2cQTiq

2026-08-26 14:50:47 +01:00

..

app.py

FIX: image-edit Modal app OOMs on A100-40GB — request A100-80GB

2026-08-26 14:50:47 +01:00