Failed to map scanout buffer on Wayland: EVDI-owned DMA-BUF returns -EOPNOTSUPP from vmap

[TigerWolf]

I'm seeing a black external display when using an EVDI-based USB display
adapter under GNOME/Wayland. (im using this one from Wavlink - https://www.wavlink.com/en_us/drivers/3dea44705bc1.html?name=Displaylink%20Based%20Dock%20or%20Adapter&category_id=7&level=2 )

The display is detected correctly, the hardware cursor is visible and moves correctly, but the rest of the screen is black.

The kernel log repeatedly reports:

evdi: [E] evdi_painter_grabpix_ioctl:1139 Failed to map scanout buffer

I traced this further and found that dma_buf_vmap_unlocked() is returning
-EOPNOTSUPP (-95) for a DMA-BUF whose owner is EVDI itself.

Environment

• EVDI: 1.15.0
• Kernel: Ubuntu 7.0.0-30-generic
• Desktop: GNOME 50
• Display server: Wayland
• Primary VM GPU: virtio-gpu (2D)
• USB display driver: Silicon Motion SMI USB Display v2.24.8.0
• USB device: Silicon Motion SMI USB Display
• Environment: QEMU/KVM VM

Symptoms

The USB display:

• is detected
• reports EDID correctly
• accepts a display mode
• shows the hardware cursor
• remains black for the main framebuffer

EVDI reports:

evdi: [I] (card2) Connector state: connected
evdi: [I] (card2) Edid property set
evdi: [I] (card2) Notifying display power state: on
evdi: [I] (card2) Notifying mode changed: 1024x768@60; bpp 32; pixel format XR24 little-endian
evdi: [E] evdi_painter_grabpix_ioctl:1139 Failed to map scanout buffer

The kernel also warns in:

dma_buf_vmap
dma_buf_vmap_unlocked
evdi_gem_vmap
evdi_painter_grabpix_ioctl

Investigation

I added logging around this call in evdi_gem_vmap():

ret = evdi_dma_buf_vmap_unlocked(
    obj->base.import_attach->dmabuf, &map);

This showed:

evdi: dma_buf_vmap_unlocked failed: ret=-95 owner=evdi

So the failing DMA-BUF is EVDI-owned rather than coming from virtio-gpu.

Looking at gem_obj_funcs, EVDI supports PRIME export:

.export = drm_gem_prime_export,
.get_sg_table = evdi_prime_get_sg_table,

but does not provide .vmap / .vunmap callbacks.

As a result, when the EVDI-owned GEM object comes back through the DMA-BUF
vmap path, the generic DRM GEM DMA-BUF vmap operation returns
-EOPNOTSUPP.

Experimental fix

I added .vmap and .vunmap implementations to EVDI's
drm_gem_object_funcs.

The vmap implementation pins the existing EVDI pages, maps them using
vmap(), and returns that mapping using iosys_map_set_vaddr().

After adding:

.vmap = evdi_gem_object_vmap,
.vunmap = evdi_gem_object_vunmap,

the external display immediately works correctly and the
Failed to map scanout buffer errors disappear.

I can provide the complete patch/diff.

Im not sure if my implementation is the correct fix here and im reporting this moreso because I experienced this issue, found a fix for it but I dont have the experience with this library to know if it is the correct way to fix it.

Possibly related

#521 - EVDI DisplayLink Driver Ubuntu 25.04 Scanout Buffer issue
#435 - Failed to map scanout buffer 2 monitors

[TigerWolf]

Should I post a Pull Request for this one? Happy to provide the fix that worked for me if that would be valuable.