Skip to content

I found a nice little performance improvement when reading the data from vector + memcpy - #473

Closed
eokeeffe wants to merge 31 commits into
KomputeProject:masterfrom
eokeeffe:master
Closed

I found a nice little performance improvement when reading the data from vector + memcpy#473
eokeeffe wants to merge 31 commits into
KomputeProject:masterfrom
eokeeffe:master

Conversation

@eokeeffe

@eokeeffe eokeeffe commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

I'm doing up some image processing pipelines and I found a large amount of the time I was spending with reading back the data from the GPU. I found that adding eHostCohorent and eHostCached reduced my read times from ~350ms to just 62ms.

Thought this might be useful for others to have as well.

koubaa and others added 30 commits February 4, 2026 10:36
@koubaa koubaa evanokeeffe
add datatype enum to python module (#404)
798ba17 
* add datatype enum to python module

Signed-off-by: koubaa <koubaa@github.com>

* add datatype enum to python module

Signed-off-by: koubaa <koubaa@github.com>

* add doc

Signed-off-by: koubaa <koubaa@github.com>

---------

Signed-off-by: koubaa <koubaa@github.com>
Co-authored-by: koubaa <koubaa@github.com>
Signed-off-by: evanokeeffe <evan.okeeffe@tas-2.com>
@ThePseudo evanokeeffe
Add noexcept to all constructors to explicitly state no exception thr…
82bc222 
…own #44 (#411)

* Sequence con/destructor is noexcept

Signed-off-by: Andrea Calabrese <andrea.calabrese@amarulasolutions.com>
Signed-off-by: Andrea Calabrese <andrea1995.c@live.com>

* Operations are noexcept

Signed-off-by: Andrea Calabrese <andrea.calabrese@amarulasolutions.com>
Signed-off-by: Andrea Calabrese <andrea1995.c@live.com>

* Algorithm is noexcept

Signed-off-by: Andrea Calabrese <andrea.calabrese@amarulasolutions.com>
Signed-off-by: Andrea Calabrese <andrea1995.c@live.com>

---------

Signed-off-by: Andrea Calabrese <andrea.calabrese@amarulasolutions.com>
Signed-off-by: Andrea Calabrese <andrea1995.c@live.com>
Signed-off-by: evanokeeffe <evan.okeeffe@tas-2.com>
@ThePseudo evanokeeffe
Remove copy operations from classes in Kompute (#412)
08a5261 
* Operations are non-copy

As for Issue #20, operations should be non-copy. Instead of creating a
base NonCopyable class, I prefer to keep this inside the classes
themselves, so that we do not create new inherited structures.

Signed-off-by: Andrea Calabrese <andrea.calabrese@amarulasolutions.com>
Signed-off-by: Andrea Calabrese <andrea1995.c@live.com>

* Make Algorithm uncopyable

Same as previous commit

Signed-off-by: Andrea Calabrese <andrea.calabrese@amarulasolutions.com>
Signed-off-by: Andrea Calabrese <andrea1995.c@live.com>

* Image is non-copy

Signed-off-by: Andrea Calabrese <andrea.calabrese@amarulasolutions.com>
Signed-off-by: Andrea Calabrese <andrea1995.c@live.com>

* Manager is non-copy

Signed-off-by: Andrea Calabrese <andrea.calabrese@amarulasolutions.com>
Signed-off-by: Andrea Calabrese <andrea1995.c@live.com>

* Memory is non-copy

Signed-off-by: Andrea Calabrese <andrea.calabrese@amarulasolutions.com>
Signed-off-by: Andrea Calabrese <andrea1995.c@live.com>

* Sequence is non-copy

Signed-off-by: Andrea Calabrese <andrea.calabrese@amarulasolutions.com>
Signed-off-by: Andrea Calabrese <andrea1995.c@live.com>

* Tensor is non-copy

Signed-off-by: Andrea Calabrese <andrea.calabrese@amarulasolutions.com>
Signed-off-by: Andrea Calabrese <andrea1995.c@live.com>

---------

Signed-off-by: Andrea Calabrese <andrea.calabrese@amarulasolutions.com>
Signed-off-by: Andrea Calabrese <andrea1995.c@live.com>
Signed-off-by: evanokeeffe <evan.okeeffe@tas-2.com>
@axsaucedo evanokeeffe
Fix noexcept missing from OpMemoryBarrier (#413)
fcf0f34 
Signed-off-by: Alejandro Saucedo <alejandro.saucedo@zalando.de>
Signed-off-by: evanokeeffe <evan.okeeffe@tas-2.com>
@axsaucedo evanokeeffe
414 support for 1 4 headers (#415)
9fb1876 
* Added guard for vk 1.4 headers

Signed-off-by: Alejandro Saucedo <alejandro.saucedo@zalando.de>

* CI test for 1.4 compatibility

Signed-off-by: Alejandro Saucedo <alejandro.saucedo@zalando.de>

CI test for 1.4 compatibility

Signed-off-by: Alejandro Saucedo <alejandro.saucedo@zalando.de>

CI test for 1.4 compatibility

Signed-off-by: Alejandro Saucedo <alejandro.saucedo@zalando.de>

CI test for 1.4 compatibility

Signed-off-by: Alejandro Saucedo <alejandro.saucedo@zalando.de>

CI test for 1.4 compatibility

Signed-off-by: Alejandro Saucedo <alejandro.saucedo@zalando.de>

CI test for 1.4 compatibility

Signed-off-by: Alejandro Saucedo <alejandro.saucedo@zalando.de>

Updated cast to match vk namespace

Signed-off-by: Alejandro Saucedo <alejandro.saucedo@zalando.de>

Updated cast to match vk namespace

Signed-off-by: Alejandro Saucedo <alejandro.saucedo@zalando.de>

* Updated debugMessageCallback func to vk:: ns

Signed-off-by: Alejandro Saucedo <alejandro.saucedo@zalando.de>

* Updated debugMessageCallback func to vk:: ns

Signed-off-by: Alejandro Saucedo <alejandro.saucedo@zalando.de>

---------

Signed-off-by: Alejandro Saucedo <alejandro.saucedo@zalando.de>
Signed-off-by: evanokeeffe <evan.okeeffe@tas-2.com>
@jsrgb evanokeeffe
Fix syntax error (MSVC) in Manager.cpp (#418)
b4f74cb 
Change VKAPI_PTR to VKAPI_ATTR

Tested only on:
MSVC version: `Microsoft (R) C/C++ Optimizing Compiler Version 19.43.34808 for
x86`
VulkanSDK version: 1.4.309.0
Latest kompute commit: 299b11f

Signed-off-by: jamorphy <maps@jax2.net>
Signed-off-by: evanokeeffe <evan.okeeffe@tas-2.com>
@zlaazlaa evanokeeffe
Fix compile error in android (#423)
63b84b3 
Signed-off-by: zlaazlaa <2889827787@qq.com>
Signed-off-by: evanokeeffe <evan.okeeffe@tas-2.com>
@zfergus evanokeeffe
Make spdlog and fmt mutually exclusive and support -Werror=missing-br…
5ad7972 
…aces (#419)

* Update bin2h.cmake to avoid -Werror=missing-braces

Signed-off-by: zachferguson <zach.ferguson@clo3d.com>
Signed-off-by: Zachary Ferguson <zach.ferguson@clo3d.com>

* Use the fmt inside spdlog if using spdlog

Signed-off-by: zachferguson <zach.ferguson@clo3d.com>
Signed-off-by: Zachary Ferguson <zach.ferguson@clo3d.com>

* Fix python build by linking fmt when KOMPUTE_OPT_LOG_LEVEL_DISABLED

Signed-off-by: Zachary Ferguson <zach.ferguson@clo3d.com>

---------

Signed-off-by: zachferguson <zach.ferguson@clo3d.com>
Signed-off-by: Zachary Ferguson <zach.ferguson@clo3d.com>
Signed-off-by: evanokeeffe <evan.okeeffe@tas-2.com>
@robquill evanokeeffe
Add missing eDeviceAndHost MemoryType to the Python Bindings (#424)
9dd9c71 
* Add eDeviceAndHost to Python bindings for MemoryTypes.

Update docstrings so that the build works.

Signed-off-by: Robert Quill <robert.quill@imgtec.com>

* Typo fix

Signed-off-by: Robert Quill <robert.quill@imgtec.com>

---------

Signed-off-by: Robert Quill <robert.quill@imgtec.com>
Signed-off-by: evanokeeffe <evan.okeeffe@tas-2.com>
@zlaazlaa evanokeeffe
feat: Add log library linking to Android build (#428)
1304c0f 
Signed-off-by: zlaazlaa <2889827787@qq.com>
Signed-off-by: evanokeeffe <evan.okeeffe@tas-2.com>
@axsaucedo evanokeeffe
Fix python SpecConst and PushConst bugs (#430)
3160b6a 
* Fix push const inconsistent type bug

Signed-off-by: Alejandro Saucedo <alejandro.saucedo@zalando.de>

* Fix push const bug

Signed-off-by: Alejandro Saucedo <alejandro.saucedo@zalando.de>

* Added test to consts

Signed-off-by: Alejandro Saucedo <alejandro.saucedo@zalando.de>

* Added tests for const types@

Signed-off-by: Alejandro Saucedo <alejandro.saucedo@zalando.de>

* Added test to consts

Signed-off-by: Alejandro Saucedo <alejandro.saucedo@zalando.de>

* Added test to consts

Signed-off-by: Alejandro Saucedo <alejandro.saucedo@zalando.de>

* Added test to consts

Signed-off-by: Alejandro Saucedo <alejandro.saucedo@zalando.de>

---------

Signed-off-by: Alejandro Saucedo <alejandro.saucedo@zalando.de>
Signed-off-by: evanokeeffe <evan.okeeffe@tas-2.com>
@n-jay evanokeeffe
Update Python Matmul examples (#434)
d76e86b 
* feat: Add utility function to compile GLSL source to SPIR-V

Signed-off-by: n-jay <nuwanjayawardene@gmail.com>

* refactor: Replace kp.Shader.compile_source with utility function compile_source

Signed-off-by: n-jay <nuwanjayawardene@gmail.com>

---------

Signed-off-by: n-jay <nuwanjayawardene@gmail.com>
Signed-off-by: evanokeeffe <evan.okeeffe@tas-2.com>
@TinyTinni evanokeeffe
update internal gtest to v1.17.0 (#437)
77a0a0d 
Signed-off-by: Matthias Möller <m_moeller@live.de>
Signed-off-by: evanokeeffe <evan.okeeffe@tas-2.com>
@robquill evanokeeffe
Fix incorrect creation of python arrays in Tensor.data (#440)
2488e92 
Previously the Python array would just be filled with the first element
of the data repeated to fill the array.

Fixes a number of the Python tests.

Signed-off-by: Robert Quill <robert.quill@imgtec.com>
Signed-off-by: evanokeeffe <evan.okeeffe@tas-2.com>
@axsaucedo evanokeeffe
Updated to pybind 3.0.0 (#431)
ad03e3b 
Signed-off-by: Alejandro Saucedo <alejandro.saucedo@zalando.de>
Signed-off-by: evanokeeffe <evan.okeeffe@tas-2.com>
@axsaucedo evanokeeffe
Clarify status of llama.cpp in README (#446)
38a1a81 
Updated description for llama.cpp to indicate it is now decommissioned.

Signed-off-by: Alejandro Saucedo <alejandro.saucedo@zalando.de>
Signed-off-by: evanokeeffe <evan.okeeffe@tas-2.com>
evanokeeffe
adding kompute example using debug printf
7f1edae 
Signed-off-by: evanokeeffe <evan.okeeffe@tas-2.com>
evanokeeffe
adding link to the draft article
67a0a84 
Signed-off-by: evanokeeffe <evan.okeeffe@tas-2.com>
evanokeeffe
Updating the readme for the debug statement example
9cc8eca 
Signed-off-by: evanokeeffe <evan.okeeffe@tas-2.com>
evanokeeffe
Adding in the example shader, gitignore targets it so needed to force in
d2e316b 
Signed-off-by: evanokeeffe <evan.okeeffe@tas-2.com>
evanokeeffe
Modifying the comment to include bytes in the comment on message length
2d69587 
Signed-off-by: evanokeeffe <evan.okeeffe@tas-2.com>
evanokeeffe
Correcting the readme for spelling mistakes in vulkan_ext_printf
1030b8b 
Signed-off-by: evanokeeffe <evan.okeeffe@tas-2.com>
@TinyTinni evanokeeffe
Fix family queue selection in tests (#448)
19caffe 
* checks if given family queue has compute capability

Signed-off-by: Matthias Möller <m_moeller@live.de>

* test will search for queues with compute capabilities

Signed-off-by: Matthias Möller <m_moeller@live.de>

---------

Signed-off-by: Matthias Möller <m_moeller@live.de>
Signed-off-by: evanokeeffe <evan.okeeffe@tas-2.com>
@TinyTinni evanokeeffe
removes compile definition defining for spdlog target (#449)
8fbcef0 
spdlog is not build when the CMake option
"KOMPUTE_OPT_USE_BUILT_IN_SPDLOG" is set to false. When activating
spdlog for logging via CMake option "KOMPUTE_OPT_USE_SPDLOG", CMake will
deliver an error, as the current CMake file tries to add a definition to
the spdlog target.

Defining the compiler definition "SPDLOG_ACTIVE_LEVEL" has no effect on
spdlog itself, it just has an effect on spdlog target consumers.
Therefore, it can be removed so that the project can be build with the
mentioned CMake options.

Signed-off-by: Matthias Möller <m_moeller@live.de>
Signed-off-by: evanokeeffe <evan.okeeffe@tas-2.com>
evanokeeffe
adding kompute example using debug printf
958a55e 
Signed-off-by: evanokeeffe <evan.okeeffe@tas-2.com>
evanokeeffe
adding link to the draft article
fb328f7 
Signed-off-by: evanokeeffe <evan.okeeffe@tas-2.com>
evanokeeffe
Updating the readme for the debug statement example
4d5f5b1 
Signed-off-by: evanokeeffe <evan.okeeffe@tas-2.com>
@eokeeffe
Merge branch 'master' into master
466aaed
@eokeeffe
Merge branch 'KomputeProject:master' into master
6c658f9
evanokeeffe
This helped with a vector and memcpy issue I was having, was spending…
0d2f96f 
… nearly 350ms just on reads for an image processing pipeline. With this simple modification I was able to drop that to 62.1ms on my machine
@eokeeffe eokeeffe changed the title I found a nice little performance improvement when rad the data from vector + memcpy I found a nice little performance improvement when reading the data from vector + memcpy Sep 8, 2026
@axsaucedo

axsaucedo commented Sep 8, 2026

Copy link
Copy Markdown
Member

Pretty cool - thanks, could you have a look at the CI? Wonder if this there is something here that is breaking across

@axsaucedo

axsaucedo commented Sep 8, 2026

Copy link
Copy Markdown
Member

Also it seems your commits are pulling quite a lot of other commits

@eokeeffe

eokeeffe commented Sep 8, 2026

Copy link
Copy Markdown
Contributor Author

Also it seems your commits are pulling quite a lot of other commits

Yes, I pulled the latest from master and this happened. I'll close and open a clean PR for you.

@eokeeffe eokeeffe closed this Sep 8, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

10 participants

@eokeeffe @axsaucedo @koubaa @ThePseudo @jsrgb @zlaazlaa @zfergus @robquill @n-jay @TinyTinni