Rendering a hardware frame and encoding it

[maxsupermanhd]

Hi, I am looking for hardware accelerated solutions of synthetic image rendering and encoding, so far go-astiav looks like does hardware encoding fairly well and I am now wondering from what libraries would I be able to render stuff on the texture (with shaders for example) and then be able to use that texture in a codec encoder.

Also a thought of "is this a good idea" comes to mind and I can't really answer it, should I just write a purely C wrapper around whole rendering chain and just feed it data and get encoded frames back?

Much thanks for your hard work.

[asticode]

I don't know much about rendering frames on texture (especially using shaders) but with go-astiav you can "load" data into an allocated frame and do whatever you want with this frame (encode it for instance). Therefore if you manage to get the frame data, you can use go-astiav to encode it.

But if you decide to use go-astiav for your use case, bare in mind that you'll need to find a GO solution for the rendering part (which might exist). Writing all your flow in pure C is not a bad idea, if you go this route I'm not sure you still need to use GO.

[maxsupermanhd]

I don't know much about rendering frames on texture

It is fairly easy, you basically tell sdl/glfw/whatever to instead of target being texture that is a window (window has it's own texture) I make a targetable texture and now you render there and all the render calls will result in stuff being drawn to the target texture. Reasoning for that is my process is going to be ran on the server and I won't have a window at all so it will be all just in-gpu rendering.

with go-astiav you can "load" data into an allocated frame

Yes I seen the bindings but would that invoke copy of a buffer from cpu to the gpu? Seems like a very bad thing to render my frame on the gpu then ship it to the cpu only for it to be returned back to the gpu for encoding. Found an SDL test to do what I described but in other direction, decode on ffmpeg and then convert it to SDL texture here: https://github.com/libsdl-org/SDL/blob/main/test/testffmpeg.c#L1046

Honestly it might not be a huge concern given my frames are not 4k or whatever (actually fairly small size, around 500x500) but would be nice to eliminate such redundant memory moving.

I would assume it would be much easier (given currently I only have basic image rendering on the cpu in golang) to write everything straight in c just to cut down on amount of bindings and glue that is needed. Unfortunately using golang is fairly important because there is a lot of preparation of the data to be rendered and http and so on that I don't feel like dealing with in c.

Thank you.

[maxsupermanhd]

On further digging found following stackoverflow answers that describe almost exactly what I want (only thing I had to do is some search engine magic and remove golang keyword...)

https://stackoverflow.com/a/14324292
https://stackoverflow.com/a/36488003 (especially this one)

but the problem is that all of them do copy buffer to ram and then back to the vram (in examples above it uses glReadPixels to move rendered stuff back to system memory and then feeds ffmpeg with it, even after some conversion work too)