DiffDRR Generation by considering the intrinsic and extrinsic matrices

[Darshan-KN]

How to generate a Digitally Reconstructed Radiograph (DRR) by considering a C-arm X-ray system as a camera, incorporating the projection matrix, and using the intrinsic and extrinsic matrices in the DiffDRR generation process for the proper pose of the DRR generation.
I have the projection matrix with its intrinsic and extrinsic matrices from that I want to generate the DRR.

[eigenvivek]

Hi @Darshan-KN, this should be fairly straightforward with DiffDRR. Basically, you're going to want to decompose your intrinsic matrix into the geometric components that DiffDRR accepts (focal length, pixel spacing, principal point). Once you initialize a diffdrr.drr.DRR object with those parameters, you can pass your extrinsic matrix to the renderer and generate a DRR.

DiffDRR has a function for parsing intrinsic matrices: https://vivekg.dev/DiffDRR/api/utils.html#parse_intrinsic_matrix

Let me know if you run into any further issues.

[Darshan-KN]

Hai sir,

import torch
from diffdrr.data import read
import matplotlib.pyplot as plt
from diffdrr.drr import DRR
from diffdrr.visualization import plot_drr

Path to your CT volume (could be a DICOM folder or NIfTI file)

ct_volume_path = "ct path"

Load the volume using the read function

subject = read(
volume=ct_volume_path, # Path to your CT data
orientation='AP', # Specify the view orientation (AP, PA, etc.)
bone_attenuation_multiplier=1.0 # Optional: Adjust bone contrast
)

Initialize the DRR generator

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
drr = DRR(
subject, # The loaded CT volume(i.e., the C-arm's focal length)
sdd=1200.0, # Source-to-detector distance
height=200, # DRR image height (Height of the rendered DRR)
delx=1.5, # Pixel spacing in mm (X-axis pixel size)
width = 200, # Width of the rendered DRR (default to height)

).to(device)

Set the camera pose with rotations and translations

rotations = torch.tensor([[-61.41059158336708, 0.04245468150454712, 0.07009996937525848]], device=device)
translations = torch.tensor([[140.0, -732.0, 547.0]], device=device)

Generate DRR using the camera pose

img = drr(rotations, translations, parameterization="euler_angles", convention="ZYX")

Visualize the DRR

plot_drr(img, ticks=False)
plt.show()

As mentioned in the above code, I took the rotation matrix and converted it into Euler angles. I applied all the different Euler angles with the translation matrix I have, but I cannot get the DRR is this method I tried is correct? If not, please suggest what to do. I have a rotation and translation matrix. From this, I want to project the DRR so that it looks similar to the X-ray Image I have.

[eigenvivek]

Hi @Darshan-KN can you please attach the image produced by DiffDRR and the real X-ray you're trying to produce?

In the meantime, here are some things to consider:

1. diffdrr.data.read automatically repositions the CT such that its isocenter is moved to the world origin. Your translations look fairly large, so this default behavior might not be what you wanted. There's a flag to turn off this default in the the read function.
2. It looks like your rotation is in degrees. DiffDRR expects rotations to be in radians.
3. You can wrap your 4x4 extrinsic matrix with a diffdrr.pose.RigidTransform class and pass that directly to the renderer. That way you don't need to convert to Euler angles.

[Darshan-KN]

Hai sir,

I have 4x4 extrinsic matrix is

From this, I want to generate the DiffDRR. Can you tell me how to do it and what the proper coding is for this?

[eigenvivek]

Let extrinsic be a torch tensor containing the above. Then you can do:

from diffdrr.pose import RigidTransform

pose = RigidTransform(extrinsic)
img = drr(pose)

The introduction in the documentation provides many examples of different ways to pass a pose to the drr: https://vivekg.dev/DiffDRR/tutorials/introduction.html

[Darshan-KN]

Hello Sir,

As you mentioned earlier, I followed the steps and successfully generated the DRR. However, I realized that I do not have the following parameters:

sdd (Source-to-Detector Distance): This represents the focal length of the C-arm, i.e., the distance from the X-ray source to the detector.
delx (X-axis Pixel Size): This specifies the size of each pixel along the X-axis in millimeters.
dely (Y-axis Pixel Size): This is the pixel size along the Y-axis. If it's not provided, it is set to the value of delx.
p_subsample (Proportion of Pixels to Subsample): This is the fraction of pixels to randomly subsample, potentially used for optimization purposes.
patch_size (Size of Patches): This defines the size of patches used to render the DRR in segments, which can be useful for large images or to speed up rendering.
Where can I obtain these parameters?
For different C-arms these parameters is different?

[eigenvivek]

Those parameters are different for each C-arm. Also, most C-arms give the ability to change those parameters. Do you have any real DICOMs from the C-arm you're interested in simulating? You can read ground truth values for those parameters from the DICOM header.

Some standard parameters are sdd=1020, delx=0.194, and dely=0.194.

p_subsample and patch_size are properties of the renderer, not the physical C-arm.

[Darshan-KN]

Hai sir,

The code to generate the diffdrr is

import torch
from diffdrr.data import read
import matplotlib.pyplot as plt
from diffdrr.drr import DRR
from diffdrr.visualization import plot_drr
from diffdrr.pose import RigidTransform # Import RigidTransform for the extrinsic matrix

Path to your CT volume

ct_volume_path = "Path"

Load the volume using the read function

subject = read(
volume=ct_volume_path, # Path to your CT data
orientation='AP', # Specify the view orientation (AP, PA, etc.)
bone_attenuation_multiplier=1.0 # Optional: Adjust bone contrast
)

Initialize the DRR generator

device = torch.device("cpu")
drr = DRR(
subject, # The loaded CT volume
sdd=800, # Source-to-detector distance
height=1024, # DRR image height
delx=2.5,
dely=2.5, # Pixel spacing in mm (X-axis pixel size)
width=1024, # Width of the rendered DRR
).to(device)

Extrinsic matrix as a torch tensor

extrinsic = torch.tensor([
[-0.976094633, 0.217345963, 0.0000174219827, 112.17262091],
[-0.217345958, -0.976094589, -0.000295290739, -708.01915233],
[-0.0000471747468, -0.000292018302, 0.999999956, 525.08483553],
[0.0, 0.0, 0.0, 1.0]
], device=device)

Create a RigidTransform from the extrinsic matrix

pose = RigidTransform(extrinsic)

Generate DRR using the extrinsic matrix pose

img = drr(pose)

Visualize the DRR

plot_drr(img, ticks=False)
plt.show()

from this, I am getting the DRR like

I want the generated DRR like

can you please rectify this what can I do to get the proper DRR is similar to x-ray

[eigenvivek]

height=1024, # DRR image height
delx=2.5,

is your issue - this means your DRR detector plane is 2.5 m x 2.5 m

you probably have the wrong value for delx

[Darshan-KN]

Sir I try to generate the Drr by using the
sdd=1020, delx=0.194, and dely=0.194, but the DRR is not generated it's given the blank image the DRR is not projected to the image plane.

[Darshan-KN]

Sir, these DiffDRR generation coordinates are similar to those of the C-arm coordinate system. If not, can you tell me how this DiffDRR is different from C-arm coordinates?