IR normalization with current variable px distributions

[readicculus]

Not a collaborator so could not re-open the closed normalization issue but I've been working on this on my own and have spent a lot of time on the IR imagery. The current issue, which I think its not possible to get around, is that there is no standard distribution of values nor can it be expected. This means that it's not too difficult to do percentile normalization on the given dataset but when it comes to images without hotspots the normalization causes warmer areas to look like hotspots.

There are a lot of pools of water that look very similar to seals in shape and size but are not. Without some sort of temperature to pixel correlation I've not found a way to do this so that it won't detect a ton of FPs on images without seals or other animals.

It sounds like in future flights the camera will include a temp correlation to at least the max pixel value but until then I'm thinking that the IR data is unfortunately relatively useless. IR imagery is suuuper important as I was getting an F1 score of around .93 on the labeled images and its WAYYYY faster to localize on them than the RGB imagery. Ideally we need to achieve about 3fps localization to be able to run this live on the plane which is seeming impossible with the large rgb images where it's taking me about 1-4 secs per image depending on the model and chip size.

Any feedback would be great as getting IR to work would be a massive step forward in this project.

[readicculus]

Also just to introduce myself I'm Yuval, CS student at UW and working on this as an internship for about 6 months now.

[jomalsan]

Hi Yuval, when working with IR imagery, you need to resist normalizing each image independently, because like you said it will eliminate all useful information from the image. The approach that we took was to do a global normalization on a per camera basis. This normalization was based on adjusting the histogram of all measured values for each camera to be approximately the same and was the best that we could do with the current state of the data. Documentation should be in the ir-normalization folder on how this was done.

In future flights having a properly calibrated camera will be the best path forward and eliminate all of this hand-wavy calibration work. Until then, make sure that all computations you do on IR images are applied to all of them to avoid washing out the valuable information