Photos aligned but no points in some areas

[zmk]

Hi all,

We've recently photographed a large fresco and have unfortunately run into issues during the creation of the sparse cloud.

The images we took are sharp, uniformly lit, correctly white balanced and have an overlap of approx.. 30-50%. When creating a sparse cloud all cameras align, however in one area on the left-hand side of the painting there are several areas without any points (see screenshot). I have tried changing the amount of key and tie points from very low to very high. I have also tried with and without generic preselection and the other options available when building a sparse cloud and disabled some cameras to see if the amount of images was ‘confusing’ the software, all without success. Does anyone have any ideas what is happening and how we can overcome this issue?

Many thanks,
Kira

[Dave Martin]

Hi Kira,

Without seeing the footprint of your images, I would have to say that even 50% overlap may be too low; and 30% is almost certainly too low.

30% overlap (call it 1/3 for ease) means that, just working along a row, adjacent photographs will overlap by 1/3 - but the middle 1/3 will only be on one photo - so whilst there may be sufficient point to identify ans key points and identify as tie points to triangulate, there will be a gap when you come to reconstruct the texture (surface) - you can't do stereo reconstruction from just one photo!.

I made a quick sketch to try and illustrate this, but I'm afraid I've had to pop it in a DropBox folder folder for now whilst the problem with attachment quotas on this forum is resolved - so you can see the sketch by clicking https://www.dropbox.com/s/5vgsc9qoi5r7zsa/Insufficient overlap.png?dl=0

In that sketch are three photographs taken from left to right; as you can see there is sufficient overlap (the yellow tinted piece) between the right-hand third pf photo 1 and the left hand third of photo 2 to align them for minimal stereo matching; similarly there is sufficient overlap between the right-hand third pf photo 2 and the left hand third of photo 3 to align them. However, the middle portion of photo 2 doesn't appear on any other photograph - so the middle 1/3 cannot be proper determined.

If there were similar stripes above and below, each with a 1/3 overlap, then the top 1/3 and bottom 1/3 would be aligned and able to be reconstructed, but there would still be a block in the middle of photo 2, 1/9 or the area (i.e. 1/3 high and 1/3 wide) which only appeared in photo 2.

I think you need considerably more overlap. In general photogrammetric terms, where photographs are taken with the camera in one orientation, one typically talks of perhaps 75% or even 80% overlap; or that everything in the area of interest appears in at least three photograph frames. An alternative to densify coverage and strengthen acquisition is multiple grids with the camera sensor rotated.

There is a guide to photogrammetric acquisition that CHI publish at https://culturalheritageimaging.org/Technologies/Photogrammetry/index.html

If you re-shoot with proper overlap I'm pretty sure the 'gaps' problem will be resolved.

Dave

[zmk]

Hi Dave,

Many thanks for the quick answer and your tips!

What I don't understand in that case is that it worked fine in other areas of similar detail where we actually reduced the overlap from 50 to 30%. 

We've also done a panorama merge in photoshop which had no issues with the overlap.

Thanks again,
Kira 

[Dave Martin]

Hi again, thanks Kira.

Just stitching a panorama in, say, Photoshop or even Microsoft Image Composite Editor, just needs alignment, and doesn't reconstruct the surface.

I've just had a thought - what exactly have you shown/screenshotted in that first post illustration - is that just a cloud or is it an ortho photo?

Dave

[zmk]

Hi Dave,

It's the sparse cloud.

I had a look at the missing areas (e.g. the eye of the shepherd) and they are captured in 4 different images. Not ideal given your comments, but as mentioned, some other areas are also only covered by four images, but have not had issues.

All best,
Kira

[Dave Martin]

So what happens if you go on to produce an ortho-photo?

[cdschroer]

Hi Kira -

As Dave notes - the recommended overlap for photogrammetry is 66% or 2/3. Our goal is to get to 9 look angles for any part of the surface.  While you might get alignment, and even some surface data with less - it isn't recommended.  Also as noted, making a panorama or stitched image can work with far less overlap 20-30% - in that case there is no reconstruction of the surface, and there will be stitching artifacts. You could do a little bit of optimization, and then build an ultra high dense cloud from a small area and see what you get.  I'd want to se other metrics like the number of projections on the images - and also you can create the "point cloud variance" when you optimize.  There's a box to check for that - then you can look at the false color of the sparse cloud to give you an idea of where you have good matches and where you don't.  Bottom line is that you have insufficient overlap for a good photogrammetry project if you are doing 30-50%

Carla

[zmk]

Hi both,
Thanks a lot for your input and suggestions. I double and triple checked image overlap and while we had a 66% overlap across most of the fresco the 'missing' areas only had a 50% overlap. That is now added to the lab notebook in red as a lesson learned. Thankfully we managed to successfully create the orthophoto in the end so all is well.
Thanks again, Kira

[Dave Martin]

Hi Kira,
Glad you're sorted and thanks for reporting back.
Dave