Aligning The Images – Why Things Go Wrong

Recently we helped IANTD run the first two day Diving Skills for Photogrammetry course at Vobster Quay Inland Diving.

The course teaches not only how to dive and work as an efficient buddy pair to capture the photos needed, but goes into some detail as to how Metashape uses the images to align and build a 3D model.

By the end of the two days all the students had achieved the course objectives and had created 3D models of their own. It was great to see progress as the processing churned away on the data they captured.

Good Data In – Good Models Out

Its an old adage but nothing beats pin-sharp, correctly exposed, evenly lit images for photogrammetry. Most of the students were using video as primary capture – many divers these days will carry an action camera of some sort or another – and with extra lighting even a modest GoPro will deliver a result. In the following example the stills were extracted from video and initially there were 720 frames to choose from. But whilst the automatic extraction of stills from video is very efficient we can end up with images that are more of a pain than benefit.

By culling images likely to cause errors before running alignment we achieve two things:

• Faster processing time – less images to process.
• Higher quality model and texture.

The question then is what do we cull and why?

Its probably fair to say more learning is achieved by initial failure than accidental success, so its worth taking the time to stop and review when things don’t align and what images are worth discarding. Paul Lijnen of IANTD Benelux and France was on the course and has kindly allowed some of his images to be used to illustrate a few of the common issues with images.

Over Exposed

Burning out, or over exposing, the subject results in a loss of detail in the frame and in this case the solid white area is caused by moving the video lights too close to the subject. We depend on detail and the key points to align images so this frame is likely to fail and can be discarded, particularly if there is a higher quality alternative.

Motion Blur

Panning the camera whilst shooting images can cause motion blur and can occur when shooting stills with flash or (more likely) when shooting video for stills extraction. The detail in the frame is distorted and makes matching common points in other frames challenging at best, so this frame can be discarded. The effect can be eliminated at time of capture by panning slowly, avoiding any rapid motion.

Out of Alignment

On action cameras like a GoPro we cannot control where the focus point is, its fixed on infinity and the sweet spot is right in the centre of the frame. Any angled subject (like the one above) will have out of focus areas at the edges of the frame and like motion blur the fuzzy detail will cause issues with point detection and alignment. The solution at time of capture is to keep the camera lens parallel to the subject, and to discard any images that are out of focus.

Missing Subject

Without the optional GoPro viewing screen composure of the image can be a bit hit-and-miss. In the example above just 1/3rd of the frame is relevant to the model and most of the potential tie points are in shadow.

Estimating the Quality

There is a tool in Metashape to help find and identify images of questionable quality before running the alignment.

After analysis the Quality column in the Photos tab is populated with a numeric value, with the higher the number indicative of higher image quality.

The tool looks at the frame and estimates the sharpness of the most focussed part of the picture and Agisoft recommend any image with an estimate below 0.5 should be excluded, assuming there are other images covering the area to be modelled exist. As you can see from the above values Metashape will align at lower values. Its a useful tool, but the human eye is still a valuable asset and reviewing images before alignment is a useful verification.