The Aerial Perspective Blog

Mapware can create a 3D digital twin of a landscape from a set of 2D aerial photos. In this article, we discuss the first step in Mapware’s photogrammetry pipeline: keypoint extraction.

Purpose of keypoint extraction

When users upload digital images to Mapware and initiate its photogrammetry process, the first step is keypoint extraction: identifying the distinctive features in each image and assigning them values that a computer can easily reference later.

Keypoint extraction begins the photogrammetry pipeline for two reasons.

First, it assists with computer vision, the science of helping a computer understand an image the way humans do by picking out the most interesting shapes from the background. This helps Mapware later in the pipeline when it stitches image sets together into a 3D digital twin.

More importantly, keypoint extraction aids in image compression. Typical photogrammetry projects can require hundreds or even thousands of photos, with each photo containing millions of pixels. Reading these large image sets can be memory-intensive, increasing the risk of system crashes. But keypoints serve as bookmarks in each image file, allowing computers to read the important features and ignore the rest. The result is faster and more-reliable processing.

The keypoint extraction process

Mapware identifies keypoints like most photogrammetry software, using a combination of corner detection, descriptor assignment, and invariance calculation.

• Corner Detection: To identify distinctive features in each photo, Mapware’s corner detection algorithms search for pixel groups whose grayscale intensity values differ substantially from their neighbors. These can either be edges—boundary lines between two areas of differing intensity—or corners—points where two lines converge. Mapware identifies edges and corners, but only designates corners as keypoints. This is because corners are easier to localize using cartesian (x, y) coordinates, whereas edges could be lines that run the entire length of an image. You might think of corner detection as loosely similar to the way human eyes notice highly-contrasting features in an image. However, the comparison is arbitrary, as computers may not notice the same image features a human would.

• Fingerprint Assignment: To aid in image compression, Mapware then runs another algorithm to mathematically reduce each keypoint into a compact hash called a fingerprint. Some other photogrammetry products refer to these as “descriptors” because they not only help a computer quickly find a keypoint later in the image, but also describe its properties. Thanks to fingerprint assignments, Mapware doesn’t have to process entire images again; it can just read their (smaller) fingerprints.

• Invariance Calculation: To help stitch photos together later in the photogrammetry pipeline, Mapware ensures each fingerprint is invariant (unchanging) with respect to scale and orientation. In real-world terms, this means a pilot can photograph the same feature twice from different heights and angles—and Mapware will assign nearly identical fingerprints to both images despite their differences. This helps Mapware match photos of the same feature in spite of the changing flight paths of a camera drone.

How Mapware uses keypoints

After Mapware has identified the keypoints in each image and assigned their fingerprints, it can gradually assemble individual images together into the composite that will become a 3D digital twin.

It starts by identifying pairs of images that have nearly identical keypoints. These keypoint pairs exist because drone pilots take photos with overlap to ensure that the same features will appear on more than one image. For example, they may ensure the same feature appearing at the back of one photo appears at the front of the next photo.

If Mapware identifies the same keypoint in both images, it knows to pair them together on their overlapping region. The next article in this series describes the keypoint pairing step, which is called homography.