OpenCV + Python Morphology

 

 

Hello. This is codingwalks.

Image processing plays an important role in various fields, and among them, morphological operations and watershed algorithms are very useful techniques for analyzing the structural characteristics of images and separating objects. Morphological operations are used to adjust the shape of objects, distinguish boundaries between objects, and remove image noise. This article covers the theoretical background of morphological operations such as erosion, dilation, opening, closing, and gradient, and the watershed algorithm, along with practical code examples using Python and OpenCV. These operations are particularly effective in applications such as coin classification and fingerprint ridge detection.

 

1. What is morphology?

Morphology is a technique for processing images based on mathematical shape theory. This technique is mainly used to analyze and process the shape or structure of objects in binary images (black and white images). Morphological operations play an important role in clearly distinguishing objects or removing noise in images by changing the size, shape, and boundaries of objects in an image.

Morphological operations change the shape of an image based on the pixel values ​​of the image, considering the relationship between neighboring pixels. This operation may physically appear to "expand" or "shrink" pixels. For example, it is useful for smoothing the boundaries of objects or removing unnecessary noise.

The main purpose of morphological operations:

• Noise removal: Preserving the important structure of an image by removing small noises.
• Object separation: Separating connected objects to make them distinguishable.
• Hole filling: Filling small holes inside an object.
• Shape analysis: Extracting the shape or boundaries of an object.

The main concept of morphological operations is to change the shape of an object through a kernel (structuring element). A kernel is a small matrix with a fixed size and shape that performs a specific operation as it moves over the image.

 

2. Basic Elements of Morphological Operations

The basic elements of morphological operations are the kernel (structuring element) and the image to which the operation is applied. These two elements have a decisive influence on the result of the morphological operation and determine how the operation is performed.

2.1. Kernel (structuring element)

A kernel is a small matrix that plays a key role in morphological operations. A kernel is usually square (e.g. 3x3, 5x5) or has another regular shape. As the kernel moves over the image, it performs a specific operation, and the result of the operation is greatly affected by the shape and size of the kernel.

• Shape of the kernel: A kernel can be defined in various shapes such as square, circle, and cross. The shape of the kernel is chosen depending on how the object is deformed.
• Size of the kernel: The size of the kernel determines the intensity of the operation. For example, a larger kernel shrinks the object more in an erosion operation, and expands the object more in a dilation operation. The kernel defines the neighboring pixels around each pixel, and can find patterns of interest in the image or enhance the boundary. The kernel moves one pixel at a time in the image and performs operations at each location.

2.2. Basic formula of morphological operation

Morphological operation is basically based on set theory. In the image, E is expressed as a binarized image, and B represents the kernel. The basic concepts of the two operations are expressed in formulas as follows.

• Erosion: The erosion operation reduces an object. Formulaically, erosion is defined as follows. \(E \ominus B = \{ z \in E \mid (B)_z \subseteq E \}\) Here, \(\ominus\) means the erosion operation, and B is the kernel. In other words, it is an operation to find all locations z where the kernel B is completely included in the image E.
• Dilation: The dilation operation expands an object. The dilation operation is defined as follows. \(E \oplus B = \{ z \in E \mid (B)_z \cap E \neq \emptyset \}\) Here, \(\oplus\) represents the dilation operation, and B is the kernel. Dilation is the process of finding all locations where a part of the kernel overlaps the image.

2.3. How the operation is applied

As the kernel moves over the image, the pixel value changes depending on whether a certain condition is satisfied within the area occupied by the kernel. For example, in the erosion operation, the pixel is kept only if the kernel is completely inside the object, and is removed otherwise. Conversely, in the dilation operation, the pixel is expanded in all cases where the kernel overlaps the object.

2.4. Binarized and grayscale images

Morphological operations are mainly used in binarized images, but they can also be applied to grayscale images. In binarized images, pixel values ​​consist of 0 (black) and 255 (white), but in grayscale images, pixel values ​​have continuous values ​​from 0 to 255. In grayscale images, morphology operations are performed based on brightness differences and are used to emphasize the features of boundaries or objects.

 

3. Morphology Algorithms

3.1. Erosion

Erosion is an operation that reduces the boundaries of objects in an image. As the kernel (structuring element) moves along the outline of the object, pixels in the area where the kernel is completely contained within the object are left, otherwise they are removed. This reduces the object and removes small noises.

• Erosion works as if it is cutting away the object, gradually reducing the boundaries of the object.
• It is mainly used to remove noise attached to the edges of the object.
• As the kernel size increases, the size reduction of the object increases.

3.2. Dilation

Dilation is the opposite operation of erosion, and expands the boundaries of an object. As the kernel moves over the image, if there is even one object pixel within the kernel area, all pixel values ​​in that area are set to 1 (white). This increases the size of the object and fills in small holes.

• The dilation operation acts as if "inflating" the object, increasing its size.
• It is useful for filling small holes or broken parts.
• The larger the kernel size, the larger the range by which the object expands.

3.3. Opening

The opening operation is an operation that performs dilation after erosion. During this process, small noises attached to the outer edge of the object are removed, but the size or shape of the object does not change significantly. Therefore, it is effective for removing small noises.

• Opening is a process that applies erosion first and then dilation.
• The erosion removes small noises, and the dilation restores the original shape.
• It is mainly used for removing small objects or noises.

3.4. Closing

The closing operation is an operation that performs erosion after dilation. It fills in small holes or gaps inside the object while maintaining the boundary of the object. This is useful for keeping the connected parts of the object intact and filling in small defects inside.

• Close applies Dilation first, then Erosion.
• The object expands with Dilation, and the original shape is restored with Erosion, but holes or gaps are filled.
• It is mainly used to fill small holes inside the object or to maintain connections between objects.

3.5. Gradient

Morphological Gradient is an operation that calculates the difference between Dilation and Erosion. It is very useful for emphasizing the boundary of an object, and is mainly used when performing edge detection. It emphasizes the boundary by calculating the difference between Dilation and Erosion.

• The Gradient represents the difference between the Dilated image and the Erosion image.
• This makes the boundary of the object appear emphasized.
• It is useful for extracting the boundary or the outline of an object.

3.6. Source Code

import cv2
import numpy as np

image = cv2.imread('resources/lena.bmp', cv2.IMREAD_GRAYSCALE)

# Kernel definition (size 5x5)
kernel = np.ones((5, 5), np.uint8)

# Apply erosion operation
erosion = cv2.erode(image, kernel, iterations=1)

# Applying dilation operation
dilation = cv2.dilate(image, kernel, iterations=1)

# Apply opening operation
opening = cv2.morphologyEx(image, cv2.MORPH_OPEN, kernel)

# Apply closing operation
closing = cv2.morphologyEx(image, cv2.MORPH_CLOSE, kernel)

# Apply gradient operation
gradient = cv2.morphologyEx(image, cv2.MORPH_GRADIENT, kernel)

titles = ['Original', 'Erosion', 'Dilation', 'Opening', 'Closing', 'Gradient']
images = [thresh, erosion, dilation, opening, closing, gradient]

plt.figure(figsize=(10, 7))
for i in range(len(images)):
	plt.subplot(2, 3, i+1), plt.imshow(images[i], cmap='gray')
    plt.title(titles[i])
    plt.xticks([]), plt.yticks([])
    
plt.tight_layout()
plt.show()

morphology example

 

4. Watershed Algorithm

Watershed algorithm is one of the image segmentation techniques used to distinguish various objects or structures. This algorithm is inspired by topographic interpretation and is effective in separating different regions (e.g., objects and background) in an image.

Watershed Concept:

• Topographic Interpretation: The Watershed algorithm interprets the image as a height map of the terrain. That is, bright areas in the image are considered mountain peaks and dark areas are considered valleys. Watershed analyzes how different regions are separated when filled with water to form boundaries.
• Markers: In the Watershed algorithm, markers are first set, which represent objects and backgrounds. Markers are set where the object interior and background can be clearly distinguished, and the algorithm finds the object boundaries based on these markers.

How the Algorithm Works:

• Setting Markers: Set markers to distinguish objects and backgrounds. Markers are where the Watershed algorithm starts, and they indicate clearly distinguished object interiors and backgrounds.
• Watershed operation: Water is filled from the point where the marker is, and a boundary is formed at the point where different markers meet.
• Result: When the boundary is completed, the image is divided into several regions. The boundary separates the object from the object, or the object from the background.

The watershed algorithm is very useful in image segmentation problems such as fingerprint recognition, medical image analysis, and coin classification.

4.1. Coin classification example using the watershed algorithm

This code is used to analyze an image containing multiple coins and separate each coin. The watershed algorithm is used to distinguish the boundaries of the coins. This code loads the coin image and applies the watershed algorithm to find the boundaries of the coins. In this process, various morphological operations such as binarization, dilation, and opening are used.

import cv2
import numpy as np
from matplotlib import pyplot as plt

image = cv2.imread('resources/coins.jpg')
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)


# Binarization (using Otsu's thresholding)
_, thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)

# Noise removal (open operation)
kernel = np.ones((3, 3), np.uint8)
opening = cv2.morphologyEx(thresh, cv2.MORPH_OPEN, kernel, iterations=2)

# Expanding the object area (dilation operation)
sure_bg = cv2.dilate(opening, kernel, iterations=3)

# Finding the exact area of an object using distance transform
dist_transform = cv2.distanceTransform(opening, cv2.DIST_L2, 5)
_, sure_fg = cv2.threshold(dist_transform, 0.7 * dist_transform.max(), 255, 0)

# Finding boundary area by difference between clear object area and background area
sure_fg = np.uint8(sure_fg)
unknown = cv2.subtract(sure_bg, sure_fg)

# Create marker
_, markers = cv2.connectedComponents(sure_fg)

# Add 1 to the marker to set the background to 1
markers = markers + 1

# Set the boundary area as a marker
markers[unknown == 255] = 0

image_color = cv2.imread('resources/coins.jpg')

# apply watershed
markers = cv2.watershed(image_color, markers)
image_color[markers == -1] = [255, 0, 0]

titles = ['Gray', 'Binary', 'Sure BG', 'Distance', 'Sure FG', 'Unknown', 'Markers', 'Result']
images = [gray, thresh, sure_bg, dist_transform, sure_fg, unknown, markers, image]

plt.figure(figsize=(10, 5))
for i in range(len(images)):
    plt.subplot(2, 4, i+1), plt.imshow(images[i], cmap='gray')
    plt.title(titles[i])
    plt.xticks([]), plt.yticks([])
    
plt.tight_layout()
plt.show()

coins segmentation

 

5. Practical Applications of Morphological Operations

• Erosion:
  • Text Extraction Preprocessing: It is useful for removing small noises and simplifying the outline of objects when extracting text from images. For example, it can remove small dots in the background from document scan images.
• Dilation:
  • Lane Recognition: It is used to highlight lanes in road images. The dilation operation increases the thickness of the lanes to facilitate detection.
• Opening:
  • Medical Image Analysis: It is used to remove small lesions or noises from MRI images. When analyzing a specific part, it removes unnecessary noises and extracts only the object.
• Closing:
  • Character Recognition (OCR): The closing operation can perform character recognition more accurately by filling in small holes or disconnected parts in the image. It plays a role in connecting broken letters in the image.
• Gradient:
  • Edge Detection: It is very useful for extracting the boundaries of objects. For example, it can be used for extracting building outlines or detecting car outlines.
• Watershed:
  • Fingerprint Analysis: Used for fingerprint recognition by detecting the boundaries of fingerprint ridges.
  • Coin Classification: Used to separate and classify each coin from a pile of coins.
  • Medical Image Segmentation: When analyzing CT scans or MRI images, the Watershed algorithm can be used to distinguish each organ or tissue. It is especially effective in distinguishing tumors from normal tissues in brain images.
  • Industrial Image Processing: In the process of inspecting defects in manufacturing, Watershed can be used to distinguish objects and defects. For example, it is useful for detecting small defects on metal surfaces.
  • Object Detection and Tracking: In surveillance systems, when multiple objects overlap, the Watershed algorithm can be used to distinguish different objects. For example, it can distinguish the boundaries between objects in situations where there are crowds or vehicles.

 

6. Conclusion

Morphological operations and the Watershed algorithm play a very important role in image analysis and processing. Basic morphological operations such as erosion, dilation, opening, closing, and gradient are useful for adjusting the outline of objects and removing noise from images, and the watershed algorithm is a powerful technique for effectively separating objects from complex images.

In this article, we will explain the theoretical background of morphological operations and the watershed algorithm in detail, and learn how they can be used in practical applications such as coin classification and fingerprint ridge detection through code examples. These technologies are used in various fields such as medical, industrial, security, object detection, and tracking, and they make a great contribution to solving image processing problems. Through this, we can improve the efficiency of image processing and analysis and obtain better results.

Morphological operations and the watershed algorithm are very powerful tools in computer vision, and they will be widely used in various fields in the future.

 

If you found this post useful, please like and subscribe below. ^^

Buy me a coffee

[Codingwalks]에게 송금하기 - AQR

★ All contents are referenced from the link below. ★

OpenCV: OpenCV-Python Tutorials

https://opencv-python.readthedocs.io/en/latest/doc/27.imageWaterShed/imageWaterShed.html