## Table of Contents

*1 Introduction *

1.1 What is Digital Image Processing?

1.2 The Origins of Digital Image Processing

1.3 Examples of Fields that Use Digital Image Processing

Gamma-Ray Imaging

X-Ray Imaging

Imaging in the Ultraviolet Band

Imaging in the Visible and Infrared Bands

Imaging in the Microwave Band

Imaging in the Radio Band

Other Imaging Modalities

1.4 Fundamental Steps in Digital Image Processing

1.5 Components of an Image Processing System

*2 Digital Image Fundamentals *

2.1 Elements of Visual Perception

Structure of the Human Eye

Image Formation in the Eye

Brightness Adaptation and Discrimination

2.2 Light and the Electromagnetic Spectrum

2.3 Image Sensing and Acquisition

Image Acquisition Using a Single Sensing Element

Image Acquisition Using Sensor Strips

Image Acquisition Using Sensor Arrays

A Simple Image Formation Model

2.4 Image Sampling and Quantization

Basic Concepts in Sampling and Quantization

Representing Digital Images

Linear vs. Coordinate Indexing

Spatial and Intensity Resolution

Image Interpolation

2.5 Some Basic Relationships Between Pixels

Neighbors of a Pixel

Adjacency, Connectivity, Regions, and Boundaries

Distance Measures

2.6 Introduction to the Basic Mathematical Tools Used in Digital Image Processing

Elementwise versus Matrix Operations

Linear versus Nonlinear Operations

Arithmetic Operations

Set and Logical Operations

*Basic Set Operations *

*Logical Operations *

*Fuzzy Sets *

Spatial Operations

*Single-Pixel Operations *

* Neighborhood Operations *

* Geometric Transformations *

* Image Registration *

Vector and Matrix Operations

Image Transforms

Probability and Random Variables

*3 Intensity Transformations and Spatial Filtering*

3.1 Background

The Basics of Intensity Transformations and Spatial Filtering

About the Examples in this Chapter

3.2 Some Basic Intensity Transformation Functions

Image Negatives

Log Transformations

Power-Law (Gamma) Transformations

Piecewise Linear Transformation Functions

*Contrast Stretching *

* Intensity-Level Slicing *

* Bit-Plane Slicing *

3.3 Histogram Processing

Histogram Equalization

Histogram Matching (Specification)

Exact Histogram Matching (Specification)

* Foundation *

Ordering

Computing the neighborhood averages and extracting the K-tuples*: *

* Exact Histogram Specification Algorithm *

Local Histogram Processing

Using Histogram Statistics for Image Enhancement

3.4 Fundamentals of Spatial Filtering

The Mechanics of Linear Spatial Filtering

Spatial Correlation and Convolution

Separable Filter Kernels

Some Important Comparisons Between Filtering in the Spatial and Frequency Domains

A Word about how Spatial Filter Kernels are Constructed

3.5 Smoothing (Lowpass) Spatial Filters

Box Filter Kernels

Lowpass Gaussian Filter Kernels

Order-Statistic (Nonlinear) Filters

3.6 Sharpening (Highpass) Spatial Filters

Foundation

Using the Second Derivative for Image Sharpening—The Laplacian

Unsharp Masking and Highboost Filtering

Using First-Order Derivatives for Image Sharpening—The Gradient

3.7 Highpass, Bandreject, and Bandpass Filters from Lowpass Filters

3.8 Combining Spatial Enhancement Methods

3.9 Using Fuzzy Techniques for Intensity Transformations and Spatial Filtering

Introduction

Principles of Fuzzy Set Theory

* Definitions *

* Some Common Membership Functions *

Using Fuzzy Sets

Using Fuzzy Sets for Intensity Transformations

Using Fuzzy Sets for Spatial Filtering

*4 Filtering in the Frequency Domain*

4.1 Background

A Brief History of the Fourier Series and Transform

About the Examples in this Chapter

4.2 Preliminary Concepts

Complex Numbers

Fourier Series

Impulses and their Sifting Properties

The Fourier Transform of Functions of One Continuous Variable

Convolution

4.3 Sampling and the Fourier Transform of Sampled Functions

Sampling

The Fourier Transform of Sampled Functions

The Sampling Theorem

Aliasing

Function Reconstruction (Recovery) from Sampled Data

4.4 The Discrete Fourier Transform of One Variable

Obtaining the DFT from the Continuous Transform of a Sampled Function

Relationship Between the Sampling and Frequency Intervals

4.5 Extensions to Functions of Two Variables

The 2-D Impulse and Its Sifting Property

The 2-D Continuous Fourier Transform Pair

2-D Sampling and the 2-D Sampling Theorem

Aliasing in Images

* Extensions from 1-D Aliasing *

* Image Resampling and Interpolation *

* Aliasing and Moiré Patterns *

The 2-D Discrete Fourier Transform and Its Inverse

4.6 Some Properties of the 2-D DFT and IDFT

Relationships Between Spatial and Frequency Intervals

Translation and Rotation

Periodicity

Symmetry Properties

Fourier Spectrum and Phase Angle

The 2-D Discrete Convolution Theorem

Summary of 2-D Discrete Fourier Transform Properties

4.7 The Basics of Filtering in the Frequency Domain

Additional Characteristics of the Frequency Domain

Frequency Domain Filtering Fundamentals

Summary of Steps for Filtering in the Frequency Domain

Correspondence Between Filtering in the Spatial and

Frequency Domains

4.8 Image Smoothing Using Lowpass Frequency Domain Filters

Ideal Lowpass Filters

Gaussian Lowpass Filters

Butterworth Lowpass Filters

Additional Examples of Lowpass Filtering

4.9 Image Sharpening Using Highpass Filters

Ideal, Gaussian, and Butterworth Highpass Filters from Lowpass Filters

The Laplacian in the Frequency Domain

Unsharp Masking, High-boost Filtering, and High-Frequency-Emphasis Filtering

Homomorphic Filtering

4.10 Selective Filtering

Bandreject and Bandpass Filters

Notch Filters

4.11 The Fast Fourier Transform

Separability of the 2-D DFT

Computing the IDFT Using a DFT Algorithm

The Fast Fourier Transform (FFT)

*5 Image Restoration and Reconstruction*

5.1 A Model of the Image Degradation/Restoration Process

5.2 Noise Models

Spatial and Frequency Properties of Noise

Some Important Noise Probability Density Functions

* Gaussian Noise *

* Rayleigh Noise *

* Erlang (Gamma) Noise *

* Exponential Noise *

* Uniform Noise *

* Salt-and-Pepper Noise *

Periodic Noise

Estimating Noise Parameters

5.3 Restoration in the Presence of Noise Only—Spatial Filtering

Mean Filters

* Arithmetic Mean Filter *

* Geometric Mean Filter *

* Harmonic Mean Filter *

* Contraharmonic Mean Filter *

Order-Statistic Filters

* Median Filter *

* Max and Min Filters *

* Midpoint Filter *

* Alpha-Trimmed Mean Filter *

Adaptive Filters

* Adaptive, Local Noise Reduction Filter *

* Adaptive Median Filter *

5.4 Periodic Noise Reduction Using Frequency Domain Filtering

More on Notch Filtering

Optimum Notch Filtering

5.5 Linear, Position-Invariant Degradations

5.6 Estimating the Degradation Function

Estimation by Image Observation

Estimation by Experimentation

Estimation by Modeling

5.7 Inverse Filtering

5.8 Minimum Mean Square Error (Wiener) Filtering

5.9 Constrained Least Squares Filtering

5.10 Geometric Mean Filter

5.11 Image Reconstruction from Projections

Introduction

Principles of X-ray Computed Tomography (CT)

Projections and the Radon Transform

Backprojections

The Fourier-Slice Theorem

Reconstruction Using Parallel-Beam Filtered Backprojections

Reconstruction Using Fan-Beam Filtered Backprojections

*6 Wavelet and Other Image Transforms*

6.1 Preliminaries

6.2 Matrix-based Transforms

Rectangular Arrays

Complex Orthonormal Basis Vectors

Biorthonormal Basis Vectors

6.3 Correlation

6.4 Basis Functions in the Time-Frequency Plane

6.5 Basis Images

6.6 Fourier-Related Transforms

The Discrete hartley Transform

The Discrete Cosine Transform

The Discrete Sine Transform

6.7 Walsh-Hadamard Transforms

6.8 Slant Transform

6.9 Haar Transform

6.10 Wavelet Transforms

Scaling Functions

Wavelet Functions

Wavelet Series Expansion

Discrete Wavelet Transform in One Dimension

* The Fast Wavelet Transform *

Wavelet Transforms in Two Dimensions

Wavelet Packets

*7 Color Image Processing *

7.1 Color Fundamentals

7.2 Color Models

The RGB Color Model

The CMY and CMYK Color Models

The HSI Color Model

* Converting Colors from RGB to HSI *

* Converting Colors from HSI to RGB *

* Manipulating HSI Component Images *

A Device Independent Color Model

7.3 Pseudocolor Image Processing

Intensity Slicing and Color Coding

Intensity to Color Transformations

7.4 Basics of Full-Color Image Processing

7.5 Color Transformations

Formulation

Color Complements

Color Slicing

Tone and Color Corrections

Histogram Processing of Color Images

7.6 Color Image Smoothing and Sharpening

Color Image Smoothing

Color Image Sharpening

7.7 Using Color in Image Segmentation

Segmentation in HSI Color Space

Segmentation in RGB Space

Color Edge Detection

7.8 Noise in Color Images

7.9 Color Image Compression

*8 Image Compression and Watermarking*

8.1 Fundamentals

Coding Redundancy

Spatial and Temporal Redundancy

Irrelevant Information

Measuring Image Information

* Shannon’s First Theorem *

Fidelity Criteria

Image Compression Models

* The Encoding or Compression Process *

* The Decoding or Decompression Process *

Image Formats, Containers, and Compression Standards

8.2 Huffman Coding

8.3 Golomb Coding

8.4 Arithmetic Coding

Adaptive context dependent probability estimates

8.5 LZW Coding

8.6 Run-length Coding

One-dimensional CCITT compression

Two-dimensional CCITT compression

8.7 Symbol-based Coding

JBIG2 compression

8.8 Bit-plane Coding

8.9 Block Transform Coding

Transform selection

Subimage size selection

Bit allocation

* Zonal Coding Implementation *

* Threshold Coding Implementation *

JPEG

8.10 Predictive Coding

Lossless predictive coding

Motion compensated prediction residuals

Lossy predictive coding

Optimal predictors

Optimal quantization

8.11 Wavelet Coding

Wavlet selection

Decomposition level selection

Quantizer design

JPEG-2000

8.12 Digital Image Watermarking

*9 Morphological Image Processing *

9.1 Preliminaries

9.2 Erosion and Dilation

Erosion

Dilation

Duality

9.3 Opening and Closing

9.4 The Hit-or-Miss Transform

9.5 Some Basic Morphological Algorithms

Boundary Extraction

Hole Filling

Extraction of Connected Components

Convex Hull

Thinning

Thickening

Skeletons

Pruning

9.6 Morphological Reconstruction

Geodesic Dilation and Erosion

Morphological Reconstruction by Dilation and by Erosion

Sample Applications

* Opening by Reconstruction *

* Automatic Algorithm for Filling Holes *

* Border Clearing *

9.7 Summary of Morphological Operations on Binary Images

9.8 Grayscale Morphology

Grayscale Erosion and Dilation

Grayscale Opening and Closing

Some Basic Grayscale Morphological Algorithms

* Morphological Smoothing *

* Morphological Gradient *

* Top-Hat and Bottom-Hat Transformations *

* Granulometry *

* Textural Segmentation *

Grayscale Morphological Reconstruction

*10 Image Segmentation I: Edge Detection,*

* Thresholding, and Region Detection*

10.1 Fundamentals

10.2 Point, Line, and Edge Detection

Background

Detection of Isolated Points

Line Detection

Edge Models

Basic Edge Detection

* The Image Gradient and Its Properties *

* Gradient Operators *

* Combining the Gradient with Thresholding *

More Advanced Techniques for Edge Detection

* The Marr-Hildreth Edge Detector *

* The Canny Edge Detector *

Linking Edge Points

* Local Processing *

* Global Processing Using the Hough Transform *

10.3 Thresholding

Foundation

* The Basics of Intensity Thresholding *

* The Role of Noise in Image Thresholding *

* The Role of Illumination and Reflectance in Image Thresholding *

Basic Global Thresholding

Optimum Global Thresholding Using Otsu’s Method

Using Image Smoothing to Improve Global Thresholding

Using Edges to Improve Global Thresholding

Multiple Thresholds

Variable Thresholding

* Variable Thresholding Based on Local Image Properties *

* Variable Thresholding Based on Moving Averages *

10.4 Segmentation by Region Growing and by Region Splitting and Merging

Region Growing

Region Splitting and Merging

10.5 Region Segmentation Using Clustering and Superpixels

Region Segmentation using K-Means Clustering

Region Segmentation using Superpixels

* SLIC Superpixel Algorithm *

* Specifying the Distance Measure *

10.6 Region Segmentation Using Graph Cuts

Images as Graphs

Minimum Graph Cuts

Computing Minimal Graph Cuts

Graph Cut Segmentation Algorithm

10.7 Segmentation Using Morphological Watersheds

Background

Dam Construction

Watershed Segmentation Algorithm

The Use of Markers

10.8 The Use of Motion in Segmentation

Spatial Techniques

* A Basic Approach *

* Accumulative Differences *

* Establishing a Reference Image *

Frequency Domain Techniques

*11 Image Segmentation II: Active Contours: Snakes **and Level Sets*

11.1 Background

11.2 Image Segmentation Using Snakes

Explicit (Parametric) Representation of Active Contours

Derivation of the Fundamental Snake Equation

Iterative Solution of the Snake Equation

External Force Based on the Magnitude of the Image

Gradient (MOG)

External Force Based on Gradient Vector Flow (GVF)

11.3 Segmentation Using Level Sets

Implicit Representation of Active Contours

Derivation of the Level Set Equation

Discrete (Iterative) Solution of The Level Set Equation

Curvature

Specifying, Initializing, and Reinitializing Level Set Functions

Force Functions Based Only on Image Properties

Edge/Curvature-Based Forces

Region/Curvature-Based Forces

Improving the Computational Performance of Level Set Algorithms

*12 Feature Extraction*

12.1 Background

12.2 Boundary Preprocessing

Boundary Following (Tracing)

Chain Codes

* Freeman Chain Codes *

* Slope Chain Codes *

Boundary Approximations Using Minimum-Perimeter Polygons

* Foundation *

* MPP Algorithm *

Signatures

Skeletons, Medial Axes, and Distance Transforms

12.3 Boundary Feature Descriptors

Some Basic Boundary Descriptors

Shape Numbers

Fourier Descriptors

Statistical Moments

12.4 Region Feature Descriptors

Some Basic Descriptors

Topological Descriptors

Texture

* Statistical Approaches *

* Spectral Approaches *

Moment Invariants

12.5 Principal Components as Feature Descriptors

12.6 Whole-Image Features

The Harris-Stephens Corner Detector

Maximally Stable Extremal Regions (MSERs)

12.7 Scale-Invariant Feature Transform (SIFT)

Scale Space

Detecting Local Extrema

* Finding the Initial Keypoints *

* Improving the Accuracy of Keypoint Locations *

* Eliminating Edge Responses *

Keypoint Orientation

Keypoint Descriptors

Summary of the SIFT Algorithm

*13 Image Pattern Classification *

13.1 Background

13.2 Patterns and Pattern Classes

Pattern Vectors

Structural Patterns

13.3 Pattern Classification by Prototype Matching

Minimum-Distance Classifier

Using Correlation for 2-D prototype matching

Matching SIFT Features

Matching Structural Prototypes

* Matching Shape Numbers *

* String Matching *

13.4 Optimum (Bayes) Statistical Classifiers

Derivation of the Bayes Classifier

Bayes Classifier for Gaussian Pattern Classes

13.5 Neural Networks and Deep Learning

Background

The Perceptron

Multilayer Feedforward Neural Networks

* Model of an Artificial Neuron *

* Interconnecting Neurons to Form a Fully Connected Neural Network *

* Forward Pass Through a Feedforward Neural Network *

* The Equations of a Forward Pass *

* Matrix Formulation *

Using Backpropagation to Train Deep Neural Networks

* The Equations of Backpropagation *

* Matrix Formulation *

13.6 Deep Convolutional Neural Networks

A Basic CNN Architecture

* Basics of How a CNN Operates *

* Neural Computations in a CNN *

* Multiple Input Images *

The Equations of a Forward Pass Through a CNN

The Equations of Backpropagation Used to Train CNNs

13.7 Some Additional Details of Implementation

* Bibliography *

* Index *