Mixed Graph Signal Analysis of Joint Image Denoising and Interpolation

Abstract

An image signal captured by a photonic sensor is corrupted by noise and other distortions, and requires several operations before rendering a presentable photo, including denoising, demosaicking, rectification, and white balancing. Specifically, denoising reduces acquisition noise without destroying image details, while interpolation generates novel pixels at new 2D locations using acquired image data. Such image operations are typically applied in sequence---e.g., denoising an image first followed by interpolation---which is sub-optimal in general. In this thesis, we jointly optimize denoising and interpolation operations from a graph signal processing (GSP) perspective. Specifically, we first develop two theorems that show one-to-one mappings between (pseudo-)linear denoisers / interpolators and undirected / directed graph filters that are solutions to variational optimization problems regularized using different graph smoothness priors. Leveraging these theorems, we investigate scenarios where joint denoising / interpolation operations would outperform separate operations in sequence, and mathematically derive those optimal joint operators. Experiments demonstrate validation results, where optimized joint operators outperformed separate operators in sequence in various practical imaging scenarios.