Sparse and Redundant Representations

Michael Elad has been working at The Technion in Haifa, Israel, since 2003 and is currently an Associate Professor. He is one of the leaders in the field of sparse representations. He does prolific research in mathematical signal processing with more than 60 publications in top ranked journals. He is very well recognized and respected in the scientific community.

A long long time ago, echoing philosophical and aesthetic principles that existed since antiquity, William of Ockham enounced the principle of parsimony, better known today as Ockham's razor: "Entities should not be multiplied without neces sity. " This principle enabled scientists to select the "best" physical laws and theories to explain the workings of the Universe and continued to guide scienti?c research, leadingtobeautifulresultsliketheminimaldescriptionlength approachtostatistical inference and the related Kolmogorov complexity approach to pattern recognition. However, notions of complexity and description length are subjective concepts anddependonthelanguage"spoken"whenpresentingideasandresults. The?eldof sparse representations, that recently underwent a Big Bang like expansion, explic itly deals with the Yin Yang interplay between the parsimony of descriptions and the "language" or "dictionary" used in them, and it became an extremely exciting area of investigation. It alreadyyielded a rich crop of mathematically pleasing, deep and beautiful results that quickly translated into a wealth of practical engineering applications. You are holding in your hands the ?rst guide book to Sparseland, and I am sure you'll ?nd in it both familiar and new landscapes to see and admire, as well as ex cellent pointers that will help you ?nd further valuable treasures. Enjoy the journey to Sparseland! Haifa, Israel, December 2009 Alfred M. Bruckstein vii Preface This book was originally written to serve as the material for an advanced one semester (fourteen 2 hour lectures) graduate course for engineering students at the Technion, Israel.

This book introduces sparse and redundant representations with a focus on applications in signal and image processing. It details mathematical modeling for signal sources along with how to use the model for tasks such as denoising, restoration and separation.

Introduces theoretical and numerical foundations before tackling applications

Sparse and Redundant Representations - Theoretical and Numerical Foundations.- Prologue.- Uniqueness and Uncertainty.- Pursuit Algorithms - Practice.- Pursuit Algorithms - Guarantees.- From Exact to Approximate Solutions.- Iterative-Shrinkage Algorithms.- Towards Average PerformanceAnalysis.- The Dantzig-Selector Algorithm.- From Theory to Practice - Signal and Image Processing Applications.- Sparsity-Seeking Methods in Signal Processing.- Image Deblurring - A Case Study.- MAP versus MMSE Estimation.- The Quest for a Dictionary.- Image Compression - Facial Images.- Image Denoising.- Other Applications.- Epilogue.