Computer Science and Electrical Engineering
University of Maryland, Baltimore County

Ph.D. Dissertation Defense

Multi-Modal Saliency Fusion for Illustrative Image Enhancement

Christopher J. Morris

10:30-12:30, Wednesday, 15 January 2013, ITE 365 & 352

Digitally manipulated or augmented images are increasingly prevalent. Multisensor systems produce augmented images that integrate data into a single context. Mixed-reality images are generated from insertion of computer generated objects into a natural scene. Digital processing for application-specific tasks (e.g., compression or network transmission) can create images distorted with processing artifacts. Digital image augmentation can lead to the inclusion of artifacts that influence the perception of the image.

Visual cues (e.g., depth or size cues) may no longer be perceptually consistent in an augmented image. A feature deemed important in its local context may no longer be in the broader integrated context. Inserted synthetic objects may not possess the appropriate visual cues for proper perception of the overall scene. Finer cues that distinguish critical features may be lost in compressed images. Enhancing augmented images to add or restore visual cues can improve the image’s perceptibility.

This dissertation presents a framework for illustrating images to enhance critical features. The enhancements improve the perception and comprehension of the augmented image. The framework uses a linear combination of image (2D), surface topology (3D), and task based saliency measures to identify the critical features in the image. The use of multi-modal saliency allows the visualization designer to adjust the definition of critical features based on the attributes of the scene and the task at hand. Upon identification, the features are enhanced using a non-photorealistic rendering (NPR) deferred illustration technique. The enhancements, inspired by an analysis of artists’ techniques, bolster the features’ perceptual cues.

To measure the amount of similar salient features between the enhanced image and the original image, the framework describes the Saliency Similarity Metric (SSM). The SSM is feedback with which to make informed decisions to tune the visualization. The benefits of illustrative enhancement are analyzed using objective and subjective evaluations. Using conventional metrics, illustrative enhancements improve the perceptual image quality of images distorted by noise or compression artifacts. User survey results reveal that enhancements must be carefully applied for perceptual improvement. The framework can be effectively utilized in mobile rendering, augmented reality, and sensor fusion applications.

Committee: Drs. Penny Rheingans (chair), Dan Bailey, Jian Chen, Thomas Jackman (Desert Research Institute), Anupam Joshi and Marc Olano

Every week we feature a set of comics created exclusively for WDD.

The content revolves around web design, blogging and funny situations that we encounter in our daily lives as designers.

These great cartoons are created by Jerry King, an award-winning cartoonist who’s one of the most published, prolific and versatile cartoonists in the world today.

So for a few moments, take a break from your daily routine, have a laugh and enjoy these funny cartoons.

Feel free to leave your comments and suggestions below as well as any related stories of your own…

Expanded butt search

Supplies and experience

One smart lawyer

Can you relate to these situations? Please share your funny stories and comments below…

Convert Users from “Try” to “Buy” with ’30 Days to Sell’ – only $7.95!

Create a single app binary that supports 64-bit on iOS 7, as well as 32-bit on iOS 7 and iOS 6, so your apps can easily remain compatible as they take advantage of the power of iPhone 5s, iPad Air, and iPad mini. Learn more.

If you have concerns about an app rejection, you can ask questions and provide clarification in the Resolution Center in iTunes Connect. Simply use the Reply feature to communicate directly with the App Store Review team.