Content Based Image Recognition (CBIR)

What is CBIR
CBIR is applying processing to images in an attempt to determine the contents of the images. This has applications in the storage and retrieval of large quantities of images. Using CBIR, techniques can be developed for automatically categorizing and storing the images. Using CBIR, techniques can also be developed for searching through a database of images. CBIR can be applied to still images or to video.

Recognition Methods
There are several ways in which the content of images can be recognized.

The simplest way is by content color and color proportions. For example, you could query a database for "all images that have a lot of red, and a little bit of green". Of course, there are a few difficulties with this technique, and this is where fuzzy logic comes into play. When comparing several shades of colors, which shades are red, and which are orange? What is the cutoff point at which blue becomes purple? And what exactly is "a lot", and what is "a little"? These are a few of the difficulties that need to be tackled.

Another way to recognize images is based on objects in the image. Basically, the image can be search based on shapes, colors, textures, edges, size, etc. This is one of the more difficult areas of research. Some of the very preliminary results allow searching for simple objects, such as circles. As more progress is made, you may be able to search for "all images that have a mostly round object, reddish in color, with a shiny surface texture". Of course, the eventual goal of this research is to be able to search for "all images that contain an apple".

A third possible way to search an images is based on faces. There is research being done on face detection. There are already some systems that can identify one or more forward-facing faces, even in front of a complex background. This allows you to search for "all images that have 3 or more faces". In the future, the ideal goal is to be able to detect partial faces, and later, to be able to recognize faces. This would allow you to provide a sample image of a face, and retrieve all images containing that face.

Applications of CBIR - News Video Recognition and Retrieval
One sample use of CBIR is a current research project in which news video is stored in a database along with a text description of the content of the video. This allows later searching for videos based on topics, reporters, time, etc. The goal of this system is to develop a method by which the contents of the video can automatically be determined by computer and automatically stored in the database.

In order to automatically determine the video content, this project uses the text contained in on-screen captions, as well as in closed captions. Once the captions are located in the video, they can be extracted and processed by standard OCR software. The resulting text can then be stored in the database.

This system is not without its difficulties. First, when you have video that contains captions over some background, such as a crowd of protestors, some way is needed to determine which part of each frame is caption and which part is background.One way to accomplish this is through Frame Filtering and Frame AND-ing. Basically, this technique combines several frames together using an AND operation. This has the effect of removing the parts of the image that change, and leaving behind the parts that stay the same. In essence, all of the motion (which would most likely be the background) is removed, and all the static images (which would most likely be the text) remain. Thus, the text captions have been isolated. Another difficulty with this system is in the resolution of the captions. Typically, the digitized video is relatively low resolution, and there are very few pixels to represent the characters in the captions. One way of dealing with this is by magnifying the captions and interpolating the pixels, in essence creating smooth, high resolution captions that can be easily processed..

 

NEXT TOPIC - CONCLUSION

PREVIOUS TOPIC - OPTICAL CHARACTER RECOGNITION

 

return to the beginning