Filling in the blanks

Several areas of research I find interesting concern themselves with various forms of signal reconstruction and synthesis as they are applied to digital images. Depending on who's doing the work, various names come into play such as texture synthesis, image inpainting, healing brushes, etc.

The problem boils down to this. Let's say your digital image contains undesirable elements or somehow lacks desirable elements. The situation may be due to a hole or scratch in the image, or perhaps you just want to remove a scar or pimple from a face.

The ideal solution paints over the undesirable image element (or completes the missing parts of a desirable element) in such a way that one can't tell the problem ever existed. Where there was once a mole, now there's only perfect skin. The scratch on the scan you made of an old photograph is nowhere to be seen--all you see is a perfect brick wall. Etc.

Not surprisingly, it's not easy to fill in the blanks. In fact, finding an algorithm that works well in all cases has proven to be a very challenging problem. Filling in solid colors is simple enough, but what about grass, sky, carpet, skin, clouds and wicker? Some elements are very orderly, some are chaotic, some repeat, others don't.

Wrestling with such problems leaves me wondering how our brains manage to fill things in so well. Our eyes have blind spots, but we're generally oblivious to these holes in our visual fields because our brains do such a wonderful job of filling in the blanks, synthesizing images that aren't really there. Can neuroscientists help us improve our algorithms?

Here's a link to a fun little tutorial on the blind spot. Try a few and see how your brain fills in the blanks in space that your eyes don't see:

Seeing more than your eye does