Implementation of the paper "Processing Images and Video for An Impressionist Effect" by Peter Litwinowicz, published on SIGGRAPH'97 Proceedings, pages 407-414.
The paper describes an image sequence transformation in a new sequence that seems to be painted by free hand, inspired in an Impressionist style.
Antonio Scuri (scuri@tecgraf.puc-rio.br)
Paula Frederick (paula@tecgraf.puc-rio.br)
Enylton Machado (machado@visgraf.impa.br)
Mario Guimarães (mario@visgraf.impa.br)
Raul Renteria (renteria@inf.puc-rio.br)
Use this e-mail to send a message to all team : ipcg@tecgraf.puc-rio.br.
The source code and the binaries are available at http://www.visgraf.impa.br/Courses/ipcg-if-1997/source-code.
The program was implemented in C with IUP (user interface), CD (graphical package) and IM (image file persistence) libraries. It works across all major-computing platforms : Windows 95/NT, Silicon Graphics, Sun, Linux e IBM RISC600.
The libraries are freeware for academical purpose. To acquire it, send a message to TeCGraf : tecgraf@tecgraf.puc-rio.br.
The documentation of the used libraries is available at :
IUP: http://www.tecgraf.puc-rio.br/manuais/iup
CD: http://www.tecgraf.puc-rio.br/manuais/cd
IM: http://www.tecgraf.puc-rio.br/manuais/im
Although there are many C files, only those beginning with imp are relevant for this project.
IMP.Ccontains the main module that manages the impressionist menu items and the algorithm starting point. The Impressionist menu items specify the algorithm parameters, except the Apply item, that set the filter over a RGB image resulting in a new image. The comprehension of the Apply function is fundamental to understand the other modules and how other pre-defined functions work , such as image creation, message display, file management, etc...
IMPBRUSH.Ccontains the drawing brush creation function and it is done.
IMPCLIP.Ccontains the brush clipping function, if the function is selected.
IMPTHETA.Ccontains the brush orientation calculus functions, if the brush is automatic through the gradient.
IMPPAINT.Ccontains the functions that draw the brush over the image, considering clipping and orientation.
Observation:the brush definition and the exported functions declaration are in IMP.H file.
The user controls the algorithm through many parameters.
Dimensions: (Length, Length Delta, Radius, Radius Delta) the Delta values works as intervals for the main values random variation. (default: 7, 3, 1.75, 0.25)
Rendering I: (Color Interpolation or Color Average) the color of the brush is determined by the color of its center. In the center, the color can be interpolated by the neighbors colors or is an average of the neighbors colors. (default: interpolation)
Rendering II: (Solid Color or Texture) the color of the brush can be modified by a texture modulation combined with a transparency value. (default: solid color)
Spacing: the brushes are placed trough the image in a uniform regular grid, controlled by a determined spacing.. (default: 2)
Clipping: Each brush drawing can respect (or not) the original image borders. (default: off)
Gaussian Kernel Size: the borders calculus uses the Sobel filter magnitude applied in the intensities image of the original image filtered by a Gaussian. (default: 11)
Orientation: (Theta, Estimate Theta, or Estimate Theta on Areas) Each stroke orientation can be determined by following an orientation or trough an angle estimate using the normal trough the gradient located under the brush.
ATTENTION: The algorithms work only for 24 bpp RGB images. For other images types the filter menu is not available.
Impressionist Filter for an image
Generate Uniform Strokes Calculate Intensity Image Calculate Gaussian for Sobel Calculate Sobel Calculate Gaussian for Orientation Calculate Stroke Orientation For Each Brush Clip Brush Paint BrushSome calculi are not executed upon the user choice. Although there is not necessary the brushes list for a unique image, it was used to prepare the code for the second part of the paper. Following the pseudo code and each proceeding description.
For Each Point in the grid [width / spacing, height / spacing] Generate a Brush in a free random position and mark the position as used.Beyond distributing the strokes uniformly and randomly ordered, the algorithm also modify the size and width of the brush with small random variations.
For Each Pixel intensity(x,y) = 0.30 red(x,y) + 0.59 green(x,y) + 0.11 blue(x,y)Works with a very used conversion to extract the intensity value of a RGB color.
for each pixel Calculate Gradient in X direction Calculate Gradient in Y direction Sobel = sqrt(Gx2 + Gy2) -1 0 1 Gradient mask in X: -2 0 2 -1 0 1 1 2 1 Gradient mask in Y: 0 0 0 -1 -2 -1
For Each Pixel Gaussian(x,y) = ( Intensity(x-1,y+1) + 2*Intensity(x,y+1) + Intensity(x+1,y+1) + 2*Intensity(x-1,y) + 4*Intensity(x,y) + 2*Intensity(x+1,y) + Intensity(x-1,y-1) + 2*Intensity(x,y-1) + Intensity(x+1,y-1) )/16; 1 2 1 Gaussian mask: 1/16 2 4 2 1 2 1
As convolution mask it was used a discretization of the Gaussian nucleus with 3x3 samples. To obtain major grade discretizations we used this same mask several times.
This algorithm was implemented in order to obtain a better performance, but we have the disadvantage of using more memory, because we pre calculate 2*Intensity for all image. The performance gain will be obtained because for each pixel the calculus will be executed once instead of the 4 that would be necessary if we have used other implementation.
Using the symmetric of the original sign (mirror) extended the image for the borders calculus.
For Each Brush Calculate Gradient on Brush Center using Bilinear Interpolation Brush Orientation is normal to the gradient directionAs each brush stroke is located in any point we interpolate the gradient in this point with the neighbors gradients. If the gradient is different from zero, the stroke angle is the gradient angle plus 90o ; if the angle is zero we use a random value for the angle.
Given a stroke, determined by its center (cx, cy) and orientation (dirx e diry) we will limit its size based on the borders showed by the Sobel image parameter. The implemented algorithm follows the stages bellow:
- starting in the center (cx,cy) we should go trough (step by step) in the orientation given by Sobel image (dirx, diry) using a Bilinear Interpolation for the value calculus in the desired point.
- this course should finish when a distance larger than the original stroke distance would be achieved or when a border would be detected. A border is detected when the Sobel image value decreases, when it is lower than the last value given.
- In the end of the last stage we would have the first edge of the stroke.
- To specify the second edge we would repeat the same process, but using as course orientation (-dirx, -diry).
Calculate Brush Color Calculate Brush Vertical Limits For Each Discrete Line Between Vertical Limits Calculate Horizontal Limits For Each Pixel in the horizontal limits pixel = brush colorIn this moment the brush drawing limits calculus considers the calculated clipping. The edges calculus try to make them round. In the brush borders there is an amortization of the pixel color value, considering the pixel value of the image.
Calculate Brush Color Calculate Brush Vertical Limits For Each Discrete Line Between Vertical Limits Calculate Horizontal Limits For Each Pixel in the horizontal limits Calculate The Pixel Texture Coordinates, u and v alpha = alpha_texture[u, v] / 255 intensity = intensity_texture[u, v] / 255 back_color = pixel pixel = (brush_color * intensity - back_color) * alpha + back_colorTwo texture images are used for the stroke render. One is used for defining intensity of the brush color and the other the brush transparency. The pixel color after the stroke will be:
actual_color * (transparency - 1) + brush_color * intensity * transparency
The intensity and transparency assume values inside the range of 0 to 1, while the pixel variable in the formula above refers to the pixel color before the stroke. The strokes are rectangular and without anti-aliasing because the borders soften effect can be obtained trough alpha texture.
The order which the stages of the algorithms were implemented determines an amount of necessary memory. We could reduce this quantity by modifying this order and executing some calculus simultaneously. These changes would complicate the comprehension of the painting process code, so we decided to use a few more memory to make the code clearer and more similar with the paper orientation.
We have achieved some images that demonstrate the paper idea. The algorithms settled well and the development process was very productive.
Some examples of the filter application:
|
Original |
Filtered |
|
bird (312Kb) |
impressionist bird (299Kb) |
|
fruit (111Kb) |
impressionist fruit (131Kb) |
|
lenna (158Kb) |
impressionist lenna (199Kb) |
|
portrait (74Kb) |
impressionist portrait (94Kb) |
|
under_water (117Kb) |
impressionistt under_water (130Kb) |
The brush stroke draw could be optimized but it would complicate even more the complex code but we could have more soft and well finished results with the stroke. The actual result can be accepted by an artistic point of view, because this is a perception problem. We have not implemented the estimate angle algorithm for zero gradient areas. In this cases a random value was used.
