Image processing operations

From TLMtracker

Jump to: navigation, search

This section describes all available image processing operations of TLM-Tracker.

Contents

Source Image

Here the source image of the current process pipeline can be chosen. In the first pipeline this can be either the brightfield- or the fluorescence image. In later pipelines also the final images of the previous process pipelines can be chosen (e.g. pipeline 1 and 2 in pipeline 3).

Usually the brightfield image is chosen as the source image of the first process pipeline and is thus the image which will be segmented. But if a constitutively expressed fluorescence marker is used, also the fluorescence image could be segmented.

Point operations

Point operations are functions that are performed on each single pixel of an image, independent of all the other pixels in that image.

Invert

Inverting the pixel values in the image produces the same image that would be found in a film negative. There are no parameters to this operation.

This operation often is the first step in a segmentation process. It is used when in the original image the cells are darker than the background, because conventionally the cells have to be brighter than the background.

Brightness

When changing the brightness of an image, the pixel values are mapped to a different range, with the result that the brightness is enhanced or decreased. The only parameter of this function is a factor in the range [-1, 1]. If this factor is smaller than 0, the brightness is decreased, if it is bigger than 0, the brightness is increased.

This operation can't enhance the quality of an image, so normally it won't be used in a segmentation process. But it can be used to make an image 'looking' better.

Contrast

When changing the contrast of an image, the pixel values are mapped to a different range, with the result that the contrast is enhanced or decreased. The only parameter of this function is a factor in the range [-1, 1]. If this factor is smaller than 0, the contrast is decreased, if it is bigger than 0, the contrast is increased.

This operation can't enhance the quality of an image, so normally it won't be used in a segmentation process. But it can be used to better visualize image details for the eye.

Gamma

When changing the gamma of an image, the pixel values are mapped to the new pixel values using a power function, with the result that both brightness and contrast change. The only parameter of this function is the gamma value, which has to be bigger than 0. If gamma is smaller than 1, the image gets brighter and the contrast is decreased, if gamma is bigger than 1, the image gets darker and the contrast is increased.

This operation can't enhance the quality of an image, so normally it won't be used in a segmentation process. But it can be used to make an image 'looking' better.

Roots

This operation generates a binary image with white pixels in the places where the pixel values in the original image equal zero. There are no parameters for this operation.

This operation can be used after a watershed transformation to get an binary image of the ridge lines.

Logical operations

Logical operations are point operations wich can only be executed on binary images. These operations consist of the application of logical operators on every pixel of the image.

Not (bw)

This operation applies a logical not on every pixel of a binary image. The result is an inverted image. There are no parameters for this operation.

This operation can be used together with the logical or operator to combine different markers (cell- and backgroundmarkers) for the marker-controlled watershed segmentation.

And (bw)

This operation combines two binary images by applying a logical and operator on every two pixels of them for determining the pixel values of the resulting image. This means that there will be a white pixel in the resulting image, if there are white pixels in both of the source images. This operation needs a second binary image as a parameter. Normally this will be the final image of a previous process pipeline.

This operation can be used to combine binary images.

Or (bw)

This operation combines two binary images by applying a logical or operator on every two pixels of them to determine the pixel values of the resulting image. This means that there will be a white pixel in the resulting image, if there are white pixels in one of the two source images or in both of them. This operation needs a second binary image as a parameter. Normally this will be the final image of a previous process pipeline.

This operation can be used together with the logical not operator to combine different markers (cell- and backgroundmarkers) for the marker-controlled watershed-segmentation.

Xor (bw)

This operation combines two binary images by applying a logical xor operator on every two pixels of them to determine the pixel values of the resulting image. This means that there will be a white pixel in the resulting image, if there is a white pixel either in the one or the other of the two source images. This operation needs a second binary image as a parameter. Normally this will be the final image of a previous process pipeline.

This operation can be used to combine binary images.

Local operations

Local operations are functions that calculate the new value of each single pixel with regard to an area in its surrounding, called its neighbourhood. They can be divided in two groups, linear and nonlinear filters, where a filter is an operation, which converts a source image via some kind of transformation to a result image. Another possibility for the subdivision of local operations is the division in low-pass and high-pass filters. Low-pass filters are used to inhibit noise and small details on images, whereas high-pass filters are used to emphasize edges.

Linear filters

Linear filters are operations, which convert the source image via a linear transformation to the result image. The new value of a pixel is calculated by a weighted sum of the pixel values in its neighbourhood.

Mean filter

When using the mean filter, the new value of a pixel is the arithmetic mean of the pixel values in its neighbourhood. This filter is a low-pass filter, i.e. it can be used to inhibit noise. The disadvantage of its usage is that edges get blurred. The only parameter of this operation is the 'radius', which actually is the half side length of a square defining the neighbourhood of a pixel. The bigger the radius, the better noise is inhibited, but also the edges get more blurred.

This filter can be used with a small radius (e.g. 1 pixel) to supress noise in the image.

http://homepages.inf.ed.ac.uk/rbf/HIPR2/mean.htm

Gaussian filter

The gaussian filter calculates the new value of a pixel by calculating the mean of the pixel values in its neighbourhood, weighted by a gaussian distribution. The result is that near pixels have a greater influence on the final value than pixels lying farther away. This filter is a low-pass filter, i.e. it can be used to inhibit noise. The disadvantage of its usage is that edges get blurred. This operation has two parameters. The first one is the 'radius', which actually is the half side length of a square defining the neighbourhood of a pixel. The radius defines which pixels are actually regarded when calculating the mean. The second parameter is sigma, which describes the width of the gaussian distribution. Sigma determines how the pixels are weighted to calculate the final value. With a big sigma the result of the gaussian filter is similiar to the result of the mean filter of the same radius.

This filter can be used to supress noise in the image (e.g. radius 3, sigma 0.7).

http://homepages.inf.ed.ac.uk/rbf/HIPR2/gsmooth.htm

Gradient filter

The gradient filter calculates the norm of the gradient operator executed on the image intensity function. The gradient operator is the sum of the partial deviations of a function in x- and y-direction. Here the gradient operator is approximated using the sobel operator. This filter is a high-pass filter, i.e. the result is an image which is bright in regions where are edges (big changes of the brightness) in the original image, and dark in regions where are uniform areas in the original image.

http://en.wikipedia.org/wiki/Gradient

http://en.wikipedia.org/wiki/Del

http://en.wikipedia.org/wiki/Sobel_operator

Edge detection

With the edge detection operation different methods of edge detection can be executed on the image. The methods which can be selected are the Sobel, Prewitt, Roberts, Log, and Canny method. The only parameter for all these methods is the 'level', which controls how strong the edges are which are recognized. With level > 1 only the stronger edges are detected, with level < 1 also weaker edges are detected. A description of the different methods can be found under the following link.

http://homepages.inf.ed.ac.uk/rbf/HIPR2/featops.htm

Non-linear filter

Nonlinear filters are operations, which convert the source image via a non-linear transformation to the result image.

Median filter

When using the median filter, the new value of a pixel is the median of the pixel values in its neighbourhood. The resulting image is then free of extreme pixel values. This filter is a low-pass filter, i.e. it can be used to inhibit noise. The disadvantage of its usage is that edges get blurred. The only parameter of this operation is the 'radius', which actually is the half side length of a square defining the neighbourhood of a pixel. The bigger the radius, the better noise is inhibited, and compared to the mean filter the edges get less blured.

This filter can be used with a small radius (e.g. 1 pixel) to supress noise in the image.

http://homepages.inf.ed.ac.uk/rbf/HIPR2/median.htm

Morphological filters

Morphological filters are operations which affect the shape of structures in the image. They mostly work with minimum- and maximum operators and thus belong to the group of non-linear filters. The minimum- and maximum operations are executed on the neighbourhood of each pixel, defined by a structuring element. A structuring element is a binary matrix whos elements with 1 define neighbourhood pixels, elements with 0 are ignored. To determine which pixels belong to the neighbourhood of a certain pixel, the structuring element is centered over this pixel and serves as a mask.

Erosion

The erosion operation is basically a minimum filter, i.e. the new pixel value is the minimum of the pixel values in its neighbourhood. Bright objects are eroded by this filter, i.e. they shrink, by ablating them from the edges. The erosion is the complementary operation to the dilation. The only parameter for this operation is a structuring element, defining the neighbourhood of a pixel. It is possible to select simple structuring elements with disk, square, octagon or diamond shapes and indicate their size. It is possible to create more complex, arbitrary structuring elements too.

This operation is mostly used in the editing of binary images.

http://en.wikipedia.org/wiki/Erosion_%28morphology%29

http://homepages.inf.ed.ac.uk/rbf/HIPR2/erode.htm

Dilation

The dilation operation is basically a maximum filter, i.e. the new pixel value is the maximum of the pixel values in its neighbourhood. Bright objects are dilated by this filter, i.e. they grow, by expanding them from the edges. The dilation is the complementary operation the erosion. The only parameter for this operation is a structuring element, defining the neighbourhood of a pixel. It is possible to select simple structuring elements with disk, square, octagon or diamond shapes and indicate their size. It is possible to create more complex, arbitrary structuring elements too.

This operation is mostly used in the editing of binary images.

http://en.wikipedia.org/wiki/Dilation_%28morphology%29

http://homepages.inf.ed.ac.uk/rbf/HIPR2/dilate.htm

Reconstruction

The reconstruction is basically a conditional dilation. The image is dilated repeatedly with an structuring element of radius 1, but the pixels are only set if they are also set in a second image, called the mask. If this is executed several times, there won't be further changes in the image, which is then reconstructed. The only parameter for this operation is the mask image, which can be one of the original brightfield or fluorescence images or the final image of a previous process pipeline.

This operation can be used in the editing of binary and gray scale images and is used in several combined operations (Opening by reconstruction, Closing by reconstruction, Substract Background).

http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.92.1083&rep=rep1&type=pdf

Morphological opening

Morphological opening performs an erosion operation followed by a dilation operation using the same structuring element. The effect of this operation is that small objects or image details a removed while the size and the shape of larger objects in the image are preserved. It is basically a low-pass filter. The only parameter for this operation is a structuring element, defining the neighbourhood of a pixel. It is possible to select simple structuring elements with disk, square, octagon or diamond shapes and indicate their size. It is possible to create more complex, arbitrary structuring elements too.

This operation is mostly used in the editing of binary images.

http://en.wikipedia.org/wiki/Opening_%28morphology%29

http://homepages.inf.ed.ac.uk/rbf/HIPR2/open.htm

Opening by reconstruction

Opening by reconstruction performs an erosion operation followed by a reconstruction operation with the original image as mask. The effect of this operation is that small objects or image details a removed while the size and the shape of larger objects in the image are preserved. It is basically a low-pass filter. It is an operation with a similiar effect as the morphological opening. The main difference is, that the shapes of the objects are reconstructed exactly, while there can be differences when using morphological opening. The only parameter for this operation is a structuring element for the erosion, defining the neighbourhood of a pixel. It is possible to select simple structuring elements with disk, square, octagon or diamond shapes and indicate their size. It is possible to create more complex, arbitrary structuring elements too.

This operation is mostly used in the editing of binary images.

Morphological closing

Morphological closing performs a dilation operation followed by an erosion operation using the same structuring element. The effect of this operation is that holes in objects are closed while the size and the shape of the object is preserved. It is also possible that several small nearby objects are combined to one large object. The only parameter for this operation is a structuring element, defining the neighbourhood of a pixel. It is possible to select simple structuring elements with disk, square, octagon or diamond shapes and indicate their size. It is possible to create more complex, arbitrary structuring elements too.

This operation is mostly used in the editing of binary images.

http://de.wikipedia.org/wiki/Closing

http://homepages.inf.ed.ac.uk/rbf/HIPR2/close.htm

Closing by reconstruction

Closing by reconstruction first performs a dilation operation. The image then is inverted, reconstructed with the inverted original image as mask, and finally inverted again to produce the final image. The effect of this operation is that holes in objects are closed while the size and the shape of the object is preserved. It is an operation with a similiar effect as the morphological closing. The main difference is, that the shapes of the objects are reconstructed exactly, while there can be differences when using morphological closing. The only parameter for this operation is a structuring element for the dilation, defining the neighbourhood of a pixel. It is possible to select simple structuring elements with disk, square, octagon or diamond shapes and indicate their size. It is possible to create more complex, arbitrary structuring elements too.

This operation is mostly used in the editing of binary images.

Substract Background

Substract background is a combined operation that determines the background of an image and substracts it from the original image, so that only foreground objects remain. The background is determined by an opening by reconstruction operation followed by a closing by reconstruction operation using the same structuring element. The only parameter for this operation is this structuring element, defining the neighbourhood of a pixel. It is possible to select simple structuring elements with disk, square, octagon or diamond shapes and indicate their size. It is possible to create more complex, arbitrary structuring elements too. The radius of the structuring element must be bigger than the radius of every object in the image.

This operation is used to create a more uniform background to facilitate the further editing of an image.

Remove objects (bw)

This operation removes small objects with a certain maximal area from a binary image. The only parameter of this operation is the maximal area of the objects, which will be removed, in pixels.

This operation can be used after a thresholding operation to remove single background pixels or small areas which erroneously have been recognized as foreground objects.

Fill gaps

This operation fills dark areas within continuous objects. There are no parameters for this operation.

This operation can be used to remove black artifact pixels inside objects after a segmentation.

Impose minima

This operation creates regional minima in an image at the places specified by a binary marker image. The regions where the marker image is true will be minima in the resulting image (dark regions). All other local minima in the original image will be removed by morphological operations, so that only the minima given by the marker image remain. The only parameter for this operation is the marker image, which can be one of the original images, either the brightfield or the fluorescence image, or the final image of one of the previous process pipelines.

This operation is used to combine the image and the markers for the marker-controlled watershed segmentation.

Distance transformation (bw)

This operation executes a distance transformation on a binary image. The pixel values in the resulting image correspond to the distance of this pixel to the next object in the original image. The brighter a pixel, the farther it is away. This operation has no parameters.

This operation can be used to produce background markers, for a marker-controlled watershed segmentation.

http://homepages.inf.ed.ac.uk/rbf/HIPR2/distance.htm

Thinning (bg)

This operation thins objects to lines. It shrinks them iteratively similiar to the erosion operation, but in contrast to this operation the objects won't vanish, so that the topology of the image is preserved. The only parameter for this operation is the number of executed iterations.

This operation is mostly used in the editing of binary images.

http://homepages.inf.ed.ac.uk/rbf/HIPR2/thin.htm

Thickening (bw)

This operations thickens objects. It expands them iteratively similiar to the dilation operation, but in contrast to this operation the objects won't fuse, so that the topology of the image is preserved. The only parameter for this operation is the number of executed iterations.

This operation is mostly used in the editing of binary images.

Label (bw)

This operation labels connected objects in binary images. Every connected object gets an integer index and the values of the pixels in resulting image, belonging to this object are set to this index. There are no parameters for this operation.

This operation is used to label segmented images to prepare the further investigation of the objects properties.

http://homepages.inf.ed.ac.uk/rbf/HIPR2/label.htm

Segmentation

The segmentation is the process of partitioning a digital image into multiple segments to detect objects in a pixel graphic. For this task several algorithms exist. The selection of the optimal algorithm highly depends on the image data, so normally it is necessary to try different algorithms to find the one serving best for a special task.

TLM-Tracker provides four different segmentation algorithms: Threshold, Robust Automatic Threshold Selection (RATS), Watersheds and Level-set methods/Active contours where the latter is divided into the three submethods Chan-Vese, GAC and Hybrid. The following table gives a brief overview of the main advantages and disadvantages of these methods. In general, the execution time of an algorithm can be (severely) reduced by the size of the analyzed image.

Method Execution speed Pro Con
Threshold fastest
  • fastest execution speed
  • works well on images with a high signal-to-noise ratio
  • error-prone for low signal-to-noise ratios
  • problems with nonuniform intensities across an image
  • not able to separate objects of the same intensity that share a boundary (touching/overlapping cells)
RATS fast
  • still fast execution speed
  • overcomes the problem with nonuniform intensities across an image by defining local thresholds for subimages
  • error-prone for low signal-to-noise ratios
  • not able to separate objects of the same intensity that share a boundary (touching/overlapping cells)
Watersheds slow
  • able to separate object of the same intensity that share a boundary (touching/overlapping cells)
  • execution speed can be slow for large images
  • tends to over-segmentations of images (can be overcome by marker-controlled watersheds)
Level-set slowest
  • best results for low and varying signal-to-noise ratios and nonuniform intensities across an image
  • three pre-defined submethods (Chan-Vese, GAC, Hybrid) to match the properties of different data sets
  • execution speed can be very slow for large images
  • might need manual parameter adaptation for optimal results


Threshold

The simplest segmentation method is the conversion of a gray-scale image to a binary image using a threshold value. Pixels brighter than the threshold are white pixels in the resulting image, pixels darker than the threshold are black pixels. After conversion, connected areas of white pixels are interpreted as single objects. This method is useful if the brightness of the objects differs significantly from the brightness of the background. There are several improvements of this method where the threshold value is selected automatically. As a parameter of this operation it is possible to select the method. Supported methods are: manual selection of the threshold value, otsu- [1], iterative- [2][3], local enthropy- [4], joint enthropy- [5], relative enthropy- [6] and renyis enthropy-method [7]. When using manual selection of the threshold value it is possible to select a threshold value between 0 (black) and 1 (white). When using one of the other methods a factor can be selected to modify the automatically determined value.

http://en.wikipedia.org/wiki/Thresholding_%28image_processing%29

RATS

This operation executed an adaptive thresholding algorithm on a gray-scale image to convert it into a binary image. The Robust Automatic Threshold Selection (RATS) is a fast and noise robust automatic thresholding method based on gradients in the image. The basic idea is to choose the threshold at the intensity where the gradients are the highest. The RATS-Algorithm [8] divides an image into a number of subimages using a quadtree and determines if in a subimage there are enough edges present to effectively threshold this subimage with a threshold-value determined for it. If this is not the case the threshold value of a parent subimage in the quadtree, where enough edges are present, is applied. This function has been implemented based on the RATS-plugin for ImageJ from Ben Tupper. They adapted it from a C-implementation by M.H.F. Wilkinson, 2000[9]. Parameters for this function are σ·λ, the product of the standard deviation of the image background (σ) and a scaling factor (λ), a factor which controls how many edges have to be found in a subimage until it own threshold value is used and the minimal height and width of the smallest leaves of the quadtree.

This function should be executed on images as small as possible, because it can be relatively slow. The critical speed factor is the minimal leave size, as the leaves are created recursively from the bigger leaves. So if the function is too slow it should be considered to increase the minimal leave size.

http://pacific.mpi-cbg.de/wiki/index.php/RATS:_Robust_Automatic_Threshold_Selection

http://rsbweb.nih.gov/ij/plugins/rats/index.html

http://books.google.de/books?hl=de&lr=&id=8JJwuU13YPAC&oi=fnd&pg=PA135&dq=wilkinson+threshold+quad+tree&ots=UwrVlX51al&sig=Dd2zYGlALujL-50nvL-VJ7Ys-ZY#v=onepage&q=&f=false

http://books.google.de/books?hl=de&lr=&id=lx2YHy6SZHQC&oi=fnd&pg=PA261&dq=Rapid+automatic+segmentation+of+fluorescent+and+phase-contrast+images+of+bacteria&ots=usCC8AYjX1&sig=r3wzR_g9FyW_H3u_VKM9AGOHjnA#v=onepage&q=Rapid%20automatic%20segmentation%20of%20fluorescent%20and%20phase-contrast%20images%20of%20bacteria&f=false

Watershed transformation

This operation executes a watershed segmentation on a gray-scale image and labels the resulting regions. This function interpretes an image as a height-profile, which is flooded starting at the regional minima. The lines where the flooded areas touch are called watersheds and marked. Finally each water bassin is interpreted as an independent object. In the resulting image each object is labeled with an inidividual index, resulting in an image similiar to the result of the label operation. There are no parameters for this operation.

As can be seen below, a simple execution of the watershed segmentation can result in an oversegmented image. For this reason normally a marker-controlled watershed segmentation is used.

http://en.wikipedia.org/wiki/Watershed_%28image_processing%29

http://cmm.ensmp.fr/~beucher/wtshed.html

Level-set methods

This operation executes different segmentation methods based on active contours and level-set functions. The resulting binary image contains the contours of the recognized objects, which can then be filled using the fill gaps operation. This function has been implemented using the 2D/3D image segmentation toolbox from Yan Zhang. The different methods which can be selected are: Chan-Vese method [10], GAC method [11] and the hybrid method [12].

The different methods have a number of different parameters. In the method-Panel a number of parameters for each method can be found controlling the evolution of the level-set function. These parameters have to be adjusted so that finally the level-set function describes the contours of the desired objects. Common parameters of all methods are Δt, which is the timestep for one iteration in the iteratively executed determination of the level-set function, and a checkbox which controls if the evolution of the level-set function can be seen on the screen. The GAC- and the hybrid-method need a gradient image of the original image to work with. In the gradient-Panel some parameters can be found, which control how this gradient image is calculated. Normally it is not necessary to change these. Finally in the Signed distance function-Panel a number of parameters can be found, which control the shape of the initial level-set function. These can be circles, rectangles and circle- and rectangle arrays of different dimensions.

Usually it is necessary to modify at least the parameters which control evolution of the level-set function to adjust the method for a certain image. These functions should be run on images as small as possible, because they are relatively slow.

If there is a problem using these functions it could be necessary to recompile their binaries. This is only possible when using the source code and is described here.

http://www.mathworks.com/matlabcentral/fileexchange/24998-2d3d-image-segmentation-toolbox

References

  1. Otsu, Nobuyuki (1975) A Threshold Selection Method from Gray-Level Histograms.
  2. Ridler, T. W. (1978) Picture Thresholding Using an Iterative Selection Method.
  3. Yang, Xinmin (2001) Evaluation of biofilm image thresholding methods.
  4. Chang, Chein-I (1994) A relative entropy-based approach to image thresholding.
  5. Chang, Chein-I (1994) A relative entropy-based approach to image thresholding.
  6. Chang, Chein-I (1994) A relative entropy-based approach to image thresholding.
  7. Sahoo, Prasanna (1997) Threshold selection using Renyi's entropy.
  8. Kitler, J., Illingworth, J. and Foglein J. (1985) Threshold selection based on a simple image statistic. Computer vision, graphics, and image processing, 30:125–147
  9. Wilkinson, M.H.F. (1998) Segmentation Techniques in Image Analysis of Microbes.
  10. Chan, T. F., Vese, L. A. (2001) Active contours without edges. IEEE Transaction on Image Processing, 10(2):266-277
  11. Caselles, V., Kimmel, R., Sapiro, G. (1997) Geodesic Active Contours. IJCV, 22(1):61-79
  12. Zhang, Y., Matuszewski, B. J., Shark, L.-K., Moore, C. J. (2008) Medical Image Segmentation Using New Hybrid Level-Set Method. IEEE International Conference on Biomedical Visualisation, MEDi08VIS, London, pp.71-76
Personal tools