Image Analysis: Morphological Operations
The following is an expert from Dewar - Characterization and Evaluation of Aged 20Cr32Ni1Nb Stainless Steels, and references a stainless steel used in this thesis. The content of this article can also be applied to a general case, and should not be limited to to the presented example.
Digital image processing has been incorperated into many scientific applications over the last decade including medical imaging, manufacturing processing with robots, and automatic vehicle driving systems. Basic concepts from image processing theory can also be applied to metallography, and assisting the user with identifying features, and quantifying the shape, size, and area fraction of these features. This is especially useful if hundreds of micrographs need to be quantified to give statistically accurate values representing the overall microstructure. In this section two methods will be proposed for assessing the area fraction of the microstructure present in 2032Nb alloys, first using elemental mapping procedures with EPMA, and WDS and then processing those maps with the Image Analysis Toolkit in Matlab, and the secondly using more complicated techniques using the OpenCV image processing library with Python to process backscattered images obtained from SEM in determining area fraction, and size of the precipitates. A few sources further describing digital image processing are the Hypermedia Image Processing Reference (HIPR2) (http://tinyurl.com/6o3n26c), the Matlab Image Processing Toolbox Documentation (http://tinyurl.com/8os5tvo), and Digital Image Processing by Gonzalez and Woods .
In element mapping from EPMA, x-ray intensities are collected from an array of points mapped over the sample, which are then output as false color values ranging over the visible spectrum from violet (low intensity) to red (high intensity). For the maps presented in Figure 1, a 1μm∕pixel resolution was used, where a 512px× 512px false color image was produced. Color thresholding is then used to separate the false color pixels that are within a certain range of the color spectrum. Since certain phases are highly concentrated with specific elements, color thresholding the element maps should be able to pick out all of the precipitates in the microstructure based on their respective constituents. For example, from stoichiometry G-phase can be differentiated from NbC based off of the niobium constituent concentration where NbC consists of ~50% niobium, whereas G-phase only consists of ~21% niobium. This statement is under the assumption that any other constituent that is soluble in the first sublattice of NbC will be dilute, and will not offset the niobium concentration by any significant amount. This assumption should be confirmed with EDS measurements. Niobium carbides can then be separated from the niobium element map by thresholding between the first few high intensity false colors (i.e. read and orange). It should be noted that x-ray intensities from each phase are on a relative scale, and will fluctuate from session to session due to instrument calibration, and sample setup. Threshold limits for each datasets must be re-evaluated by comparing the false color maps to the backscattered images. Error associated with discrepancies between the thresheld image, and the backscattered micrograph should be incorporated into the total error.
While G-phase could also be segmented from the just the niobium map, it is more reliable to also threshold the silicon map and evaluate where areas of both high silicon, and high niobium exist. The same technique of thresholding for chromium can be done to section the chrome carbide precipitates. After thresholding, the images are converted into binary format (0 for black, 1 for other) for further analysis with Matlab. Thresholding procedures can either be accomplished from the Photoshop script (EPMAColorThreshold.jsx), or the Matlab script (EPMASegmentation.m) discussed further in section A.2.
Figure 2 shows how the silicon and niobium maps after thresholding are multiplied together to define where G-phase might exist. Since both the silicon and niobium maps after thresholding are in binary format, only pixels for each map that are both 1’s will appear when the two maps are multiplied together. The error between the multiplied map and the backscattered image are compared, and an error is added to the area fraction estimates.
Converting the element maps into binary format is important for the computer to be able to distinguish between different connected components. A connected component is defined as a group of pixels that are connected either by their faces, or their edges. Figure 3 illustrates the two different types of connected components as 4-connected (faces), or 8-connected (faces and edges). For area thresholding operations, and contour drawing algorithms each connected component specified by the connectivity criteria are labeled iteratively as seen in Table 1. Labeling the components of the image helps categorize them where they can then undergo an area, or surface area thresholding operation, or the size and area fraction data can be output. Since some of the precipitates are very fine in the microstructure (< 1μm), they cannot be fully resolved by EPMA (spatial resolution = 1μm) where two separate precipitates may be touching each other either on a pixel edge or face. A 4-connectivity is used to try and minimize any connectivity error as it is assumed that all the precipitates are elliptical or globular and would not be connected only by a single pixel edge. Improving the spatial resolution of the image will help with any connectivity issues, where features should be separated by at least a few pixels.
While improving the resolution of a micrograph is the best way to reduce connectivity error there are morphological operations that can be utilized to artificially reduce this error. Morphological operations deal with extracting image components that can be used to define a features shape, and its boundaries . The two basic morphological operations are called dilation, and erosion which were derived using set theory. A structural element (called a kernel) which is by default a 3 × 3 pixel array is iteratively placed through a binary image where intersections with connected components cause the element to be added or subtracted from the image.
Table 2 shows a dilation operation of a 2 × 2 kernel where the kernel is added to the image at any iteration where there is an intersection with a connected component. From set theory dilation is defined as,
where A is the original component, B is the kernel, and z is translation of a reflected kernel around the origin of A . Eq. 6 defines dilation as reflected kernels that are intersecting with A which exist as a subset of A. Variability in the shape of the kernel can give rise to some interesting results, such as using a rectangular, or spherical kernel, but for this study the kernel was limited to a 3 × 3 square, where multiple dilation operations were performed where needed.
Erosion follows the same algorithm as dilation where a kernel is iteratively moved along the boundary of a connected component, however if at least one of the pixels of the kernel is intersecting with the original component that kernel erases the intersecting pixels.Table 3 visually shows the erosion operation, where after the operation only one pixel remains of the original component. Erosion can be used in some cases as a basic operation for removing elements smaller than the defined kernel, or for separating objects with different morphologies, for example squares and circles of the same size. Erosion operations are less computationally intensive than size thresholding operations. The set theory definition for erosion is
which says that translated B elements contained in A should be subtracted from A. It should be noted that any morphological operations will degrade the exact morphology of the original components by some extent no matter how small the kernal is.
Dilation and erosion are proved to be exact opposite operations of each other which can be easily visualized. With this in mind these two operations can be combined in different orders to obtain different effects. Opening is defined as an erosion operation followed by a dilation operation of the same structural element B. Opening will erode all of the elements in A, where any elements smaller than the kernel will be removed. The surviving elements will then be dilated back to their original size, and shape. Some morphology of the elements will be lost with this type of operation, but should not be drastic provided the correct kernel is chosen. On the other hand Closing is defined as a dilation operation followed by an erosion operation of the same kernel. The closing operation is mainly used to fill internal holes in the original element that are smaller than the kernel while still maintaining the original shape and size. Closing can also be used to join very small elements together into one component which could be useful in determining grain boundary area, or even used to get well defined grain boundaries for grain size calculations.
Morphological operations are very useful for resolving resolution problems in micrographs where thresholding may cause some separate components to be joined together. However, using morphological operations will slightly degrade the morphology of the original image, and should be avoided for any size, or area fraction thresholding/analysis. Countour finding, or border following algorithms are common in many image processing libraries to compute connected components area, perimeter, and topological structure. In the border following algorithm proposed by Suzuki and Abe  border points for the outer borders of connected components are found, and then traced until it is reconnected with the original border point. Hole borders are then found inside the outer border, and traced as well. Each outer border and hole border is then labeled and categorized in a hierarchical tree. For some micrographs (especially optical), pits or holes in the microstructure can be filled by only plotting the outer borders for each component. In cases where a phase is encased by a separate phase (i.e. NbC and G-phase) holes must not be filled, or the NbC volume fraction should be subtracted from the G-phase fraction.
With morphological operations interdendritic structures can be seperated from intradendritic structures. From the EPMA maps, it can be observed that the interdendritic and intradendritic G-phase exist in the fully aged microstructure. The steps of the following procedure are shown in Figure 4. After thresholding the g-phase constituents described in section 10.1, the binary image (A) is dilated by a structural element B until the interdendritic regions are completely agglomerated. A closing operation is then performed to get rid of any noise in the image, followed by an area thresholding operation to subtract the intradendritic region from the image. The result can then be used as a mask on the original image to separate the interdendritic and intradendritic microstructure.
Element mapping is the major technique for evaluating phase fractions in micrographs, however using instrumentation like an EPMA can be very expensive, and time consuming. The same image analysis techniques discussed above can be applied to backscattered micrographs to determine phase fraction and the size of the precipitates in the microstructure. Producing a high resolution backscattered image in an SEM of 1000μm2 can take between 20-30 minutes to raster, whereas five element maps in EPMA of the same area can take upwards of 5-6 hours. With the current design matrix shown inTable 2 mapping would take over 320 hours of EPMA instrument time to process, and only 20 hours of instrument time in an SEM. Image thresholding on one greyscale backscattered image compared to five false color element maps is less reliable, and will produce much more noise and a larger error in the binary images for each phase; however morphological techniques can be utilized to minimize these factors. This requires more post processing time from the user than element mapping. This technique also assumes that proper characterization of the material has been done with EDS/WDS, and that the system is properly understood in advance. An example of the output results from the python image analysis script is shown in Figure 5.
 Shinozaki, K., Kuroki, H., Nakao, Y., Nishimoto, K., Inui, M., Takahashi, M.. Deterioration of weldability of long term aged HP heat-resistant cast steel containing Nb, Mo and W. Quarterly Journal of the Japan Welding Society 1998;16(2):223–232.
 Zhang, T., Wang, Q., Song, X., Fu, H.. Effect of electromagnetic centrifugal casting on solidification microstructure of cast high speed steel roll. Materialwissenschaft und Werkstofftechnik 2011;42(6):557–561.
 Wolczynski, W., Krajewski, W., Ebner, R., Kloch, J.. The use of equilibrium phase diagram for the calculation of non-equilibrium precipitates in dendritic solidification. theory. Calphad 2001;25(3):401–408.
- Auger Electron Microscope. 4, 9, 10
- Backscattered Electron. 7
- Energy Dispersive X-ray Spectroscopy. 6, 8–10
- Electron Probe Microanalysis. 4, 10
- Secondary Electron. 6, 7
- Scanning Electron Microscope. 4, 6, 9
- Wavelength Dispersive Spectroscopy. 9, 10
- X-ray Diffraction. 4, 10