Observer-dependent variability of the thresholding step in the quantitative analysis of soil images and X-ray microtomography data

Philippe C. Baveye, Magdeline Laba, Wilfred Otten, Liesbeth Bouckaert, Rohit R. Goswami, Dmitri V. Grinev, Alasdair N. Houston, Yaoping Hu, Jianli Liu, Sacha Mooney, Steven Sleutel, Ana Tarquis, Wei Wang, Qiao Wei, Mehmet Sezgin

Research output: Contribution to journalArticle

  • 78 Citations

Abstract

For the investigation of many geometrical features of soils, computer-assisted image analysis has become a method of choice over the last few decades. This analysis involves numerous steps, regarding which subjective decisions have to be made by the individuals conducting the research. This is particularly the case with the thresholding step, required to transform the original (color or greyscale) images into the type of binary representation (e.g., pores in white, solids in black) needed for fractal analysis or simulation with Lattice–Boltzmann models. Limited information exists at present on whether different observers, analyzing the same soil, would be likely to obtain similar results. In this general context, the first objective of the research reported in this article was to determine, through a so-called “round-robin” test, how much variation exists among the outcomes of various image thresholding strategies (including any image pre-treatment deemed appropriate), routinely adopted by soil scientists. Three test images – of a field soil, a soil thin section, and a virtual section through a 3-dimensional CT data set – were thresholded by 13 experts, worldwide. At the same time, variability of the outcomes of a set of automatic thresholding algorithms, applied to portions of the test images, was also investigated. The experimental results obtained illustrate the fact that experts rely on very different approaches to threshold images of soils, and that there is considerable observer influence associated with this thresholding. This observer dependence is not likely to be alleviated by adoption of one of the many existing automatic thresholding algorithms, many of which produce thresholded images that are equally, or even more, variable than those of the experts. These observations suggest that, at this point, analysis of the same image of a soil, be it a simple photograph or 3-dimensional X-ray CT data, by different individuals can lead to very different results, without any assurance that any of them would be even approximately “correct” or best suited to the objective at hand. Different strategies are proposed to cope with this situation, including the use of physical “standards”, adoption of procedures to assess the accuracy of thresholding, benchmarking with physical measurements, or the development of computational methods that do not require binary images.
Original languageEnglish
Pages (from-to)51–63
Number of pages13
JournalGeoderma
Volume157
Issue number12
DOIs
StatePublished - Jun 2010

Fingerprint

soil
testing
image analysis
methodology
micro-computed tomography
photographs
quantitative analysis
hands
pretreatment
color
fractal analysis
benchmarking
thin section
photograph
transform
simulation

Cite this

Baveye, P. C., Laba, M., Otten, W., Bouckaert, L., Goswami, R. R., Grinev, D. V., ... Sezgin, M. (2010). Observer-dependent variability of the thresholding step in the quantitative analysis of soil images and X-ray microtomography data. Geoderma, 157(12), 51–63. DOI: 10.1016/j.geoderma.2010.03.015

Baveye, Philippe C.; Laba, Magdeline; Otten, Wilfred; Bouckaert, Liesbeth; Goswami, Rohit R.; Grinev, Dmitri V.; Houston, Alasdair N.; Hu, Yaoping; Liu, Jianli; Mooney, Sacha; Sleutel, Steven; Tarquis, Ana; Wang, Wei; Wei, Qiao; Sezgin, Mehmet / Observer-dependent variability of the thresholding step in the quantitative analysis of soil images and X-ray microtomography data.

In: Geoderma, Vol. 157, No. 12, 06.2010, p. 51–63.

Research output: Contribution to journalArticle

@article{01c061d2d2af4d7491bdb31f3082f0c0,
title = "Observer-dependent variability of the thresholding step in the quantitative analysis of soil images and X-ray microtomography data",
abstract = "For the investigation of many geometrical features of soils, computer-assisted image analysis has become a method of choice over the last few decades. This analysis involves numerous steps, regarding which subjective decisions have to be made by the individuals conducting the research. This is particularly the case with the thresholding step, required to transform the original (color or greyscale) images into the type of binary representation (e.g., pores in white, solids in black) needed for fractal analysis or simulation with Lattice–Boltzmann models. Limited information exists at present on whether different observers, analyzing the same soil, would be likely to obtain similar results. In this general context, the first objective of the research reported in this article was to determine, through a so-called “round-robin” test, how much variation exists among the outcomes of various image thresholding strategies (including any image pre-treatment deemed appropriate), routinely adopted by soil scientists. Three test images – of a field soil, a soil thin section, and a virtual section through a 3-dimensional CT data set – were thresholded by 13 experts, worldwide. At the same time, variability of the outcomes of a set of automatic thresholding algorithms, applied to portions of the test images, was also investigated. The experimental results obtained illustrate the fact that experts rely on very different approaches to threshold images of soils, and that there is considerable observer influence associated with this thresholding. This observer dependence is not likely to be alleviated by adoption of one of the many existing automatic thresholding algorithms, many of which produce thresholded images that are equally, or even more, variable than those of the experts. These observations suggest that, at this point, analysis of the same image of a soil, be it a simple photograph or 3-dimensional X-ray CT data, by different individuals can lead to very different results, without any assurance that any of them would be even approximately “correct” or best suited to the objective at hand. Different strategies are proposed to cope with this situation, including the use of physical “standards”, adoption of procedures to assess the accuracy of thresholding, benchmarking with physical measurements, or the development of computational methods that do not require binary images.",
author = "Baveye, {Philippe C.} and Magdeline Laba and Wilfred Otten and Liesbeth Bouckaert and Goswami, {Rohit R.} and Grinev, {Dmitri V.} and Houston, {Alasdair N.} and Yaoping Hu and Jianli Liu and Sacha Mooney and Steven Sleutel and Ana Tarquis and Wei Wang and Qiao Wei and Mehmet Sezgin",
year = "2010",
month = "6",
doi = "10.1016/j.geoderma.2010.03.015",
volume = "157",
pages = "51–63",
journal = "Geoderma",
issn = "0016-7061",
publisher = "Elsevier",
number = "12",

}

Baveye, PC, Laba, M, Otten, W, Bouckaert, L, Goswami, RR, Grinev, DV, Houston, AN, Hu, Y, Liu, J, Mooney, S, Sleutel, S, Tarquis, A, Wang, W, Wei, Q & Sezgin, M 2010, 'Observer-dependent variability of the thresholding step in the quantitative analysis of soil images and X-ray microtomography data' Geoderma, vol 157, no. 12, pp. 51–63. DOI: 10.1016/j.geoderma.2010.03.015

Observer-dependent variability of the thresholding step in the quantitative analysis of soil images and X-ray microtomography data. / Baveye, Philippe C.; Laba, Magdeline; Otten, Wilfred; Bouckaert, Liesbeth; Goswami, Rohit R.; Grinev, Dmitri V.; Houston, Alasdair N.; Hu, Yaoping; Liu, Jianli; Mooney, Sacha; Sleutel, Steven; Tarquis, Ana; Wang, Wei; Wei, Qiao; Sezgin, Mehmet.

In: Geoderma, Vol. 157, No. 12, 06.2010, p. 51–63.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Observer-dependent variability of the thresholding step in the quantitative analysis of soil images and X-ray microtomography data

AU - Baveye,Philippe C.

AU - Laba,Magdeline

AU - Otten,Wilfred

AU - Bouckaert,Liesbeth

AU - Goswami,Rohit R.

AU - Grinev,Dmitri V.

AU - Houston,Alasdair N.

AU - Hu,Yaoping

AU - Liu,Jianli

AU - Mooney,Sacha

AU - Sleutel,Steven

AU - Tarquis,Ana

AU - Wang,Wei

AU - Wei,Qiao

AU - Sezgin,Mehmet

PY - 2010/6

Y1 - 2010/6

N2 - For the investigation of many geometrical features of soils, computer-assisted image analysis has become a method of choice over the last few decades. This analysis involves numerous steps, regarding which subjective decisions have to be made by the individuals conducting the research. This is particularly the case with the thresholding step, required to transform the original (color or greyscale) images into the type of binary representation (e.g., pores in white, solids in black) needed for fractal analysis or simulation with Lattice–Boltzmann models. Limited information exists at present on whether different observers, analyzing the same soil, would be likely to obtain similar results. In this general context, the first objective of the research reported in this article was to determine, through a so-called “round-robin” test, how much variation exists among the outcomes of various image thresholding strategies (including any image pre-treatment deemed appropriate), routinely adopted by soil scientists. Three test images – of a field soil, a soil thin section, and a virtual section through a 3-dimensional CT data set – were thresholded by 13 experts, worldwide. At the same time, variability of the outcomes of a set of automatic thresholding algorithms, applied to portions of the test images, was also investigated. The experimental results obtained illustrate the fact that experts rely on very different approaches to threshold images of soils, and that there is considerable observer influence associated with this thresholding. This observer dependence is not likely to be alleviated by adoption of one of the many existing automatic thresholding algorithms, many of which produce thresholded images that are equally, or even more, variable than those of the experts. These observations suggest that, at this point, analysis of the same image of a soil, be it a simple photograph or 3-dimensional X-ray CT data, by different individuals can lead to very different results, without any assurance that any of them would be even approximately “correct” or best suited to the objective at hand. Different strategies are proposed to cope with this situation, including the use of physical “standards”, adoption of procedures to assess the accuracy of thresholding, benchmarking with physical measurements, or the development of computational methods that do not require binary images.

AB - For the investigation of many geometrical features of soils, computer-assisted image analysis has become a method of choice over the last few decades. This analysis involves numerous steps, regarding which subjective decisions have to be made by the individuals conducting the research. This is particularly the case with the thresholding step, required to transform the original (color or greyscale) images into the type of binary representation (e.g., pores in white, solids in black) needed for fractal analysis or simulation with Lattice–Boltzmann models. Limited information exists at present on whether different observers, analyzing the same soil, would be likely to obtain similar results. In this general context, the first objective of the research reported in this article was to determine, through a so-called “round-robin” test, how much variation exists among the outcomes of various image thresholding strategies (including any image pre-treatment deemed appropriate), routinely adopted by soil scientists. Three test images – of a field soil, a soil thin section, and a virtual section through a 3-dimensional CT data set – were thresholded by 13 experts, worldwide. At the same time, variability of the outcomes of a set of automatic thresholding algorithms, applied to portions of the test images, was also investigated. The experimental results obtained illustrate the fact that experts rely on very different approaches to threshold images of soils, and that there is considerable observer influence associated with this thresholding. This observer dependence is not likely to be alleviated by adoption of one of the many existing automatic thresholding algorithms, many of which produce thresholded images that are equally, or even more, variable than those of the experts. These observations suggest that, at this point, analysis of the same image of a soil, be it a simple photograph or 3-dimensional X-ray CT data, by different individuals can lead to very different results, without any assurance that any of them would be even approximately “correct” or best suited to the objective at hand. Different strategies are proposed to cope with this situation, including the use of physical “standards”, adoption of procedures to assess the accuracy of thresholding, benchmarking with physical measurements, or the development of computational methods that do not require binary images.

U2 - 10.1016/j.geoderma.2010.03.015

DO - 10.1016/j.geoderma.2010.03.015

M3 - Article

VL - 157

SP - 51

EP - 63

JO - Geoderma

T2 - Geoderma

JF - Geoderma

SN - 0016-7061

IS - 12

ER -

Baveye PC, Laba M, Otten W, Bouckaert L, Goswami RR, Grinev DV et al. Observer-dependent variability of the thresholding step in the quantitative analysis of soil images and X-ray microtomography data. Geoderma. 2010 Jun;157(12):51–63. Available from, DOI: 10.1016/j.geoderma.2010.03.015