Do we trust image measurements? Variability, accuracy and traceability of image features

摘要

The paper addresses the problem of understanding quality of image measurements extracted using widely used software libraries from large images. Image measurements (features) are extracted using software packages that vary in terms of programming languages, theoretical formulas for the same image feature, algorithmic implementations, input parameters, units of measurements, and definitions of image regions of interest. Our motivation is to quantify numerical variability of image features across software packages and determine image accuracy with respect to reference images. In addition, our objective is to enable scientists to extract any image features of interest from heterogeneous software libraries and gain provenance of every extracted numerical feature value. The provenance information is critical to achieve traceability of computations in terascale imaging. We pursue this objective by designing a client-server system that integrates image feature extractions from open source libraries such as ImageJ/Fiji, Python (scikit-image), CellProfiler, and in-house Java software packages. The system becomes useful for evaluating quality of image measurements, leveraging distributed computational resources for feature computations over big image data, sharing resulting feature values, and reproducing the feature values based on provenance. As an application of the designed system, we report the quality evaluations of 319 image features extracted using ImageJ/Fiji, Python (scikit-image), CellProfiler and inhouse Java software packages with 43 duplicate features across the four packages. Using the normalized difference as metric, we identified 6 out of the 43 common features to differ over 1% in value and discuss the sources of these numerical differences.