Segmentation and visualization of the Shampula dragonfly eye glass bead CT images using a deep learning method | npj Heritage Science

Heritage Science volume 12, Article number: 381 (2024) Cite this article

694 Accesses

Metrics details

Micro-computed tomography (CT) of ancient Chinese glass dragonfly eye beads has enabled detailed exploration of their internal structures, contributing to our understanding of their manufacture. Segmentation of these CT images is essential but challenging due to variation in grayscale values and the presence of bubbles. This study introduces a U-Net-based model called EBV-SegNet, which enables efficient and accurate segmentation and visualization of these beads. We developed, trained, and tested the model using a dataset comprising four typical Shampula dragonfly eye beads, and the results demonstrated high-precision segmentation and precise delineation of the beads’ complex structures. These segmented data were further analyzed using the Visualization Toolkit for advanced volume rendering and reconstruction. Our application of EBV-SegNet to Shampula beads suggests the likelihood of two distinct manufacturing techniques, underscoring the potential of the model for enhancing the analysis of cultural artifacts using three-dimensional visualization and deep learning.

The Shampula Cemetery, discovered in the 1980s in Yutian, is a large public cemetery located on the southern edge of the Taklamakan Desert in the northern foothills of the Kunlun Mountains [1]. This region lies along the southern route of the Silk Road and once served as a critical hub for economic, trade, and cultural exchange. In total, 1369 glass beads were excavated from the cemetery [2], with a rich variety of colors, shapes, and decorative patterns. Dragonfly eye beads are one type of glass bead crafted by concentrically embedding multi-colored glass to create a multilayered structure that mimics the compound eye of dragonflies [1]. They originated in Egypt around the fourteenth century BCE, and in Western Asia were thought to have symbolized the eyes of deities [3]. They served as amulets against evil, and as the Silk Road flourished were disseminated from the West into Central Asia. In total, 21 dragonfly eye beads were identified from the 1369 glass beads unearthed from the Shampula cemetery.

Due to its capability for non-invasive examination of internal structures, computed tomography (CT) is used extensively in archaeology to study materials such as ceramics [4, 5], bronze [6, 7], beads [1, 8, 9] and others [10,11,12]. Indeed, CT technology has already been used to help clarify the complex process used to produce ancient dragonfly eye beads. In 2013, Yang et al. [13] used synchrotron radiation micro-CT and µ-XRF to analyze dragonfly eye bead artifacts from the Shenmingpu Chu tombs in Henan, including their materials and manufacturing techniques. In 2019, Cheng et al. [14] found that a bead from Shampula Cemetery was made using an embedding technique. However, those two studies only used CT images to make visual observations, and did not conduct further image processing. In 2021, Zhang et al. [2] manually segmented CT images of beads from the Shampula Cemetery and identified ‘single pupil’ and ‘double pupil’ manufacturing styles, which are crucial elements when considering the technological evolution of dragonfly eye bead production. Zhang et al. [2] spent 2 days segmenting each bead manually, a method that is very time-consuming and inefficient. However, their findings highlighted the importance of segmenting CT images of eye beads, along with the need for an automated segmentation technique to streamline the process and obtain accurate results—ideally a technique that could be extended to a large assemblage of artifacts.

At present, archaeologists primarily rely on various software tools for manually segmenting glass beads [1, 9, 14, 15], which is time-consuming and inefficient. Image segmentation and three-dimensional (3D) visualization reconstruction technologies can be used to facilitate CT image analysis, but segmenting archaeological CT images is challenging [16] due to image noise, severe artifacts, uneven grayscale values, and unclear material boundaries. Scholars working in the field of computer science initially used thresholding [17], region growing [18], and edge segmentation [19], but these methods typically need a clearly delineated boundary between the target region and background. Target extraction requires extensive manual intervention and these methods struggle with noise and blurring. Segmenting accuracy and efficiency has recently been improving significantly with the help of deep learning methods [20,21,22,23,24], which include fully convolutional network (FCNs) [22], U-Net [21], SegNet [25], DeepLab [26], and DeepLabV3 [27]. Large models have accelerated the use of artificial intelligence in various fields, with models such as Meta’s Segment Anything Model (SAM) [28] excelling in general segmentation tasks. However, specific applications like segmenting dragonfly eye beads or artifact crafts require tailored deep learning networks due to unique segmentation challenges, as detailed in the 'Research Objectives' section and partially illustrated in Fig. 1.

Challenges in the CT segmentation of dragonfly eye glass beads. a SC-1 bead; b S-2 bead; c SC-4 bead; d SC-5 bead. The questions highlighted at the top indicate four major challenges in segmenting the eye regions, which are marked with colored arrows in the figure. Each arrow color corresponds to a specific challenge as described in the legend

Segmented CT images can isolate areas of interest to researchers, but they are observed in traditional two-dimensional (2D) slices, which lack a 3D perspective. Some scholars have drawn from methods used in medical CT reconstruction [29], including surface and volume rendering, to engage in 3D reconstruction of segmented artifact CT images. By integrating changes in voxel grayscale values, opacity projection is applied to achieve 3D reconstruction [30]. This results in a 3D model of the artifact, but this model still lacks details of the internal structure visible in CT slices. Therefore, a new method is needed for artifact analysis: one that can integrate both 3D reconstruction and CT slice information.

To address these issues, we have developed a novel end-to-end approach named EBV-SegNet, which includes an improved segmentation method based on U-Net and a hierarchical reconstruction visualization technique using the Visualization Toolkit(VTK) [31]. Our approach enables the automatic segmentation and layered visualization of internal structure, allowing researchers to view the overall, local, and cross-sectional information in CT images from any angle, facilitating interactive analysis of the internal structure of artifacts. To the best of our knowledge, this is the first application of deep learning for the segmentation and visual reconstruction of Shampula dragonfly eye beads. Our approach is not limited to dragonfly eye glass beads; it will be a valuable reference for any researcher conducting CT analysis, by enhancing interactive analysis with automated and visual tools.

Segmenting and visualizing CT images to isolate structures of interest is crucial for researchers analyzing manufacturing techniques. Specifically, dragonfly eye bead CT images present issues such as discontinuities in gray levels at boundaries, uncertain boundary positions, and grayscale imbalances. These issues hinder the segmentation of the eye region from the rest of the bead structure, as shown in Fig. 1. Excessive noise, indistinct structural boundaries, and poor reconstruction quality are inherent challenges in CT acquisition and reconstruction. Addressing these issues at the segmentation stage remains essential, particularly when further optimization of the acquisition and reconstruction processes is limited. Previous studies [32,33,34] have demonstrated that CT segmentation can significantly improve reconstruction outcomes in similar noisy or complex imaging scenarios, making it a necessary step when further optimization of acquisition or reconstruction is not feasible. In the case of the dragonfly eye beads, the segmentation of the 'eye' region is crucial for isolating the structure of interest from the surrounding noisy and indistinct areas, which facilitates a more precise 3D reconstruction.

To address these challenges, this study developed a method that utilizes deep learning to automatically segment the eye and body parts of glass beads, accurately delineating the edges crafted through the inlay technique. We have also incorporated 3D reconstruction and visualization methods to provide archaeologists with an intuitive way to represent and examine the internal structures and manufacturing processes of eye beads. The research objectives of this paper are as follows: (I) to develop an efficient and precise deep learning approach for the automatic segmentation and visualization of eye beads, optimized for a limited annotated dataset; (II) to offer researchers an end-to-end model for one-click segmentation of the eye structure, facilitating analysis of the eye beads molding process; (III) to create a visualization technique for 3D reconstruction and virtual sectioning, enabling the observation of the CT sections from any angle and position.

We developed EBV-SegNet, a deep learning framework tailored for the automatic segmentation and visualization of CT images of dragonfly eye glass beads. Specifically, we employ advanced deep learning techniques, ResNeSt-U-Net, to achieve high segmentation accuracy, reducing errors associated with manual methods. EBV-SegNet combines 3D reconstruction and virtual slicing techniques to visualize segmented structures from multiple perspectives, addressing the constraints of traditional 2D CT imaging, which is limited to a single viewing angle. EBV-SegNet operates as a comprehensive end-to-end solution, processing CT datasets into 3D see-through models without manual intervention. Figure 2 details the EBV-SegNet workflow, which consists of four main stages: (a) preparation and labeling of dragonfly eye CT image data; (b) deployment of the ResNeSt-U-Net architecture, which is well-suited for capturing fine details in the complex geometric edges of dragonfly eye glass beads; (c) post-processing of the segmentation mask to ensure that only the “eye” structure is retained, optimizing the continuity and edges of the segmented output; (d) reconstruction of the segmented structures into a 3D model, allowing virtual slicing from any angle and enabling internal structures to be viewed from arbitrary positions and perspectives within the model.

EBV-SegNet workflow. The four main stages of the EBV-SegNet process: a preparing and labeling data from CT images of dragonfly eye beads, essential for training the model; b deployment of the ResNeSt-U-Net architecture and its application within the framework; c image post-processing to enhance the quality and accuracy of the segmentation; and d 3D reconstruction and virtual slicing, facilitating multi-perspective visualization of the segmented structures

Datasets play a pivotal role in deep learning models, providing essential data for training, validation, and testing, thus directly influencing the model’s capacity to learn, generalize, and perform accurately in applications. In this study, we selected four typical samples from the Shampula Cemetery, Xinjiang, labeled SC-1, S-2, SC-4, and SC-5; these were sourced from the Palace Museum. Each of these eye beads features a central perforation and decorative eye-shaped patterns on the outer surface. Despite their similar shapes, the beads differ only in the color of the eyes. The CT datasets were acquired using the X-ray imaging BL13W-Line Station at the Shanghai Light Source [14]. This imaging line station operates at a X-ray energy of 65 keV, with a sample-to-detector distance of 70 cm, an exposure time of 0.35 s, and an image voxel size of 9 × 9 × 9 μm3.

Each dragonfly eye bead was scanned separately for segmentation. The original CT data was acquired in DICOM format and subsequently converted into 2D PNG images to standardize the image format for deep learning training. After conversion, the resulting images had a resolution of 1217 × 1210 pixels. The number of CT images obtained for each of the four beads varied, with 1054, 874, 1272, and 943 images for SC-1, S-2, SC-4, and SC-5, respectively, for a total of 4143 images. This comprehensive dataset facilitates iterative validation through image labeling, providing ground truth data essential for accurate analysis. A professional archaeologist oversaw the labeling process, using the polygon annotation tool in LabelMe software to delineate the eye part, body part and background in each image, distinguished by different colors. Bubbles formed by the compression of glass at boundary were deliberately excluded from the annotations to ensure clarity. The precision of these annotations is critical, as slight variations in the contour points selected during the annotation process can influence the final segmentation results. These variations may lead to minor differences in the segmented boundaries, highlighting the importance of high annotation accuracy. After a meticulous review, a manual annotation dataset for the four eye beads was compiled, and the resulting JSON label files were exported. Figure 3 illustrates representative results of the labeling process, with eye part marked in red, body part in blue, and the background in black. To evaluate the performance of our model, we adopted a leave-one-out cross-validation approach using the dataset of four dragonfly eye glass beads. Specifically, we conducted four independent experiments. In each experiment, one bead was selected as the test set, while the remaining three beads were combined and then split into training and validation sets in an 8:2 ratio. This approach ensured that each bead was used as an independent test set once, allowing for a thorough evaluation of the model's ability to generalize to unseen samples.

Annotation of CT images of dragonfly eye glass beads. Four beads (SC-1, S-2, SC-4, and SC-5) with representative CT images, each showing the number of images analyzed. Annotations mark eye part in red, body part in blue, and backgrounds in black to demonstrate segmentation accuracy

The conventional U-Net architecture, originally proposed by Ronneberger et al. [21], has shown strong performance in biomedical image segmentation. However, its performance in the semantic segmentation is limited due to challenges in retaining spatial resolution within the encoder network and insufficient feature extraction capabilities [35, 36]. These limitations are particularly evident in the complex structures found in CT images of beads, leading to imprecise segmentation and the loss of fine details. Nonetheless, U-Net can still be effective for segmentation tasks involving small datasets, thanks to its simple architecture, which allows it to perform well even with limited training data. To enhance feature extraction, we utilized the joint attention mechanism of the ResNeSt module [37], which was introduced by Zhang et al. The integration of the ResNeSt module into the U-Net architecture enhances the model's ability to capture complex spatial features, especially in challenging segmentation tasks. It should be noted that the combination of ResNeSt and U-Net has been explored in prior works [38], and we leverage this approach to address the specific challenges presented by our dataset. Furthermore, we also incorporate a packet channel attention mechanism [39] into the convolution network to improve prediction accuracy. Figure 4 presents the workflow of the proposed ResNeSt-UNet network. The network architecture consists of two main phases: downsampling and upsampling. In the downsampling phase, the preprocessed image undergoes convolution, batch normalization, and the ReLU operation within the U-Net framework, followed by four enhanced downsampling operations using the SplitAttenConv module and feature extraction through the ResNeSt feature extraction module. The upsampling phase involves four conventional U-Net upsampling operations, culminating in the final output after the upsampling convolution operation.

The workflow of the ResNeSt-U-Net network [40]

In the enhanced SplitAttenConv module, the conventional convolution unit in the original U-Net downsampling module is replaced by a convolution module equipped with a channel attention mechanism. This SplitAttenConv module is structured into three subgroups: a convolution module, BatchNormalization layer, and ReLU layer. Diverging from the traditional downsampling approach that utilizes standard convolution, this module leverages the channel attention mechanism to more effectively sample context information from feature images, thereby significantly improving the accuracy of the semantic image segmentation.

To enhance the feature extraction capability of the proposed network, we incorporate a joint attention mechanism known as the ResNeSt module [37] into the encoder. This module integrates concatenated residual transformations, commonly used in deep neural networks such as ResNet [40], ResNeXt [41], and Squeeze-and-Excitation Netwoks(SE-Net) [39] to recalibrate and regroup channels adaptively. The goal is to improve segmentation performance significantly without the addition of extra parameters. The structure of the ResNeSt module is depicted in Fig. 5. The operational approach is as follows: the network initially processes 256 × 256 images, subdividing them into k cardinal groups via a branching structure. Within each cardinal group, slices are introduced and subjected to convolutions with 1 × 1 and 3 × 3 kernels. Subsequently, different features are assigned varying weights through the Split Attention mechanism. These features are then concatenated, and the resultant output is further concatenated with the output from the residual module.

Architecture of the ResNeSt module. A schematic illustrates of the modular structure of ResNeSt, showing the flow from input through multiple cardinal splits with convolution operations and Split Attention, to the concatenated output

Before training the model, data preprocessing steps were implemented on the original dragonfly eye beads images to enhance the generalizability of the model. These steps included image enhancements and the application of Gaussian blurring. Additionally, to mitigate the effects of outlier data, normalization parameters from ImageNet were used to standardize the training data based on their mean and standard deviation. The model was trained in a hardware environment equipped with an 11th Generation Intel(R) Core™ i7-11700 processor and an NVIDIA GeForce RTX 3090 GPU. The software environment consisted of Python 3.7, PyTorch 1.3.0, and CUDA 11.6. Table 1 lists the specific hyperparameters used for network training, including an early stopping mechanism where training stopped if the validation loss did not improve for 20 consecutive epochs. To assess the performance of the proposed semantic segmentation model on dragonfly eye images, the conventional U-Net network was employed as a benchmark for comparison. The ResNeSt-U-Net architecture consisted of 34,447,603 (34 M) trainable parameters, while the U-Net architecture consisted of 31,037,763 (31 M) trainable parameters. The input to the architecture was a single PNG image slice paired with its corresponding mask.

The output of the ResNeSt-U-Net model comprises a set of masks for each image. In these masks, the dragonfly eye glass bead body is depicted in blue, the eye in red, and the background in black. To segment the original image using these masks, a mapping operation is performed. As shown in Fig. 6, an all-zero binary mask of the same dimensions as the sub-image is first created. In this mask, only the pixels corresponding to the red color values of (255, 0, 0) are set to 1, with all others set to 0. This binary mask is then multiplied by the pixel values of the original image, isolating the eye structure in the resulting segmented image.

Segmentation process using ResNeSt-U-Net. (I) Original CT image with predicted segmentation mask: body in blue, eye in red, and background in black. (II) Binary mask creation, isolating the eye in white. (III) Final segmented images of the eye structure derived from applying the binary mask

The 3D reconstruction and virtual slicing techniques offer archaeologists effective, non-invasive tools for examining the internal structure of cultural relics [42]. These tools permit observations of segmented structures and CT slices from any perspective, enabling comprehensive exploration of cultural artifacts interactively. After the segmentation of the sequence of PNG images, we perform a 3D visualization by assembling the 2D image slices to effectively demonstrate the spatial relationships of the segmented structures. For these tasks, the Visualization Tool Kit (VTK) was selected due to its robustness and widespread recognition in the visualization community [43], making it particularly suitable for demonstrating archaeological crafts. The chosen reconstruction techniques include a surface rendering algorithm, MarchingCubes, and a volume rendering algorithm, RayCastMapper. While surface rendering displays only the outermost layer of an object, volume rendering allows for visualization of intricate internal details, enhancing the accuracy of representing the three-dimensional structure and physical attributes of the eye beads.

The visualization of reconstructed dragonfly eye glass beads facilitates interactive exploration, including operations such as rotation, zooming, and virtual slicing to unveil hidden details. We utilized the vtkRenderWindowInteractor class in Python 3.9 to enable interactive functionalities including scaling, rotating, and moving the model. This class also enhances the 3D effects by supporting additional rendering of light and shadows. For virtual slicing operations, we used the vtkImageReslice and vtkImagePlaneWidget classes. While vtkImageReslice provides precise control but may be slower and require adjustments for real-time interaction; in contrast, vtkImagePlaneWidget offers greater flexibility, allowing users to manipulate slices interactively using a mouse. Consequently, our research primarily utilizes the vtkImagePlaneWidget class for rapid access to volume data, channeling it through a texture mapping pipeline for sectional visualization.

The performance of the proposed model has been quantitatively evaluated using several metrics: accuracy, sensitivity, specificity, and Intersection-over-Union (IoU). These metrics are crucial for assessing the effectiveness of the segmentation process and are defined as follows:

Accuracy is the proportion of pixels correctly classified by the model.

Sensitivity (also known as Recall) measures the ratio of correctly identified target pixels to all actual target pixels.

Specificity assesses the ratio of correctly identified background pixels to all actual background pixels.

IoU calculates the ratio between the intersection and union of the predicted and actual labeled areas.

The results of these metrics can be defined as follows: True Positive (TP) denotes pixels correctly identified as the object of interest; True Negative (TN) refers to pixels correctly identified as background; False Positive (FP) pertains to pixels incorrectly classified as the object; and False Negative (FN) signifies pixels incorrectly categorized as background. Table 2 summarizes the performance metrics, including the accuracy, sensitivity, specificity, and IoU of the segmented results using the proposed ResNeSt-U-Net model and conventional U-Net model [21]. The results demonstrate that ResNeSt-U-Net substantially outperforms the conventional U-Net across all evaluation metrics. Specifically, the average segmentation accuracy of ResNeSt-U-Net reaches 96.22, the sensitivity is 93.44, the specificity is 97.34 and the IoU score is 90.35.

To visually demonstrate the enhancements achieved by our proposed ResNeSt-U-Net model over the conventional U-Net, we conducted a comparative analysis on four samples of dragonfly eye glass beads (SC-1, S-2, SC-4, SC-5). Figure 7 illustrates the segmentation results of both models, along with the original images and ground truth. The ResNeSt-U-Net model had superior boundary precision than the conventional U-Net. For instance, in samples S-2 and SC-4, the U-Net model deviated notably from the ground truth, particularly around complex boundary regions of the beads. In contrast, the ResNeSt-U-Net model closely followed the actual contours, reflecting its robustness in terms of handling intricate object boundaries. Furthermore, the U-Net model struggled with the presence of bubbles and other part, often misclassifying these as part of the bead. This was evident in samples SC-1 and SC-5, where the U-Net results contained several misclassified regions. In contrast, the ResNeSt-U-Net model effectively minimized these errors, resulting in cleaner, more accurate segmentation outputs that are almost identical to the ground truth.

Segmentation comparison of dragonfly eye glass beads. From top to bottom: Original CT images, ground truth, U-Net results, and ResNeSt-U-Net results. The segmentation masks color-code the eye part(red), body part(blue), and background (black), demonstrating the superior accuracy of the ResNeSt-U-Net model

Under the computer configuration described in Sect. “Hyperparameter optimization and model training”, the ResNeSt-U-Net model takes 0.01 s to segment a single image. The total segmentation time for an entire bead depends on the number of CT images it contains. For instance, SC-1 in this experiment contains 1054 images, resulting in a total segmentation time of approximately 10 s. The time required for the automatic segmentation of each of the four beads is shown in Table 3. This represents a significant improvement in research efficiency compared to the two days of manual work spent by Zhang et al.[1] to segment a single bead.

Following the segmentation of the dragonfly eye glass beads, clear 3D reconstruction of the internal structures is crucial for researchers. Figure 8 provides a comprehensive visual representation of each step from original imaging to detailed 3D reconstruction, as outlined in Sect. “3D reconstruction and virtual slicing”. The initial step involves the extraction of segmented eye bead images. These segmented images effectively isolate the eye beads, serving as the basis for subsequent reconstructions. Then, the eye bead images are subjected to two distinct 3D reconstruction techniques: MarchingCubes for surface rendering and RayCastMapper for volume rendering. As shown in Fig. 8, surface rendering captures only the external outlines of the beads, lacking internal detail, while volume rendering includes comprehensive internal features, preserving the intricate details and overall structure of the beads. These reconstructions demonstrate the qualitative superiority of the volume rendering technique in capturing the full complexity of the artifacts.

Stages of image processing and 3D reconstruction for dragonfly eye beads. From top to bottom: original photographs of the beads (SC-1, S-2, SC-4, and SC-5); the corresponding CT images; segmented images of eye regions; 3D surface reconstructions using MarchingCubes; and 3D volume reconstructions using RayCastMapper. This sequence illustrates the progression from raw images to advanced 3D models, highlighting the differences in detail capture between the two reconstruction techniques

To compare the surface rendering and volume rendering techniques used in the 3D reconstruction of dragonfly eye beads, Table 4 lists key differences in the algorithm features, reconstruction quality, run time, major deficiencies, and usage scenarios for each method, and summarizes the characteristics and suitability of each rendering technique. Surface rendering is notable for its speed and clarity, but lacks the ability to capture internal details and may produce blurred edges due to insufficient edge extraction. Conversely, volume rendering is slower, but excels at revealing the hierarchical internal structure of objects and is better suited for applications requiring detailed internal visualization.

We also investigated how segmentation affected the quality of dragonfly eye bead reconstruction from CT images. Direct reconstructions from original, unsegmented CT images typically results in less-than-ideal visualizations, as shown in Fig. 9a. Specifically, the reconstructions of sample SC-1 without prior segmentation are marred by numerous impurities and noise artifacts. These noise artifacts arise from non-uniform gray values within the CT images, which challenge the removal of extraneous material and obscure crucial details of the eye beads. In contrast, segmenting the CT images to isolate only the eye regions before reconstruction significantly refines the visual outcomes. As shown in Fig. 9b, reconstructions performed on these segmented images reveal the fine details of the beads, including edge shape, density, porosity, and relative positioning. This approach markedly enhances the structural clarity and allows more accurate assessment of the eye mosaic techniques used in the manufacture of the dragonfly eye beads. This comparison highlights the essential role of segmentation in achieving detailed accurate 3D reconstructions, which are vital for thorough artifact analysis.

Effects of segmentation on the quality of dragonfly eye bead reconstruction. a Reconstructions from unsegmented CT images of SC-1, displaying prominent impurities and noise that detract from the visualization quality. b Reconstructions from segmented CT images of samples SC-1, S-2, SC-4, and SC-5, illustrating improved clarity and detail that enhance the understanding of the beads' structural and decorative features

Figure 10 presents the application of virtual slicing technology to the four glass dragonfly eye beads, enabling detailed internal examinations through interactive 3D reconstructions. Each panel presents a reconstructed volume of a bead, along with a display of its virtual slices, which were interactively manipulated using the VTKImagePlaneWidget tool, which allows dynamic interactions such as zooming, translating, and rotating the reconstructed volume using mouse inputs. As users select the slicing direction and entry point, the tool automatically establishes a central axis. A virtual section is then executed along this axis, providing a real-time display of cross-sectional images during the slicing process. The ability to slice the object interactively and view it from various angles offers a significant advantage when studying the intricate internal compositions of dragonfly eye beads.

Interactive virtual slicing of dragonfly eye glass beads. The effects of virtual slicing on four samples (SC-1, S-2, SC-4, and SC-5) showcase the 3D reconstructed volumes (left) and their corresponding slices (right). The slices, facilitated by VTKImagePlaneWidget, allow interactive exploration of the internal structures of the beads

This study is a significant advance in the analysis of dragonfly eye beads, enhancing segmentation in terms of both efficiency and accuracy using an automated, end-to-end process that precisely delineates eye edges. Our novel segmentation method has confirmed previous findings regarding the manufacturing of these artifacts: Fig. 11 demonstrate how the EBV-SegNet segmentation method reveals the complex structure of the eye beads, which are made from glass materials of varying densities and colors arranged in concentric rings. The figure reveals that beads SC-1 and S-2 have a single-pupil structure, whereas SC-4 and SC-5 have a double-pupil structure. It should be noted that these observations were originally made by Zhang et al. [1] through manual segmentation, and our automated approach successfully replicates these findings, thereby validating its effectiveness. This automated segmentation technique offers precision and accuracy comparable to that of the time-intensive manual segmentation methods used by Zhang et al. [1], as depicted in Fig. 11, but with significantly enhanced efficiency.

Comparison of the reconstructed eye part between automatic segmentation and manual segmentation by Zhang et al.[1]

The production of dragonfly eye beads was prolific, necessitating the use of automatic segmentation methods for efficient analysis of large quantities. Manual segmentation of each bead would be impractically time-consuming and labor-intensive. Automatic segmentation accurately delineates the boundaries between the eye and body parts of the beads and also swiftly clarifies the spatial relationships among multiple eyes on the same bead. This automation significantly reduces the time that would otherwise be spent on manual segmentation, thereby enhancing the efficiency of archaeological studies into the manufacturing of these beads. Consequently, automatic segmentation provides critical data for tracing the origins of the eye beads and identifying their production techniques, substantially advancing archaeological research.

This paper introduces EBV-SegNet, a novel end-to-end approach for studying dragonfly eye beads, incorporating improved segmentation and reconstruction visualization techniques. It clearly outlines the edges formed by the inlay technique, and offers archaeologists a clear view for studying the internal structures and manufacturing processes of eye beads. In terms of segmentation, an enhanced deep learning-based ResNeSt-U-Net architecture was used to segment the eye region in CT images, achieving a segmentation accuracy of 96.22%. This overcomes the limitations of manual segmentation, which often results in incomplete selection of areas of interest. In terms of visualization, we compared the effects of surface and volume rendering algorithms, and found that high-precision reconstruction of internal structures was achieved using the RayCastMapper algorithm. The VTKImagePlaneWidget tool was used to extract CT slices at any angle and position, offering researchers perspectives from multiple views. Our method significantly enables researchers to process and interpret data from CT images of artifacts in any working environment, reducing operational costs and enhancing the utility and presentation of cultural relics. The application of this approach extends far beyond the specific beads we studied: it will facilitate the mass analysis of artifact production processes and improve knowledge about cultural relics.

No datasets were generated or analysed during the current study.

Zhang X, Lei Y, Cheng Q, Zhou G. Application of computed tomography in the analysis of the manufacture of eye beads technique. Microchem J. 2020;156:104798.

Article CAS Google Scholar

Zhang XY, Guo JL, Cheng Q, Wang B, Lei Y. Structural and craft research on eyeballs unearthed from the Shanpula Cemetery (山普拉墓地出土眼珠的结构工艺研究). CT Theory Appl (CT理论与应用研究). 2021;30:161–9 (in Chinese).

CAS Google Scholar

Li J, Sun F, Zhang Y, Ha W, Yan H, Zhai C. Scientific analysis of two compound eye beads unearthed in Hejia Village, Zhouling. Herit Sci. 2024;12:127.

Article CAS Google Scholar

Takenouchi K, Yamahana K. Fine pottery shaping techniques in Predynastic Egypt: a pilot study on non-destructive analysis using an X-Ray CT scanning system. J Archaeol Sci Rep. 2021;37:102989.

Google Scholar

Kozatsas J, Kotsakis K, Sagris D, David K. Inside out: assessing pottery forming techniques with micro-CT scanning—an example from Middle Neolithic Thessaly. J Archaeol Sci. 2018;100:102–19.

Article CAS Google Scholar

Wang Z, Xi X, Li L, Zhang Z, Han Y, Wang X, et al. Tracking the progression of the simulated bronze disease—a laboratory X-ray microtomography study. Molecules. 2023;28:4933.

Article PubMed PubMed Central CAS Google Scholar

Sun Z, Gao Y, He X, Liu S, Chen K, Chen J, Gong X. Material characterization uncovers sophisticated mould-making techniques of the middle-Shang period (14th-13th BCE). J Archaeol Sci. 2023;160:105870.

Article Google Scholar

Chen D, Zeng Q, Li C, Qiao B, Luo W. Exotic treasures or local innovation: gold and silver beads of Han dynasty excavated from Nanyang City, Henan Province, central China. J Archaeol Sci. 2024;161:105903.

Article CAS Google Scholar

Nykonenko D, Yatsuk O, Guidorzi L, Lo Giudice A, Tansella F, Cesareo LP, et al. Glass beads from a Scythian grave on the island of Khortytsia (Zaporizhzhia, Ukraine): insights into bead making through 3D imaging. Herit Sci. 2023;11:238.

Article Google Scholar

Re A, Corsi J, Demmelbauer M, Martini M, Mila G, Ricci C. X-ray tomography of a soil block: a useful tool for the restoration of archaeological finds. Herit Sci. 2015;3:4.

Article Google Scholar

Karl S, Kazimierski KS, Hauzenberger CA. An interdisciplinary approach to studying archaeological vase paintings using computed tomography combined with mineralogical and geochemical methods: a Corinthian alabastron by the Erlenmeyer Painter revisited. J Cult Herit. 2018;31:63–71.

Article Google Scholar

Reinhardt J, Tischer M, Schmid S, Kollofrath J, Burger R, Jatzlau P, et al. X-ray-based examination of artworks by Cy Twombly: art technology and condition of the ‘Original Sculptures.’ Herit Sci. 2023;11:237.

Article Google Scholar

Yang Y, Wang L, Wei S, Song G, Kenoyer JM, Xiao T, et al. Nondestructive analysis of dragonfly eye beads from the warring states period, excavated from a Chu tomb at the Shenmingpu site, Henan Province, China. Microsc Microanal. 2013;19:335–43.

Article PubMed CAS Google Scholar

Cheng Q, Zhang X, Guo J, Wang B, Lei Y, Zhou G, et al. Application of computed tomography in the analysis of glass beads unearthed in Shanpula cemetery (Khotan), Xinjiang Uyghur Autonomous Region. Archaeol Anthropol Sci. 2019;11:937–45.

Article Google Scholar

Ngan-Tillard DJM, Huisman DJ, Corbella F, Van Nass A. Over the rainbow? Micro-CT scanning to non-destructively study Roman and early medieval glass bead manufacture. J Archaeol Sci. 2018;98:7–21.

Article CAS Google Scholar

Kiss MB, Bossema FG, van Laar PJC, Meijer S, Lucka F, van Leeuwen T, et al. Beam filtration for object-tailored X-ray CT of multi-material cultural heritage objects. Herit Sci. 2023;11:130.

Article Google Scholar

Siri SK, Kumar SP, Latte MV. Threshold-based new segmentation model to separate the liver from CT scan images. IETE J Res. 2022;68:4468–75.

Article Google Scholar

Zhou Z, Wan H, Zhang H, Chen X, Wang X, Lili S, et al. Segmentation of spontaneous intracerebral hemorrhage on CT With a region growing method based on watershed preprocessing. Front Neurol. 2022. https://doi.org/10.3389/fneur.2022.865023.

Article PubMed PubMed Central Google Scholar

Wang K, Zhao Y, Dou Y, Wen D, Gao Z. COVID Edge-Net: automated COVID-19 lung lesion edge detection in chest CT images. In: Dong Y, Kourtellis N, Hammer B, Lozano JA, editors. Machine learning and knowledge discovery in databases applied data science track. Cham: Springer International Publishing; 2021. p. 287–301.

Chapter Google Scholar

Rekik I, Allassonnière S, Carpenter TK, Wardlaw JM. Medical image analysis methods in MR/CT-imaged acute-subacute ischemic stroke lesion: segmentation, prediction and insights into dynamic evolution simulation models—a critical appraisal. NeuroImage Clin. 2012;1:164–78.

Article PubMed PubMed Central Google Scholar

Ronneberger O, Fischer P, Brox T. U-Net: convolutional networks for biomedical image segmentation. In: Navab N, Hornegger J, Wells WM, Frangi AF, editors. Medical image computing and computer-assisted intervention—MICCAI 2015. Cham: Springer International Publishing; 2015. p. 234–41.

Google Scholar

Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation. 2015. pp. 3431–40. https://openaccess.thecvf.com/content_cvpr_2015/html/Long_Fully_Convolutional_Networks_2015_CVPR_paper.html. Accessed 24 Sept 2024.

Zhu XX, Tuia D, Mou L, Xia G-S, Zhang L, Xu F, et al. Deep learning in remote sensing: a comprehensive review and list of resources. IEEE Geosci Remote Sens Mag. 2017;5:8–36.

Article Google Scholar

Liao L, Sun Z, Liu S, Ma S, Chen K, Liu Y, et al. Applying a mask R-CNN machine learning algorithm for segmenting electron microscope images of ceramic bronze-casting moulds. J Archaeol Sci. 2024;170:106049.

Article CAS Google Scholar

SegNet: a deep convolutional encoder-decoder architecture for image segmentation. https://ieeexplore.ieee.org/abstract/document/7803544/. Accessed 7 Aug 2023.

Chen L-C, Papandreou G, Kokkinos I, Murphy K, Yuille AL. Semantic image segmentation with deep convolutional nets and fully connected CRFs. arXiv. 2016. http://arxiv.org/abs/1412.7062. Accessed 7 Aug 2023.

Chen L-C, Papandreou G, Kokkinos I, Murphy K, Yuille AL. DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Trans Pattern Anal Mach Intell. 2018;40:834–48.

Article PubMed Google Scholar

Kirillov A, Mintun E, Ravi N, Mao H, Rolland C, Gustafson L, et al. Segment anything. 2023. pp. 4015–26. https://openaccess.thecvf.com/content/ICCV2023/html/Kirillov_Segment_Anything_ICCV_2023_paper.html. Accessed 7 June 2024.

Khan U, Yasin A, Abid M, Shafi I, Khan SA. A methodological review of 3D reconstruction techniques in tomographic imaging. J Med Syst. 2018;42:190.

Article PubMed Google Scholar

Jakhongir N, Abdusalomov A, Whangbo TK. 3D volume reconstruction from MRI slices based on VTK. In: 2021 International Conference on Information and Communication Technology Convergence (ICTC). 2021. pp. 689–92. Available from: https://ieeexplore.ieee.org/abstract/document/9621022. Accessed 8 June 2024.

VTK-The Visualization Toolkit. https://vtk.org/. Accessed 24 Sept 2024.

Then-Obłuska J, Dussubieux L. Teardrops at the Lake: chemistry of new kingdom to Makuria glass beads and pendants between the first and second Nile cataracts. Afr Archaeol Rev. 2023;40:295–315.

Article Google Scholar

Szczykutowicz TP, Toia GV, Dhanantwari A, Nett B. A review of deep learning CT reconstruction: concepts, limitations, and promise in clinical practice. Curr Radiol Rep. 2022;10:101–15.

Article Google Scholar

Hesamian MH, Jia W, He X, Kennedy P. Deep learning techniques for medical image segmentation: achievements and challenges. J Digit Imaging. 2019;32:582–96.

Article PubMed PubMed Central Google Scholar

Lin T-Y, Dollar P, Girshick R, He K, Hariharan B, Belongie S. Feature pyramid networks for object detection. 2017. pp. 2117–25. https://openaccess.thecvf.com/content_cvpr_2017/html/Lin_Feature_Pyramid_Networks_CVPR_2017_paper.html. Accessed 12 Oct 2024.

Oktay O, Schlemper J, Folgoc LL, Lee M, Heinrich M, Misawa K, et al. Attention U-Net: learning where to look for the pancreas. arXiv. 2018. http://arxiv.org/abs/1804.03999. Accessed 12 Oct 2024.

Zhang H, Wu C, Zhang Z, Zhu Y, Lin H, Zhang Z, et al. ResNeSt: split-attention networks. 2022. pp. 2736–46. https://openaccess.thecvf.com/content/CVPR2022W/ECV/html/Zhang_ResNeSt_Split-Attention_Networks_CVPRW_2022_paper.html. Accessed 4 Oct 2023.

Diakogiannis FI, Waldner F, Caccetta P, Wu C. ResUNet-a: a deep learning framework for semantic segmentation of remotely sensed data. ISPRS J Photogramm Remote Sens. 2020;162:94–114.

Article Google Scholar

Hu J, Shen L, Sun G. Squeeze-and-excitation networks. 2018. pp. 7132–41. https://openaccess.thecvf.com/content_cvpr_2018/html/Hu_Squeeze-and-Excitation_Networks_CVPR_2018_paper.html. Accessed 4 Oct 2023.

He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. 2016. pp. 770–8. https://openaccess.thecvf.com/content_cvpr_2016/html/He_Deep_Residual_Learning_CVPR_2016_paper.html. Accessed 29 Sept 2024.

Zhou T, Zhao Y, Wu J. ResNeXt and Res2Net structures for speaker verification. In: 2021 IEEE Spoken Language Technology Workshop (SLT). 2021. pp. 301–7. https://ieeexplore.ieee.org/abstract/document/9383531. Accessed 4 Oct 2023.

Marić J, Bašić Ž, Jerković I, Mihanović F, Anđelinović Š, Kružić I. Facial reconstruction of mummified remains of Christian Saint-Nicolosa Bursa. J Cult Herit. 2020;42:249–54.

Article Google Scholar

Bai K, Li Z, Li Y, Wang L, Zhang Y. P-2.16: research on three-dimensional reconstruction based on VTK. SID Symp Digest Tech Pap. 2023;54:537–40.

Article Google Scholar

Download references

Not applicable.

This research was supported by the National Key Research and Development Program (2023YFF0906600), National Natural Science Foundation of China Project (U2032130) and the second-phase opening project of the Palace Museum (Research on Multiple Information Management and Visualization for Cultural Relics Protection), which is funded by the Forbidden City Cultural Heritage Conservation Foundation and the Longfor-Forbidden City Cultural Heritage Foundation.

Beijing Engineering Research Center of Mixed Reality and Advanced Display, School of Optics and Photonics, Beijing Institute of Technology, Beijing, 100081, China

Lingyu Liao, Shining Ma, Yue Liu, Yongtian Wang & Weitao Song

STARC, The Cyprus Institute, 20 Konstantinou Kavaf Street, Aglantzia, 2121, Nicosia, Cyprus

Qian Cheng

China National Centre for Archaeology, Heping Street 21, Chaoyang District, Beijing, 100013, China

Qian Cheng

Department of Conservation Standards, The Palace Museum, Beijing, 100009, China

Xueyan Zhang & Liang Qu

Institute for Cultural Heritage and History of Science & Technology, University of Science and Technology, Beijing, China

Siran Liu & Kunlong Chen

Zhengzhou Academy Of Intelligent Technology, Beijing Institute of Technology, 450000, Beijing, China

Lingyu Liao & Weitao Song

You can also search for this author in PubMed Google Scholar

You can also search for this author in PubMed Google Scholar

You can also search for this author in PubMed Google Scholar

You can also search for this author in PubMed Google Scholar

You can also search for this author in PubMed Google Scholar

You can also search for this author in PubMed Google Scholar

You can also search for this author in PubMed Google Scholar

You can also search for this author in PubMed Google Scholar

You can also search for this author in PubMed Google Scholar

You can also search for this author in PubMed Google Scholar

LL wrote the main manuscript text and implemented the deep learning algorithms. QC and XZ processed the CT data. LQ, SL, and SM provided profound insights into the primary methods. KC, YL, and YW prepared Figs. 1, 2, 3. WS prepared Figs. 4, 5, 6, 7, 8, 9, 10, 11. All authors reviewed the manuscript and declared that there is no conflict of interest.

Correspondence to Weitao Song.

The authors declare no competing interests.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

Liao, L., Cheng, Q., Zhang, X. et al. Segmentation and visualization of the Shampula dragonfly eye glass bead CT images using a deep learning method. Herit Sci 12, 381 (2024). https://doi.org/10.1186/s40494-024-01505-w

Download citation

Received: 06 August 2024

Accepted: 26 October 2024

Published: 04 November 2024

DOI: https://doi.org/10.1186/s40494-024-01505-w

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative