3DSEM: a 3D Microscopy Dataset

 

 

1. Background:

 

The  scanning electron microscope (SEM) as 2D imaging instrument has been widely used in biology, mechanical, and materials sciences to determine the surface attributes (e.g., compositions or geometries) of microscopic specimens. A SEM offers an excellent capability to overcome the limitation of human eyes by achieving increased magnification, contrast, and resolution greater than 1 nanometer. However, SEM micrographs still remain two-dimensional (2D). Having truly three-dimensional (3D) shapes from SEM micrographs would provide anatomic surfaces allowing for quantitative measurements and informative visualization of the objects being investigated. In biology, for example, 3D SEM reconstructions would enable researchers to investigate surface characteristics and recognize roughness, flatness, and waviness of a biological structure. There are also various applications in material and mechanical engineering in which 3D representations of material properties would allow us to accurately measure a fractal dimension and surface roughness and design a micro article which needs to fit into a tiny appliance.

3D SEM surface reconstruction employs several computational technologies, such as multi-view geometry, computer vision, optimization strategies, and machine learning to tackle the inverse problem going from 2D to 3D. In this contribution, an attempt is made to provide a 3D microscopy dataset along with the underlying algorithms [1-5] publicly and freely available for the research community.

Keywords: 3D Microscopy Dataset, 3D Microscopy Vision, 3D SEM Surface Reconstruction, 3D Surface Reconstruction from SEM images, Scanning Electron Microscope, SEM, Three Dimensional Reconstruction, 3D Surface Reconstruction, Surface Reconstruction, SEM Applications, Point Cloud Reconstruction, 3D Surface Modeling.

 

2. Highlights:

 

  • Discovering 3D surface structure from SEM images would provide anatomic surfaces and allows informative visualization of the objects being investigated.

  • Many research and educational questions truly require knowledge and information about 3D microscopic structures. The present dataset along with the algorithms would be helpful in this way.

  • The current dataset which includes 2D SEM images and 3D surface models, and the underlying methodology may serve as a guide for 3D SEM surface reconstruction, leading to reproducible research in 3D microscopy vision.

  • The present work is expected to highlight the important roles and applications of 3D microscopy vision, particularly 3D surface reconstruction from SEM images, and open the doors for several interesting directions to advance the level of the research area.

 

3. Dataset:

 

The 3D Microscopy Dataset which is provided here includes both 2D images and 3D reconstructed surfaces of real microscopic samples as follows. Datasets (1), (2), (3), and (4) were generated using an optimized multi-view 3D SEM surface reconstruction algorithm [4, 5]. Datasets (5) and (6) were made using an optimized, adaptive, and intelligent 3D SEM surface reconstruction [1, 2]. The 3DSEM is an ongoing research project being developed at University of Wisconsin-Milwaukee. We will continuously add more and more data here. At present, the dataset includes:

(1) tapetal cell of Arabidopsis thaliana (.zip, 11.3 MB): This dataset contains five 2D images from a biological sample called “tapetal cell of Arabidopsis thaliana” and its 3D surface model (.off format). The set of 2D images were obtained by tilting the SEM specimen stage 9 degrees from one to the next in the image sequence.

(2) pollen grain from Brassica rapa (.zip, 620 KB): This dataset contains four 2D images from a biological sample called “pollen grain from Brassica rapa” and its 3D point cloud (.ply format) which could be easily converted to a surface model by MeshLab. The set of 2D images were obtained by tilting the specimen stage 3 degrees from one to the next in the image sequence.

(3) TEM copper grid (.zip, 18.9 MB): This dataset contains five 2D images from a material object called “TEM copper grid” and a part of its 3D surface model (.off format). The set of 2D images were obtained by tilting the SEM specimen stage 7 degrees from one to the next in the image sequence. We will provide more samples in near future.

(4) diatom frustule (.zip, 10.4 MB): This dataset contains three 2D images from a “diatom frustule” and its 3D point cloud (.off format) which could be easily converted to a surface model by MeshLab. The set of 2D images were obtained by tilting the specimen stage 15 degrees from one to the next in the image sequence.

(5) Hexagon TEM copper grid (.zip, 19.6 MB): This dataset contains five 2D images from a Hexagon TEM copper grid sample and its 3D surface model (.off format). The set of 2D images were obtained by tilting the specimen stage 10 degrees from one to the next in the image sequence.

(6) TEM copper grid bar (.zip, 856 KB): This dataset contains five 2D images from a TEM copper grid bar sample and its 3D surface model (.off format). The set of 2D images were obtained by tilting the specimen stage 11 degrees from one to the next in the image sequence.

The models and the underlying technologies/algorithms are fully explained in the following papers and the PhD thesis:

3D SEM Surface Reconstruction: An Optimized, Adaptive, and Intelligent Approach” [1].

3DSEM++: adaptive and intelligent 3D SEM surface reconstruction” [2].

Recent Advances in 3D SEM Surface Reconstruction” [4].

The raw image data including 2D SEM images (SEM micrographs) are provided by:

Electron Microscope Laboratory in Biological Sciences Department at University of Wisconsin-Milwaukee, USA.

The 3D surface models including 3D point clouds (.ply format) and 3D surfaces (.off format) are provided by:

Biomedical Modeling and Visualization Laboratory in Computer Science Department at University of Wisconsin-Milwaukee, USA.

 

4. Terms of Usage:

 

The dataset is freely available for any academic, educational, and research purposes. The terms of usage include:

1) You agree to use the data only for academic education or academic research purposes.

2) You agree to cite all of the four following papers and the PhD thesis, if the data is used for published research.

3) You agree to not distribute the data.

4) You agree that the data may not be modified or used for non-academic purposes without prior approval.

 

5. Citing the Dataset:

 

@article{ref01, title={3D SEM Surface Reconstruction: An Optimized, Adaptive, and Intelligent Approach}, author={Pahlavan Tafti, Ahmad}, year={2016}}

@article{ref02, title={3DSEM++: Adaptive and intelligent 3D SEM surface reconstruction}, author={Tafti, Ahmad P and Holz, Jessica and Baghaie, Ahmadreza and Owen, Heather and He, Max M and Yu, Zeyun}, journal={Micron}, volume={87}, pages={33--45}, year={2016}, publisher={Elsevier}}

@article{ref03, title={3DSEM: A 3D Microscopy Dataset}, author={Tafti, Ahmad P and Kirkpatrick, Andrew B and Holz, Jessica D and Owen, Heather A and Yu, Zeyun}, journal={Data in Brief}, year={2015}, publisher={Elsevier}}

@article{ref04, author = {Tafti, Ahmad P and Kirkpatrick, Andrew B and Alavi, Zahrasadat and Owen, Heather A and Yu, Zeyun}, title = {Recent Advances in 3D SEM Surface Reconstruction}, journal = {Micron}, Volume = 78, pages= {54--66}, year = {2015}, publisher={Elsevier}}

@incollection{ref05, author = {Tafti, A Pahlavan and Kirkpatrick, AB and Owen, HA and Yu, Z}, title = {3D Microscopy Vision Using Multiple View Geometry and Differential Evolutionary Approaches}, booktitle={Advances in Visual Computing}, pages={141--152}, year={2014}, publisher={Springer}}

6. Collaborators:

 

1) Ahmad P. Tafti, Ph.D., Associate Research Scientist, Biomedical Informatics Research Center, Marshfield Clinic Research Institute, USA.

2) Mehdi Assefi, Ph.D. Student, Department of Computer Science, University of Georgia, USA.

3) Zahrasadat Alavi, Ph.D., Assistant Professor, Electrical and Computer Engineering, CSU, Chico, USA.

4) Jessica D. Holz, M.S. Student, Department of Biological Sciences, UW-Milwaukee, USA.

5) Ahmadreza Baghaie, Ph.D., Post Doctoral Research Assistant, Purdue University, USA.

6) Emad Omrani, Ph.D. Student, Department of Materials Sciences, UW-Milwaukee, USA.

7) Afsaneh Dorri Moghadam, Ph.D., College of Engineering and Applied Science, UW-Milwaukee, USA.

8) Mojtaba Fathi, Ph.D. Student, College of Engineering and Applied Science, UW-Milwaukee, USA.

9) Andrew B. Kirkpatrick, Ph.D. Student, Department of Molecular Genetics, Ohio State University, USA.

10) Heather A. Owen, Ph.D., Associate Scientist, Department of Biological Sciences, UW-Milwaukee, USA.

11) Zeyun Yu, Ph.D., Associate Professor, Department of Computer Science, UW-Milwaukee, USA.

 

8. Acknowledgment:

 

We would like to acknowledge several people who without whom, the 3DSEM project was not possible. We would like to express our deepest appreciation to the following people for their insightful thoughts, motivation, and suggestions:

Prof. Ichiro Suzuki
Prof. Ethan Munson

 

9. Publications and References:

 

[1] Pahlavan Tafti, A., 2016. 3D SEM Surface Reconstruction: An Optimized, Adaptive, and Intelligent Approach. [UWM Digital Commons]

[2] Tafti, A.P., Holz, J.D., Baghaie, A., Owen, H.A., He, M.M. and Yu, Z., 2016. 3DSEM++: Adaptive and intelligent 3D SEM surface reconstruction. Micron, 87, pp.33-45. [ScienceDirect]

[3] Omrani, E., Tafti, A.P., Fathi, M.F., Moghadam, A.D., Rohatgi, P., D'Souza, R.M. and Yu, Z., 2016. Tribological study in microscale using 3D SEM surface reconstruction. Tribology International, 103, pp.309-315. [ScienceDirect]

[4] Tafti, A.P., Kirkpatrick, A.B., Holz, J.D., Owen, H.A. and Yu, Z., 2016. 3DSEM: A 3D microscopy dataset. Data in Brief, 6, pp.112-116. [ScienceDirect]

[5] Tafti, A.P., Kirkpatrick, A.B., Alavi, Z., Owen, H.A. and Yu, Z., 2015. Recent advances in 3D SEM surface reconstruction. Micron, 78, pp.54-66. [ScienceDirect]

[6] Pahlavan Tafti, A., Kirkpatrick, A.B., Owen, H.A. and Yu, Z., 2014. 3D Microscopy Vision Using Multiple View Geometry and Differential Evolutionary Approaches. In Advances in Visual Computing (pp. 141-152). Springer International Publishing. [SpringerLink]

 

10. Scientists who cited the 3DSEM dataset :

 
[1] Kudryavtsev, A.V., Dembélé, S. and Piat, N., 2017, July. Stereo-image rectification for dense 3D reconstruction in scanning electron microscope. In Manipulation, Automation and Robotics at Small Scales (MARSS), 2017 International Conference on (pp. 1-6). IEEE.

[2] Matev, A., Velev, P. and Herzog, M., 2017. An investigation on the possibility of fiberglass manufacture using a matrix obtained from water diluted mixtures of unsaturated lyester and  polyurethane resins. Journal of Chemical Technology & Metallurgy, 52(5).

[3] Shams, S., 2016. Experimental, analytical and numerical characterization of effects of fiber waviness defects in laminated composites (Doctoral dissertation, The University of Wisconsin-Milwaukee).

[4] Baghaie, A., 2016. Study of Computational Image Matching Techniques: Improving Our View of Biomedical Image Data (Doctoral dissertation, Ph. D. Thesis, University of Wisconsin-Milwaukee).

 

11. Useful Links:

 

Microscopy Info

Microscopy Society of America

European Microscopy Society

Royal Microscopical Society

 

Number of Visitors:

Leave a Reply

Your email address will not be published. Required fields are marked *