Advancements in 3D Cellular Microscopy Imaging Through Fine-Tuned CellSAM and DevoNet Models
- Mentors
- Jesse Parent, Bradly Alicea, Jiahang Li, Mayukh Deb
- Organization
- INCF
- Technologies
- python, pytorch, chainer, topological data analysis
- Topics
- machine learning, computer vision, deep learning, Graph Neural Network
This project centers on the enhancement and integration of the CellSAM and DevoNet models to elevate the analysis of 3D microscopy images stored in TIFF format. CellSAM, initially a segment-anything model, was fine-tuned to allow promptable segmentation of cellular structures in microscopy images. Conversely, DevoNet, a dual vision encoder model, operates seamlessly on 3D TIFF files. One encoder is trained on segmentation maps while the other specializes in analyzing the centroid position of cells. This synchronized operation aids in the precise calculation of cellular volume, area, and centroid locations. The collaborative mechanism between the fine-tuned CellSAM and DevoNet models provides a robust framework for more accurate and insightful analysis of 3D cellular microscopy images, thereby broadening the understanding of cellular morphologies and interactions within a three-dimensional space.