📄 Project Overview

<aside> <img src="/icons/cursor-click_gray.svg" alt="/icons/cursor-click_gray.svg" width="40px" /> Medip Software: is a software used for the visualization and analysis of CT and MRI scans. It is primarily utilized by doctors and clinicians to visualize body organs, measure their size, and identify affected lesions in CT and MRI images.

In this project, my responsibility was to enhance the classes of body organ segmentation and integrate deep learning-based interactive segmentation. To expand the segmentation capabilities of the software, I deployed vision transformers (ViT) that were trained on a large volume of CT datasets. Furthermore, I incorporated the MONAI library and utilized unsupervised learning methods to enable interactive segmentation. These algorithms enable one-click whole body segmentation from CT scans.

</aside>

🎯 Product Goals

<aside> 🎯 Develop and enhance product features.

</aside>

<aside> 🎯

Expand the segmentation classes for body organs.

</aside>

<aside> 🎯 Improve user interface and optimize deep learning models for faster inference.

</aside>

<aside> 🎯 Deploy algorithms and incorporate user feedback.

</aside>

<aside> 🎯 Conduct testing and validation.

</aside>

📥 My Contributions

<aside> 🛣️ Overseeing the project lifecycle. This includes planning and managing different stages, from conducting research to deploying the AI models.

</aside>

<aside> 🖥️ I extended the segmentation classes of body organs, increasing them from 20 to 124 classes.

</aside>

<aside> 🏗️ Add interactive segmentation based on deep learning

</aside>

<aside> <img src="/icons/groups_green.svg" alt="/icons/groups_green.svg" width="40px" /> Led a team of three junior developers in creating a robust C++-based API that integrates deep learning models with the software.

</aside>

<aside> 🛡️ Developed an innovative encryption strategy to protect the trained deep learning models, securing intellectual property of the company.

</aside>

<aside> 🏎️ I utilized TensorRT and ONNX to speed-up model inference from 48 seconds to 18 seconds, resulting in a three-fold speed improvement and enhanced performance.

</aside>

<aside> 🖥️ Design and implement a multi-thread API based on C++.

</aside>

<aside> 🏁 I collaborated with the IT team to deploy the vision transformer algorithms, verify results, and confirm post-deployment software refinement.

</aside>

🛠️ Tech Stacks Used

<aside> <img src="https://prod-files-secure.s3.us-west-2.amazonaws.com/cb4bc682-8064-42da-8247-5f27942c5d27/9f4fd066-9250-40d7-a6bd-90b00a6316e6/sagemaker_logo.svg" alt="https://prod-files-secure.s3.us-west-2.amazonaws.com/cb4bc682-8064-42da-8247-5f27942c5d27/9f4fd066-9250-40d7-a6bd-90b00a6316e6/sagemaker_logo.svg" width="40px" />

AWS SageMaker

</aside>

<aside> <img src="https://prod-files-secure.s3.us-west-2.amazonaws.com/cb4bc682-8064-42da-8247-5f27942c5d27/9acd32ee-f449-47d5-a3d3-7d4f551eb43a/qt.png" alt="https://prod-files-secure.s3.us-west-2.amazonaws.com/cb4bc682-8064-42da-8247-5f27942c5d27/9acd32ee-f449-47d5-a3d3-7d4f551eb43a/qt.png" width="40px" /> QT framework

</aside>

<aside> <img src="https://prod-files-secure.s3.us-west-2.amazonaws.com/cb4bc682-8064-42da-8247-5f27942c5d27/6551a5ba-e238-42c4-93e1-d9a80e75b741/python.png" alt="https://prod-files-secure.s3.us-west-2.amazonaws.com/cb4bc682-8064-42da-8247-5f27942c5d27/6551a5ba-e238-42c4-93e1-d9a80e75b741/python.png" width="40px" /> Python

</aside>

<aside> <img src="https://prod-files-secure.s3.us-west-2.amazonaws.com/cb4bc682-8064-42da-8247-5f27942c5d27/435bd915-7351-43bf-b1c0-7bb37e27d23f/PyTorch_logo_icon.svg.png" alt="https://prod-files-secure.s3.us-west-2.amazonaws.com/cb4bc682-8064-42da-8247-5f27942c5d27/435bd915-7351-43bf-b1c0-7bb37e27d23f/PyTorch_logo_icon.svg.png" width="40px" /> LibTorch & Torchscript

</aside>

<aside> <img src="https://prod-files-secure.s3.us-west-2.amazonaws.com/cb4bc682-8064-42da-8247-5f27942c5d27/34f66807-94c2-408b-a902-54208874ee0f/version-control-tortoise-svn.png" alt="https://prod-files-secure.s3.us-west-2.amazonaws.com/cb4bc682-8064-42da-8247-5f27942c5d27/34f66807-94c2-408b-a902-54208874ee0f/version-control-tortoise-svn.png" width="40px" /> Tortoise SVN

</aside>

<aside> <img src="https://prod-files-secure.s3.us-west-2.amazonaws.com/cb4bc682-8064-42da-8247-5f27942c5d27/5e0da7cd-de65-4980-ac72-f21daf00e6a4/gitlab.png" alt="https://prod-files-secure.s3.us-west-2.amazonaws.com/cb4bc682-8064-42da-8247-5f27942c5d27/5e0da7cd-de65-4980-ac72-f21daf00e6a4/gitlab.png" width="40px" /> Gitlab

</aside>

<aside> <img src="https://prod-files-secure.s3.us-west-2.amazonaws.com/cb4bc682-8064-42da-8247-5f27942c5d27/8b22e90c-6534-4563-9c2f-c10d89026041/AWS_Simple_Icons_AWS_Cloud.svg.png" alt="https://prod-files-secure.s3.us-west-2.amazonaws.com/cb4bc682-8064-42da-8247-5f27942c5d27/8b22e90c-6534-4563-9c2f-c10d89026041/AWS_Simple_Icons_AWS_Cloud.svg.png" width="40px" /> AWS (EC2)

</aside>

<aside> <img src="https://prod-files-secure.s3.us-west-2.amazonaws.com/cb4bc682-8064-42da-8247-5f27942c5d27/2a3854ad-4823-48ae-a08e-18af6513219d/redmine.png" alt="https://prod-files-secure.s3.us-west-2.amazonaws.com/cb4bc682-8064-42da-8247-5f27942c5d27/2a3854ad-4823-48ae-a08e-18af6513219d/redmine.png" width="40px" /> RedMine

</aside>

<aside> <img src="/icons/database_yellow.svg" alt="/icons/database_yellow.svg" width="40px" /> Data science

</aside>

<aside> <img src="https://prod-files-secure.s3.us-west-2.amazonaws.com/cb4bc682-8064-42da-8247-5f27942c5d27/89076378-e166-4668-baa0-a9b336f920b0/machine_learning.png" alt="https://prod-files-secure.s3.us-west-2.amazonaws.com/cb4bc682-8064-42da-8247-5f27942c5d27/89076378-e166-4668-baa0-a9b336f920b0/machine_learning.png" width="40px" /> Machine Learning

</aside>

<aside> <img src="https://prod-files-secure.s3.us-west-2.amazonaws.com/cb4bc682-8064-42da-8247-5f27942c5d27/852d3ad9-75b9-4a94-9858-add1a882171c/deep-learning-1524275-1290822.png" alt="https://prod-files-secure.s3.us-west-2.amazonaws.com/cb4bc682-8064-42da-8247-5f27942c5d27/852d3ad9-75b9-4a94-9858-add1a882171c/deep-learning-1524275-1290822.png" width="40px" /> Deep Learning

</aside>

<aside> <img src="https://prod-files-secure.s3.us-west-2.amazonaws.com/cb4bc682-8064-42da-8247-5f27942c5d27/7aab5311-e12d-41ab-ac0f-35dfee03c374/Tensorflow_logo.svg.png" alt="https://prod-files-secure.s3.us-west-2.amazonaws.com/cb4bc682-8064-42da-8247-5f27942c5d27/7aab5311-e12d-41ab-ac0f-35dfee03c374/Tensorflow_logo.svg.png" width="40px" /> Tensorboard

</aside>

📝 Introduction

Medical images (CT, MRI, X-Ray) are extensively used for diagnosis and prognosis analysis of various diseases such as lesion and cancer. Organ segmentation is necessary in order to accurately identify the disease in the organ and measure its status. What’s more, the segmented organ can be further used for 3D printing which subsequently can be used for surgery simulation and medical education. There has not been a comprehensive medical image analysis software on the market to enable doctors and clinician to efficiently segment body organs from CT or MRI images. Currently, these segmentations are performed manually using software such as 3D slicer.

We develop Medip - a comprehensive medical image analysis solution. Advanced algorithms are integrated to automatically segment body organ from CT images. More features such as 3D modeling, mesh processing and mesh smoothing are integrated with Medip to enable 3D model rendering of body organs.

🖇️ My contribution

• I was leading meetings with various US companies such as NVIDIA, Amazon AWS, New York University, Harvard University Hospital as well as Universities in Brazil, Italy and Turkey to build partnership.
• I have successfully built the first partnership with NVIDIA and Medical IP, Amazon and Medical IP and Brazil and Medical IP and Italy and Medical IP.
• I was also in charge of enhancing the features of the Medip software by incorporating novel approaches to extend the classification of body organs, optimize deep learning models, and accelerate the inference speed.
• I was collaborating with IT team in the deployment of AI models and enhancing user experience

🪖Challenges

• Training AI models to segment body organs from different CT modality and scan quality.
• Designing data pipeline for automated cleaning and pre-processing of large scale datasets
• Handling the data imbalances and solving various AI model training issues
• Converting the trained AI models into their equivalent C++ version for deployment to Medip.
• Encryption of trained models to protect the intellectual property of the company
• AI model optimization using FP16, Int8 and weight prunning for faster inference
• Collaborating with IT team for AI model deployment
• Continuous development and continuous integration of AI models

🌏 International Relation

• Along side my research and development responsibilities, I was also in charge of expanding the global presence of Medical IP by building partnership with various international companies.
• On behalf Medical IP, I attended international meetings and conferences to discuss the usability and implementation feasibility of Medical IP products.
• We build the first partnership between Medical IP Inc and NVIDIA, which led to the deployment of Medical IP’s software onto NVIDIA’s marketplace.
• We build the first partnership between Medical IP and Amazon , which enabled us to upload our product on Amazon market place.

⚒️ Product Development

1. Large scale data-pipeline processing
2. Swin Vision Transformer development and taining
3. SegCore API to deploy the developed AI models on to the product

⛓️ Feature Enhancement

• Multithreading
• API status monitoring
• Error and exception handling
• Best practices of C++ data structure such as vector, tuple and pointers.
• Extensibility and modularity
• Easy debugging and enhancement

🧪 Research Resources

Demo Video - Showcasing 124 anatomy segmentation in Medip software