Artificial Intelligence • EngerLab

Mission

Our Artificial Intelligence Group aims to move towards personalized healthcare for cancer patients, applied to different medical modalities such as diagnostic and histopathology images, textual reports, genomics data, etc. using deep learning methods. Our AI group is in active partnership and collaboration with the Mila – Quebec AI Institute (the world’s largest academic research center for deep learning), TransMedTech, Radiomics, Exactis, MyPL, and the AC Camargo Cancer Center, Sao Paulo, Brazil. Founded in 2021, we are also home to an eight-week summer school McMedHacks on deep learning applications in medical image analysis which attracted 356 participants from 38 countries and 1100 participants from 62 countries.

Members

Projects

Robust Optimization in HDR brachytherapy

Bjorn Moren

Robust optimization for IMBT rectal cancer Intensity modulated brachytherapy (IMBT) is a delivery technique for brachytherapy (BT) in which shields are used to further modulate the dose distribution, as compared to conventional high dose-rate (HDR) BT. For rectal cancer, our group has designed and built a shielded applicator. In this project, we investigate rotational uncertainties of the applicator and what effect those could have on evaluation criteria such as the dose-volume histogram. Furthermore, a robust optimization approach is proposed to mitigate these effects from uncertainty. Catheter placement for prostate HDR brachytherapy – Investigating trade-offs between spatial properties and number of catheters We study the treatment planning project of HDR BT for prostate cancer including both decisions on catheter placements and dwell times. Our focus is to improve spatial properties such as contiguous volumes that receive a high or a low dose, referred to as hot spots and cold spots, respectively. Further, we investigate the trade-offs between the number of inserted catheters and the sizes of hot spots and cold spots.

AI-based treatment planning for brachytherapy applications

Hossein Jafarzadeh

Treatment plan optimization is a routine part of both external beam radiotherapy and high dose rate brachytherapy. High dose rate brachytherapy is a mode of internal radiotherapy in which a source of radiation is placed inside the tumor through hollow needles known as catheters (interstitial brachytherapy) or near the tumor through site specific applicators (intracavitary brachytherapy). For every patient, the process of treatment plan optimization ensures that the dose to the tumor region is sufficient while the dose to the surrounding organs at risk is minimal. Therefore, treatment plan optimization is a key process in ensuring positive radiotherapy outcome.
In interstitial high dose rate brachytherapy, treatment planning is a labor-intensive and time-consuming task. The treatment planning cannot begin before the patient is put under anesthesia and the catheters are inserted. This makes the process painful for the patient and costly for the hospital. In addition, the current optimization method only controls the amount of time for which the source dwells at a specific location inside a catheter. The number and the location of the catheters are not optimized.
Hossein investigates the use of reinforcement learning for optimizing the number and position of the catheters prior to catheter insertion, in addition to the dwell time optimization. This way, quality of the treatment will improve because the catheters will be inserted according to an optimized plan.

Named-entity Recognition and Relation Extraction from Textual Clinical Reports

Parsa Bagherzadeh

Deep Learning-Based Multimodality Treatment Outcome Prediction

Yujing Zou, Alana Thibodeau-Antonacci

Cancer is the leading cause of death in Canada. Radiotherapy is used in about half of all cancer treatments. However, it is clinically challenging to predict which patients will benefit from which treatment combination despite improved protocols, imaging techniques for cancer management, and combining various radiotherapy treatment modalities. An accurate method for predicting a patient’s likelihood of response may reduce unnecessary interventions, lower healthcare costs, and reduce side effects. Therefore, investigating how pre-treatment patient characteristics influence treatment efficacy as measured by post-treatment response is crucial. The aim of Yujing and Alana’s work is to develop a patient-specific, machine learning-based multi-modal treatment outcome prediction model for patients suffering from gynecological cancers. Our group has recently shown that the patient-specific radiation dose response may be influenced by inter-patient variation in tumor nuclei size. Therefore, the models developed by Yujing and Alana will not only integrate diagnostic images such as Computed Tomography, Magnetic Resonance, and Ultrasound images to detect high-order features that determine treatment response on a patient-specific basis, but also tumor nucleus size and cell spacing data obtained from scanned digital images of histopathology slides. Correlation between cell morphology and treatment outcome will also be investigated

Inter-observer variability of the manual segmentation

Alana Thibodeau-Antonacci

Alana is investigating the inter-observer variability of the manual segmentation of tumor regions in endoscopy images and its effect on treatment outcomes. Furthermore, they are developing a deep learning-based segmentation tool that can learn from multiple observers labels with high inter-observer variability.

AI-based dosimetry and catheter reconstruction for brachytherapy application

Sebastien Quetin

Brachytherapy is a form of radiotherapy where a sealed radiation source (seed) is placed inside or in close proximity of the tumor. The treatment starts by a radiation oncologist inserting catheters or an applicator inside or in proximity of the clinical treatment volume. Once this is done, CT or MR scans of the area are acquired. Organs at risk and the tumor are contoured by a radiation oncologist and the catheters/applicator are manually reconstructed by a medical physicist on the image set. Finally an optimized treatment plan is created to deliver an optimal dose to the tumor while sparing organs at risk. However, before optimization of the treatment plan, the absorbed dose contribution per second from each possible seed position along a catheter (dwell position) is determined. This is done by using precalculated dose distributions around a single seed and scaling it with respect to the daily air kerma strength of the seed.

The precalculated dose used by clinical treatment planning systems is based on the American Association of Physicists in Medicine (AAPM) Task Group 43 (TG-43) formalism which describes dose deposition around a single source centrally positioned in a spherical water phantom with unit density. The influence of patient tissue and applicator heterogeneities, intersource attenuation, and finite patient dimensions are ignored. Recently, more advanced model based dose calculation algorithms (MBDCAs) calculating dose to medium in medium have been developed. In MBDCAs, dose calculations are performed using the patient’s CT or MR images. Voxel-by-voxel assignment of tissue mass density and elemental composition is required. Tissue mass density is obtained from CT or synthetic CT images using a Hounsfield Unit (HU) to density calibration curve. Tissue composition and nominal mass density can also be assigned to contoured organs. Exact description of the source and applicator geometry, material composition, and nominal mass density is required as well. MBDCAs such as the Monte Carlo method, provide a detailed and accurate method for calculation of absorbed dose in heterogeneous systems such as the human body however are too time consuming to be used in a clinical workflow.

The main goal of Sébastien’s PhD project is to develop a precise and automated dosimetry algorithm that will take into account patient’s tissue/applicator heterogeneities and replace the time consuming Monte-Carlo simulations. Sébastien is also working on automating patient organ/tumor segmentation as well as catheter/applicator reconstruction to decrease the total treatment time.

Artificial intelligence with radiomics and deep learning for the diagnosis of malignancy in pancreatic cysts: a pilot study

Joshua Roccamo

Pancreatic cysts are common, with a prevalence of 15% in the general population, and up to 37% in older populations. While some cystic lesions of the pancreas, such as serous cystadenomas, pseudocysts and epidermoid cysts, have little to no malignant potential, others including intraductal papillary mucinous neoplasm (IPMN) and mucinous cystic neoplasm carry a risk of invasive adenocarcinoma. Even among the potentially malignant cyst types, it is difficult to predict which cysts harbor malignancy or have a high risk of malignant transformation. In this project, we aim to 1) develop a deep learning algorithm using radiomics from EUS, CT and MR images of pancreas cysts to predict malignancy; 2) To assess the diagnostic ability of this prediction algorithm in differentiating malignant from benign pancreatic cysts; 3) To compare the diagnostic ability of this AI algorithm to standard of care criteria for predicting malignant pancreatic cystic neoplasms; 4) To assess feasibility of the current methodology and AI algorithm for a large multicenterd trial with an external validation cohort.

Auto-segmentation of brain structures

James Tsui

Radiation therapy to brain tumor requires a delicate balance between delivering adequate tumorcidal dose to the tumor lesion while minimizing radiation to healthy organs. Segmentation of organs at risk (OARs) is time-consuming and subject to inter-observer variability. Auto-segmentation can 1) increase workflow output by lightening segmentation time, and 2) decrease inter-observer variability. The Radiation Oncology Department at the McGill University Health Centre in collaboration with the Montreal Children Hospital treat pediatric patients with brain tumors. Compared to adult population, radiation therapy to pediatric population is higher stake for several reasons.1) As opposed to adult patients with mature anatomy, radiation can several impair normal growth of pediatric patients; 2) There are more brain structures in pediatric patients to consider; 3) Most children brain structures are more sensitive to radiation than those of adults. Thus, accurate segmentation is of great importance. The project is to investigate deep learning algorithms that can accurately auto-segment OARs in pediatric patients undergoing brain radiation therapy.

McMedHacks

McMedHacks is an 8-week program about deep learning and medical image analysis and accumulates in a hackathon, where participants can solve real-life clinical medical image analysis challenges. The program consists of weekly presentations and workshops that are led by students as well as leaders in medical image analysis from academia and industry. Our mission is to bridge domains and bring deep learning in to the clinic by making it accessible to anyone.

Learn more

Publications

2022

Sebastien, Quetin

Artificial Intelligence-based Brachytherapy Presentation

From New avenues in the non-operative management of patients with rectal cancer Conference, 14.10.2022.

BibTeX

Zou, Yujing

TransMedTech Excellence Scholarship (Doctoral Award) award

2022.

Links | BibTeX

Sebastien, Quetin; Zou, Yujing

Deep Learning Framework : Tensorboard and Pytorch Lightning Workshop

2022.

Links | BibTeX

Zou, Yujing; Alvarez, David-Santiago Ayala

ESTRO 2022: innovations in brachytherapy Journal Article

In: ESTRO Newsletter, pp. 754–767, 2022.

Links | BibTeX

Sebastien, Quetin

Deep Learning Framework : Pytorch tensors and Autograd Workshop

2022.

Links | BibTeX

Sebastien, Quetin; Zou, Yujing

Introduction to medical image processing with Python : DICOM and histopathology images Workshop

2022.

Links | BibTeX

Zou, Yujing

Fonds de recherche du Québec - Nature et technologies (FRQNT) PhD doctoral training scholarship award

2022, ($84,000 for 2022 - 2026; Declined due to acceptance of FRQS).

Links | BibTeX

Zou, Yujing

Fonds de recherche du Québec - Santé (FRQS) PhD doctoral training scholarship award

2022, ($84,000 for 2022 - 2026).

Links | BibTeX

Zou, Yujing; Lecavalier-Barsoum, Magali; Pelmus, Manuela; Maleki, Farhad; Enger, Shirin A.

Predictive modeling of post radiation-therapy recurrence for gynecological cancer patients using clinical and histopathology imaging features Conference

Curietherapies 2022.

Abstract | Links | BibTeX

@conference{nokey,

title = {Predictive modeling of post radiation-therapy recurrence for gynecological cancer patients using clinical and histopathology imaging features},

author = {Yujing Zou and Magali Lecavalier-Barsoum and Manuela Pelmus and Farhad Maleki and Shirin A. Enger},

url = {https://www.researchgate.net/publication/361436138_Predictive_modeling_of_post_radiation-therapy_recurrence_for_gynecological_cancer_patients_using_clinical_and_histopathology_imaging_features},

year  = {2022},

date = {2022-05-23},

urldate = {2022-05-23},

organization = {Curietherapies},

abstract = {Purpose: To build a machine-learning (ML) classifier to predict the clinical endpoint of post-Radiation-Therapy (RT) recurrence of gynecological cancer patients, while exploring the outcome predictability of cell spacing and nuclei size pre-treatment histopathology image features and clinical variables. Materials and Methods: Thirty-six gynecological (i.e., cervix, vaginal, and vulva) cancer patients (median age at diagnosis = 59.5 years) with a median follow-up time of 25.7 months, nine of which (event rate of 25%) experienced post-RT recurrence, were included in this analysis. Patient-specific nuclei size and cell spacing distributions from cancerous and non-tumoral regions of pre-treatment hematoxylin and eosin (H&E) stained digital histopathology Whole-Slide-Images (WSI) were extracted. The mean and standard deviation of these distributions were computed as imaging features for each WSI. Clinical features of clinical and radiological stage at the time of radiation, p16 status, age at diagnosis, and cancer type were also obtained. Uniquely, a Tree-based Pipeline Optimization Tool (TPOT) AutoML approach, including hyperparameter tuning, was implemented to find the best performing pipeline for this class-imbalanced and small dataset. A Radial Basis Function Kernel (RBF) sampler (gamma = 0.25) was applied to combined imaging and clinical input variables for training. The resulting features were fed into an XGBoost (ie., eXtreme gradient-boosting) classifier (learning rate = 0.1). Its outputs were propagated as “synthetic features” followed by polynomial feature transforms. All raw and transformed features were trained with a decision tree classification algorithm. Results of model evaluation metrics from a 10-fold stratified shuffle split cross-validation were averaged. A permutation test (n=1000) was performed to validate the significance of the classification scores. Results: Our model achieved a 10-fold stratified shuffle split cross-validation scores of 0.87 for mean accuracy, 0.92 for mean balanced accuracy, 0.78 for precision, 1 for recall, 0.85 for F1 score, and 0.92 for Area Under the Curve of Receiver Operating Characteristics Curve, to predict our patient cohort’s post-RT recurrence binary outcome. A p-value of 0.036 was obtained from the permutation test. This implies real dependencies between our combined imaging and clinical features and outcomes which were learned by the classifier, and the primising model performance was not by chance. Conclusions: Despite the small dataset and low event rate, as a proof of concept, we showed that a decision-tree-based ML classification algorithm using an XGBoost algorithm is able to utilize combined (cell spacing & nuclei size) imaging and clinical features to predict post-RT outcomes for gynecological cancer patients.},

keywords = {},

pubstate = {published},

tppubtype = {conference}

}

Purpose: To build a machine-learning (ML) classifier to predict the clinical endpoint of post-Radiation-Therapy (RT) recurrence of gynecological cancer patients, while exploring the outcome predictability of cell spacing and nuclei size pre-treatment histopathology image features and clinical variables. Materials and Methods: Thirty-six gynecological (i.e., cervix, vaginal, and vulva) cancer patients (median age at diagnosis = 59.5 years) with a median follow-up time of 25.7 months, nine of which (event rate of 25%) experienced post-RT recurrence, were included in this analysis. Patient-specific nuclei size and cell spacing distributions from cancerous and non-tumoral regions of pre-treatment hematoxylin and eosin (H&E) stained digital histopathology Whole-Slide-Images (WSI) were extracted. The mean and standard deviation of these distributions were computed as imaging features for each WSI. Clinical features of clinical and radiological stage at the time of radiation, p16 status, age at diagnosis, and cancer type were also obtained. Uniquely, a Tree-based Pipeline Optimization Tool (TPOT) AutoML approach, including hyperparameter tuning, was implemented to find the best performing pipeline for this class-imbalanced and small dataset. A Radial Basis Function Kernel (RBF) sampler (gamma = 0.25) was applied to combined imaging and clinical input variables for training. The resulting features were fed into an XGBoost (ie., eXtreme gradient-boosting) classifier (learning rate = 0.1). Its outputs were propagated as “synthetic features” followed by polynomial feature transforms. All raw and transformed features were trained with a decision tree classification algorithm. Results of model evaluation metrics from a 10-fold stratified shuffle split cross-validation were averaged. A permutation test (n=1000) was performed to validate the significance of the classification scores. Results: Our model achieved a 10-fold stratified shuffle split cross-validation scores of 0.87 for mean accuracy, 0.92 for mean balanced accuracy, 0.78 for precision, 1 for recall, 0.85 for F1 score, and 0.92 for Area Under the Curve of Receiver Operating Characteristics Curve, to predict our patient cohort’s post-RT recurrence binary outcome. A p-value of 0.036 was obtained from the permutation test. This implies real dependencies between our combined imaging and clinical features and outcomes which were learned by the classifier, and the primising model performance was not by chance. Conclusions: Despite the small dataset and low event rate, as a proof of concept, we showed that a decision-tree-based ML classification algorithm using an XGBoost algorithm is able to utilize combined (cell spacing & nuclei size) imaging and clinical features to predict post-RT outcomes for gynecological cancer patients.

Zou, Yujing; Lecavalier-Barsoum, Magali; Pelmus, Manuela; Maleki, Farhad; Enger, Shirin A.

Young Investigator Competition Winner at the Curietherapies Conference award

2022.

Abstract | Links | BibTeX

@award{nokey,

title = {Young Investigator Competition Winner at the Curietherapies Conference },

author = {Yujing Zou and Magali Lecavalier-Barsoum and Manuela Pelmus and Farhad Maleki and Shirin A. Enger },

url = {https://www.researchgate.net/publication/360979157_SP-0014_McMedHacks_Deep_learning_for_medical_image_analysis_workshops_and_Hackathon_in_radiation_oncology},

year = {2022},

date = {2022-05-23},

urldate = {2022-05-23},

organization = {Curietherapies},

abstract = {Purpose/Objective: The McMedHacks workshop and presentation series was created to teach individuals from various backgrounds about deep learning (DL) for medical image analysis in May, 2021. Material/Methods: McMedHacks is a free and student-led 8-week summer program. Registration for the event was open to everyone, including a form to survey participants’ area of expertise, country of origin, level of study, and level of programming skills. The weekly workshops were instructed by 8 students and experts assisted by 20 mentors who provided weekly tutorials. Recent developments in DL and medical physics were highlighted by 21 leaders from industry and academia. A virtual grand challenge Hackathon took place at the end of the workshop series. All events were held virtually and recorded on Zoom to accommodate all time zones and locations. The workshops were designed as interactive coding demos and shared through Google Colab notebooks. Results: McMedHacks gained 356 registrations from participants of 38 different countries (Fig. 1) from undergraduates, to PhDs and MDs. A vast number of disciplines and professions were represented, dominated by medical physics students, academic, and clinical medical physicists (Fig. 2). Sixty-nine participants earned a certificate of completion by having engaged with at least 12 of all 14 events. The program received participant feedback average scores of 4.768, 4.478, 4.579, 4.292, 4.84 out of five for the qualities of presentation, workshop session, tutorial and mentor, assignments, and course delivery, respectively. The eight-week long workshop’s duration allowed participants to digest the taught materials in a continuous manner as opposed to bootcamp-style conference workshops. Conclusion: The overwhelming interest and engagement for the McMedHacks workshop series from the Radiation Oncology (RadOnc) community illustrates a demand for Artificial Intelligence (AI) education in RadOnc. The future of RadOnc clinics will inevitably integrate AI. Therefore, current RadOnc professionals, and student and resident trainees should be prepared to understand basic AI principles and its applications to troubleshoot, innovate, and collaborate. McMedHacks set an excellent example of promoting open and multidisciplinary education, scientific communication, and leadership for integrating AI education into the RadOnc community on an international level. Therefore, we advocate for implementation of AI curriculums in professional education programs such as Commission on Accreditation of Medical Physics Education Programs (CAMPEP). Furthermore, we encourage experts from around the world in the field of AI, or RadOnc, or both, to take initiatives like McMedHacks to collaborate and push forward AI education in their departments and lead practical workshops, regardless of their levels of education.},

keywords = {},

pubstate = {published},

tppubtype = {award}

}

Purpose/Objective: The McMedHacks workshop and presentation series was created to teach individuals from various backgrounds about deep learning (DL) for medical image analysis in May, 2021. Material/Methods: McMedHacks is a free and student-led 8-week summer program. Registration for the event was open to everyone, including a form to survey participants’ area of expertise, country of origin, level of study, and level of programming skills. The weekly workshops were instructed by 8 students and experts assisted by 20 mentors who provided weekly tutorials. Recent developments in DL and medical physics were highlighted by 21 leaders from industry and academia. A virtual grand challenge Hackathon took place at the end of the workshop series. All events were held virtually and recorded on Zoom to accommodate all time zones and locations. The workshops were designed as interactive coding demos and shared through Google Colab notebooks. Results: McMedHacks gained 356 registrations from participants of 38 different countries (Fig. 1) from undergraduates, to PhDs and MDs. A vast number of disciplines and professions were represented, dominated by medical physics students, academic, and clinical medical physicists (Fig. 2). Sixty-nine participants earned a certificate of completion by having engaged with at least 12 of all 14 events. The program received participant feedback average scores of 4.768, 4.478, 4.579, 4.292, 4.84 out of five for the qualities of presentation, workshop session, tutorial and mentor, assignments, and course delivery, respectively. The eight-week long workshop’s duration allowed participants to digest the taught materials in a continuous manner as opposed to bootcamp-style conference workshops. Conclusion: The overwhelming interest and engagement for the McMedHacks workshop series from the Radiation Oncology (RadOnc) community illustrates a demand for Artificial Intelligence (AI) education in RadOnc. The future of RadOnc clinics will inevitably integrate AI. Therefore, current RadOnc professionals, and student and resident trainees should be prepared to understand basic AI principles and its applications to troubleshoot, innovate, and collaborate. McMedHacks set an excellent example of promoting open and multidisciplinary education, scientific communication, and leadership for integrating AI education into the RadOnc community on an international level. Therefore, we advocate for implementation of AI curriculums in professional education programs such as Commission on Accreditation of Medical Physics Education Programs (CAMPEP). Furthermore, we encourage experts from around the world in the field of AI, or RadOnc, or both, to take initiatives like McMedHacks to collaborate and push forward AI education in their departments and lead practical workshops, regardless of their levels of education.

Zou, Yujing

Biological & Biomedical Engineering PhD Recruitment Award award

2022, (These awards are designed to help recruit top students to our program and are offered to applicants wishing to start in Fall or Winter. The standard recruitment awards are $10,000/year for three years for Doctoral students.).

Links | BibTeX

Jafarzadeh, Hossein

Biological & Biomedical Engineering PhD Recruitment Award award

2022.

Links | BibTeX

Zou, Yujing

Graduate Research Enhancement and Travel Awards (GREAT Awards) award

2022, (In 2009, Graduate and Postdoctoral Studies introduced the Graduate Research Enhancement and Travel awards (GREAT awards) program in consultation with the Faculty Deans and Associate Deans. These awards cover dissemination of research through graduate student presentations at conferences, and other graduate student research-enhancement activities, such as travel for fieldwork, archival inquiry and extra-mural collaborative research. GREAT budgets are allocated to Faculties each year, and as such, are managed directly by the Associate Dean's office.).

Links | BibTeX

Rahbaran, Maryam; Kalinowski, Jonathan; Tsui, James Man Git; DeCunha, Joseph; Enger, Shirin A.

Monte-Carlo Based Simulations of the Uncertainties in Clinical Water-Based Intravascular Brachytherapy Dosimetry Presentation

11.04.2022.

Abstract | BibTeX

@misc{nokey,

title = {Monte-Carlo Based Simulations of the Uncertainties in Clinical Water-Based Intravascular Brachytherapy Dosimetry},

author = {Maryam Rahbaran and Jonathan Kalinowski and James Man Git Tsui and Joseph DeCunha and Shirin A. Enger},

year = {2022},

date = {2022-04-11},

urldate = {2022-04-11},

journal = {MCMA},

abstract = {"Introduction

Coronary artery disease (CAD) is the most common form of cardiovascular disease and is caused by excess plaque along the arterial wall, blocking blood flow to the heart (stenosis). Percutaneous transluminal coronary angioplasty widens a narrowed artery, leaving behind metal stents (1). However, in-stent restenosis (ISR) may occur due to damage to the arterial wall tissue, triggering neointimal hyperplasia which produces fibrotic and calcified plaques, narrowing the artery again. Drug-eluting stents (DES) slowly release medication to inhibit neointimal hyperplasia to prevent ISR but they fail in 3% to 20% of cases (2). Intravascular brachytherapy (IVBT), which uses b-emitting radionuclides to prevent ISR, is used in these failed cases. However, current dosimetry for IVBT is water based and does not consider attenuation of the radiation by heterogeneities such as the IVBT device guidewire, non-uniform distribution of calcified plaques, and stent material, or the angular dependence of dose distribution (3, 4, 5). The aim of this study was to investigate the uncertainties in clinical water based IVBT dosimetry, considering the effect of heterogeneities on dose distribution.

Materials & Methods

An inhouse Monte-Carlo based dosimetry package for IVBT applications based on Geant4 10.04 (patch 2) was developed. Patient’s artery was modelled as a 32 mm long, 8.4 mm diameter cylinder comprised of three layers: tunica media, represented with muscle, tunica intima, represented with fibrotic plaque, and tunica adventitia, represented with collagen. These layers had mass densities 1.06 g/cm3, 1.22 g/cm3 and 1.07 g/cm3 respectively. The innermost layer consisted of calcified plaque of density 1.45 g/cm3 with varying thicknesses between 0.9 and 1.9 mm with an eccentric shape and a rough surface. The stents had similar composition to Boston Scientific Synergy stents and were modelled to not overlap. The Novoste Beta-Cath 3.5F IVBT device model was used, which has a 90Sr90Y source. The geometry is shown in Figure 1a. A cylindrical scoring geometry was implemented. Two set of simulations were performed. In the first simulation called water phantom, the entire system consisted of water with unit density, and dose to water was calculated similar to the clinical water based dosimetry. In the second simulation called the artery model proper material and mass densities were assigned to each component. To ensure uncertainties below 0.8% within a 1 mm radial distance to the source and 2% within 4.2 mm from the source, 100 million decay events were simulated. The Penelope physics list was used to simulate the electromagnetic interactions between particles. Average, minimum, and maximum dose was calculated at 2.0 mm from the source center and directly and 1 mm behind the outermost stents and guidewire. Absorbed dose was normalized to 23 Gy at 2.0 mm from the source center.

Results

International Conference on Monte Carlo Techniques for Medical Applications, 2022

Compared to the water phantom (Figure 1b), average dose in the artery model (Figure 1c) was attenuated by 50.9% at 2 mm from the source centre and directly behind the guidewire and outermost stent by 66.2%, and by 69.5% 1 mm behind this region. There was significant variation in dose around the source due to the guidewire attenuating dose the most, and heterogeneous distribution of calcification.

Discussion & Conclusions

Dosimetry for IVBT based on dose rate in water is not accurate. Heterogeneities need to be considered to deliver adequate dose to the lesion area. Stent material, heterogenous distribution of calcification and the off cantered placement of the guidewire affects the uniformity of dose distribution around the source. Patients may benefit from personalized treatment planning taking dose-attenuating by different tissue/material heterogeneities into account.

References

[1] Virani, Salim, S., et al. ""Heart Disease and Stroke Statistics—2020 Update"". Circulation, vol. 141, no. 9, March 03, 2020, pp. e336. doi: 10.1161/CIR.0000000000000757.

[2] Lee M, Banka G. In-stent restenosis. Interv Cardiol Clin 2016;5: 211e220.

[3] Chiu-Tsao ST, Schaart DR, Soares CG, et al. Dose calculation formalisms and consensus dosimetry parameters for intravascular brachytherapy dosimetry: Recommendations of the AAPM Therapy Physics Committee Task Group No. 149. Med Phys 2007;34: 4126e4157.

[4] Rivard MJ, Coursey BM, DeWerd LA, et al. Update of AAPM Task Group No. 43 Report: A revised AAPM protocol for brachytherapy dose calculations. Med Phys 2004;31:633e674.

[5] Nath R, Amols H, Coffey C, et al. Intravascular brachytherapy physics: Report of the AAPM Radiation Therapy Committee Task group No. 60. Med Phys 1999;26:119e152.’

[6] Agostinelli S, Allison J, Amako K, et al. Geant4da simulation toolkit. Nucl Instrum Methods Phys Res 2003;506:230e303."},

keywords = {},

pubstate = {published},

tppubtype = {presentation}

}

"Introduction
Coronary artery disease (CAD) is the most common form of cardiovascular disease and is caused by excess plaque along the arterial wall, blocking blood flow to the heart (stenosis). Percutaneous transluminal coronary angioplasty widens a narrowed artery, leaving behind metal stents (1). However, in-stent restenosis (ISR) may occur due to damage to the arterial wall tissue, triggering neointimal hyperplasia which produces fibrotic and calcified plaques, narrowing the artery again. Drug-eluting stents (DES) slowly release medication to inhibit neointimal hyperplasia to prevent ISR but they fail in 3% to 20% of cases (2). Intravascular brachytherapy (IVBT), which uses b-emitting radionuclides to prevent ISR, is used in these failed cases. However, current dosimetry for IVBT is water based and does not consider attenuation of the radiation by heterogeneities such as the IVBT device guidewire, non-uniform distribution of calcified plaques, and stent material, or the angular dependence of dose distribution (3, 4, 5). The aim of this study was to investigate the uncertainties in clinical water based IVBT dosimetry, considering the effect of heterogeneities on dose distribution.

Materials & Methods
An inhouse Monte-Carlo based dosimetry package for IVBT applications based on Geant4 10.04 (patch 2) was developed. Patient’s artery was modelled as a 32 mm long, 8.4 mm diameter cylinder comprised of three layers: tunica media, represented with muscle, tunica intima, represented with fibrotic plaque, and tunica adventitia, represented with collagen. These layers had mass densities 1.06 g/cm3, 1.22 g/cm3 and 1.07 g/cm3 respectively. The innermost layer consisted of calcified plaque of density 1.45 g/cm3 with varying thicknesses between 0.9 and 1.9 mm with an eccentric shape and a rough surface. The stents had similar composition to Boston Scientific Synergy stents and were modelled to not overlap. The Novoste Beta-Cath 3.5F IVBT device model was used, which has a 90Sr90Y source. The geometry is shown in Figure 1a. A cylindrical scoring geometry was implemented. Two set of simulations were performed. In the first simulation called water phantom, the entire system consisted of water with unit density, and dose to water was calculated similar to the clinical water based dosimetry. In the second simulation called the artery model proper material and mass densities were assigned to each component. To ensure uncertainties below 0.8% within a 1 mm radial distance to the source and 2% within 4.2 mm from the source, 100 million decay events were simulated. The Penelope physics list was used to simulate the electromagnetic interactions between particles. Average, minimum, and maximum dose was calculated at 2.0 mm from the source center and directly and 1 mm behind the outermost stents and guidewire. Absorbed dose was normalized to 23 Gy at 2.0 mm from the source center.

Results
International Conference on Monte Carlo Techniques for Medical Applications, 2022
Compared to the water phantom (Figure 1b), average dose in the artery model (Figure 1c) was attenuated by 50.9% at 2 mm from the source centre and directly behind the guidewire and outermost stent by 66.2%, and by 69.5% 1 mm behind this region. There was significant variation in dose around the source due to the guidewire attenuating dose the most, and heterogeneous distribution of calcification.

Discussion & Conclusions
Dosimetry for IVBT based on dose rate in water is not accurate. Heterogeneities need to be considered to deliver adequate dose to the lesion area. Stent material, heterogenous distribution of calcification and the off cantered placement of the guidewire affects the uniformity of dose distribution around the source. Patients may benefit from personalized treatment planning taking dose-attenuating by different tissue/material heterogeneities into account.

References
[1] Virani, Salim, S., et al. ""Heart Disease and Stroke Statistics—2020 Update"". Circulation, vol. 141, no. 9, March 03, 2020, pp. e336. doi: 10.1161/CIR.0000000000000757.
[2] Lee M, Banka G. In-stent restenosis. Interv Cardiol Clin 2016;5: 211e220.
[3] Chiu-Tsao ST, Schaart DR, Soares CG, et al. Dose calculation formalisms and consensus dosimetry parameters for intravascular brachytherapy dosimetry: Recommendations of the AAPM Therapy Physics Committee Task Group No. 149. Med Phys 2007;34: 4126e4157.
[4] Rivard MJ, Coursey BM, DeWerd LA, et al. Update of AAPM Task Group No. 43 Report: A revised AAPM protocol for brachytherapy dose calculations. Med Phys 2004;31:633e674.
[5] Nath R, Amols H, Coffey C, et al. Intravascular brachytherapy physics: Report of the AAPM Radiation Therapy Committee Task group No. 60. Med Phys 1999;26:119e152.’
[6] Agostinelli S, Allison J, Amako K, et al. Geant4da simulation toolkit. Nucl Instrum Methods Phys Res 2003;506:230e303."

Weishaupt, Luca L.; Sayed, Hisham Kamal; Mao, Ximeng; Choo, Richard; Stish, Bradley J; Enger, Shirin A; Deufel, Christopher

Approaching automated applicator digitization from a new angle: Using sagittal images to improve deep learning accuracy and robustness in high-dose-rate prostate brachytherapy Journal Article

In: Brachytherapy, 2022.

BibTeX

Jafarzadeh, Hossein; Mao, Ximeng; Enger, Shirin A.

Bayesian Optimization in Treatment Planning of High Dose Rate Brachytherapy Inproceedings

In: MEDICAL PHYSICS, pp. E200–E200, WILEY 111 RIVER ST, HOBOKEN 07030-5774, NJ USA 2022.

BibTeX

Zou, Yujing; Lecavalier-barsoum, Magali; Pelmus, Manuela; Enger, Shirin A.

Patient-Specific Nuclei Size and Cell Spacing Distribution Extraction From Histopathology Whole Slide Images for Treatment Outcome Prediction Modelling Inproceedings

In: MEDICAL PHYSICS, pp. E266–E266, WILEY 111 RIVER ST, HOBOKEN 07030-5774, NJ USA 2022.

Abstract | Links | BibTeX

@inproceedings{zou2022patient,

title = {Patient-Specific Nuclei Size and Cell Spacing Distribution Extraction From Histopathology Whole Slide Images for Treatment Outcome Prediction Modelling},

author = {Yujing Zou and Magali Lecavalier-barsoum and Manuela Pelmus and Shirin A. Enger},

url = {https://w4.aapm.org/meetings/2022AM/programInfo/programAbs.php?sid=10686&aid=66642},

year  = {2022},

date = {2022-01-01},

urldate = {2022-01-01},

booktitle = {MEDICAL PHYSICS},

volume = {49},

number = {6},

pages = {E266--E266},

organization = {WILEY 111 RIVER ST, HOBOKEN 07030-5774, NJ USA},

abstract = {MO-C930-IePD-F5-1 (Monday, 7/11/2022) 9:30 AM - 10:00 AM [Eastern Time (GMT-4)]



Exhibit Hall | Forum 5



Purpose: To deliver a fully automated and generalizable approach extracting patient-specific nuclei size (ns) and cell spacing (cs) distributions from cancerous and non-tumoral regions of hematoxylin and eosin (H&E) stained digital histopathology Whole-Slide-Images (WSI) for gynecological cancer multiscale treatment outcome modelling.



Methods: Each pre-treatment gigapixel H&E WSI digitized at 40 x magnification (0.2482 microns/pixel) were divided into 5000 x 5000-pixel patches. Within each patch, the nucleus centers were identified by a difference of gaussians blob detection algorithm obtaining Delaunay triangulations and Voronoi diagrams providing cs radius. The ns radius was computed from stained pixels dominated by hematoxylin content with an automatic thresholding algorithm. With multiprocessing CPUs on a PC for each WSI, eight feature types were calculated preserving biopsies tissue heterogeneity: the mean and standard deviation of cs and ns distributions concatenated from all patches for cancerous and non-tumoral regions. This method was applied to 40 patients (1 WSI per patient) with treatment outcomes of post radiation-therapy (RT) recurrence (n = 9),and death (n = 8)).



Results: The WSI cancerous region cs distribution mean among patients without post-RT recurrence has a median of 6.64 microns, and those with post-RT recurrence with a median of 7.11 microns. This indicates the potential of utilizing such distribution features in treatment outcome prognosis modelling. Furthermore, at the third quartile, the WSI non-tumoral region ns distribution standard deviation among patients without post-RT recurrence has a value of 2.16 microns, and 1.46 microns for those with post-RT recurrence.



Conclusion: Our approach derives patient-specific microscopic data distributions from histopathology WSI that can be directly associated with retrospective patient outcomes. They are complementary and spatially orthogonal to information served by other medical imaging modalities such as CT, MR, and Ultrasound. Therefore, it has the unique potential to augment treatment outcome model inference when properly fused with radiological scans.



Funding Support, Disclosures, and Conflict of Interest: CIHR grant number 103548 and Canada Research Chairs Program (grant #252135)



Keywords



Image Analysis, Feature Extraction, Radiation Therapy



Taxonomy



IM/TH- Image Analysis (Single Modality or Multi-Modality): Imaging biomarkers and radiomics



Contact Email



yujing.zou@mail.mcgill.ca},

keywords = {},

pubstate = {published},

tppubtype = {inproceedings}

}

MO-C930-IePD-F5-1 (Monday, 7/11/2022) 9:30 AM - 10:00 AM [Eastern Time (GMT-4)]

Exhibit Hall | Forum 5

Purpose: To deliver a fully automated and generalizable approach extracting patient-specific nuclei size (ns) and cell spacing (cs) distributions from cancerous and non-tumoral regions of hematoxylin and eosin (H&E) stained digital histopathology Whole-Slide-Images (WSI) for gynecological cancer multiscale treatment outcome modelling.

Methods: Each pre-treatment gigapixel H&E WSI digitized at 40 x magnification (0.2482 microns/pixel) were divided into 5000 x 5000-pixel patches. Within each patch, the nucleus centers were identified by a difference of gaussians blob detection algorithm obtaining Delaunay triangulations and Voronoi diagrams providing cs radius. The ns radius was computed from stained pixels dominated by hematoxylin content with an automatic thresholding algorithm. With multiprocessing CPUs on a PC for each WSI, eight feature types were calculated preserving biopsies tissue heterogeneity: the mean and standard deviation of cs and ns distributions concatenated from all patches for cancerous and non-tumoral regions. This method was applied to 40 patients (1 WSI per patient) with treatment outcomes of post radiation-therapy (RT) recurrence (n = 9),and death (n = 8)).

Results: The WSI cancerous region cs distribution mean among patients without post-RT recurrence has a median of 6.64 microns, and those with post-RT recurrence with a median of 7.11 microns. This indicates the potential of utilizing such distribution features in treatment outcome prognosis modelling. Furthermore, at the third quartile, the WSI non-tumoral region ns distribution standard deviation among patients without post-RT recurrence has a value of 2.16 microns, and 1.46 microns for those with post-RT recurrence.

Conclusion: Our approach derives patient-specific microscopic data distributions from histopathology WSI that can be directly associated with retrospective patient outcomes. They are complementary and spatially orthogonal to information served by other medical imaging modalities such as CT, MR, and Ultrasound. Therefore, it has the unique potential to augment treatment outcome model inference when properly fused with radiological scans.

Funding Support, Disclosures, and Conflict of Interest: CIHR grant number 103548 and Canada Research Chairs Program (grant #252135)

Keywords

Image Analysis, Feature Extraction, Radiation Therapy

Taxonomy

IM/TH- Image Analysis (Single Modality or Multi-Modality): Imaging biomarkers and radiomics

Contact Email

yujing.zou@mail.mcgill.ca

Zou, Yujing; Weishaupt, Luca; Enger, Shirin A.

SP-0014 McMedHacks: Deep learning for medical image analysis workshops and Hackathon in radiation oncology Journal Article

In: Radiotherapy and Oncology, vol. 170, pp. S4–S5, 2022.

Abstract | Links | BibTeX

@article{zou2022sp,

title = {SP-0014 McMedHacks: Deep learning for medical image analysis workshops and Hackathon in radiation oncology},

author = {Yujing Zou and Luca Weishaupt and Shirin A. Enger},

url = {https://www-sciencedirect-com.proxy3.library.mcgill.ca/science/article/pii/S0167814022038695?via%3Dihub},

doi = {10.1016/S0167-8140(22)03869-5},

year  = {2022},

date = {2022-01-01},

urldate = {2022-01-01},

journal = {Radiotherapy and Oncology},

volume = {170},

pages = {S4--S5},

publisher = {Elsevier},

abstract = {Purpose/Objective: The McMedHacks workshop and presentation series was created to teach individuals from various backgrounds about deep learning (DL) for medical image analysis in May, 2021. Material/Methods: McMedHacks is a free and student-led 8-week summer program. Registration for the event was open to everyone, including a form to survey participants’ area of expertise, country of origin, level of study, and level of programming skills. The weekly workshops were instructed by 8 students and experts assisted by 20 mentors who provided weekly tutorials. Recent developments in DL and medical physics were highlighted by 21 leaders from industry and academia. A virtual grand challenge Hackathon took place at the end of the workshop series. All events were held virtually and recorded on Zoom to accommodate all time zones and locations. The workshops were designed as interactive coding demos and shared through Google Colab notebooks. Results: McMedHacks gained 356 registrations from participants of 38 different countries (Fig. 1) from undergraduates, to PhDs and MDs. A vast number of disciplines and professions were represented, dominated by medical physics students, academic, and clinical medical physicists (Fig. 2). Sixty-nine participants earned a certificate of completion by having engaged with at least 12 of all 14 events. The program received participant feedback average scores of 4.768, 4.478, 4.579, 4.292, 4.84 out of five for the qualities of presentation, workshop session, tutorial and mentor, assignments, and course delivery, respectively. The eight-week long workshop’s duration allowed participants to digest the taught materials in a continuous manner as opposed to bootcamp-style conference workshops. Conclusion: The overwhelming interest and engagement for the McMedHacks workshop series from the Radiation Oncology (RadOnc) community illustrates a demand for Artificial Intelligence (AI) education in RadOnc. The future of RadOnc clinics will inevitably integrate AI. Therefore, current RadOnc professionals, and student and resident trainees should be prepared to understand basic AI principles and its applications to troubleshoot, innovate, and collaborate. McMedHacks set an excellent example of promoting open and multidisciplinary education, scientific communication, and leadership for integrating AI education into the RadOnc community on an international level. Therefore, we advocate for implementation of AI curriculums in professional education programs such as Commission on Accreditation of Medical Physics Education Programs (CAMPEP). Furthermore, we encourage experts from around the world in the field of AI, or RadOnc, or both, to take initiatives like McMedHacks to collaborate and push forward AI education in their departments and lead practical workshops, regardless of their levels of education.},

keywords = {},

pubstate = {published},

tppubtype = {article}

}

Weishaupt, Luca L.; Vuong, Te; Thibodeau-Antonacci, Alana; Garant, A; Singh, K; Miller, C; Martin, A; Schmitt-Ulms, F; Enger, Shirin A.

PO-1325 Automated rectal tumor segmentation with inter-observer variability-based uncertainty estimates Journal Article

In: Radiotherapy and Oncology, vol. 170, pp. S1120–S1121, 2022.

BibTeX

Weishaupt, Luca L; Vuong, Te; Thibodeau-Antonacci, Alana; Garant, A; Singh, KS; Miller, C; Martin, A; Enger, Shirin A.

A121 QUANTIFYING INTER-OBSERVER VARIABILITY IN THE SEGMENTATION OF RECTAL TUMORS IN ENDOSCOPY IMAGES AND ITS EFFECTS ON DEEP LEARNING Journal Article

In: Journal of the Canadian Association of Gastroenterology, vol. 5, no. Supplement_1, pp. 140–142, 2022.

BibTeX

Thibodeau-Antonacci, Alana; Vuong, Te; Liontis, B; Rayes, F; Pande, S; Enger, Shirin A.

Development of a Novel MRI-Compatible Applicator for Intensity Modulated Rectal Brachytherapy Inproceedings

In: MEDICAL PHYSICS, pp. E240–E240, WILEY 111 RIVER ST, HOBOKEN 07030-5774, NJ USA 2022.

BibTeX

Thibodeau-Antonacci, Alana; Enger, Shirin A.; Bekerat, Hamed; Vuong, Te

Gafchromic film and scintillator detector measurements in phantom with a novel intensity-modulated brachytherapy endorectal shield Inproceedings

In: MEDICAL PHYSICS, pp. 5688–5689, WILEY 111 RIVER ST, HOBOKEN 07030-5774, NJ USA 2022.

BibTeX

2021

Weishaupt, Luca L.

T I Gurman Prize in Physics award

2021.

Abstract | Links | BibTeX

Thibodeau-Antonacci, Alana

Canada Graduate Scholarship – Doctoral Program award

2021.

Links | BibTeX

Weishaupt, Luca L.; Thibodeau-Antonacci, Alana; Garant, Aurelie; Singh, Kelita; Miller, Corey; Vuong, Té; Enger, Shirin A.

Deep learning based tumor segmentation of endoscopy images for rectal cancer patients Presentation

ESTRO Annual meeting, 27.08.2021.

Abstract | Links | BibTeX

@misc{Weishaupt2021b,

title = {Deep learning based tumor segmentation of endoscopy images for rectal cancer patients},

author = {Luca L. Weishaupt and Alana Thibodeau-Antonacci and Aurelie Garant and Kelita Singh and Corey Miller and Té Vuong and Shirin A. Enger},

url = {https://www.estro.org/Congresses/ESTRO-2021/610/posterdiscussion34-deep-learningforauto-contouring/3710/deeplearning-basedtumorsegmentationofendoscopyimag},

year = {2021},

date = {2021-08-27},

urldate = {2021-08-27},

abstract = {Purpose or Objective

The objective of this study was to develop an automated rectal tumor segmentation algorithm from endoscopy images. The algorithm will be used in a future multimodal treatment outcome prediction model. Currently, treatment outcome prediction models rely on manual segmentations of regions of interest, which are prone to inter-observer variability. To quantify this human error and demonstrate the feasibility of automated endoscopy image segmentation, we compare three deep learning architectures.

Material and Methods

A gastrointestinal physician (G1) segmented 550 endoscopy images of rectal tumors into tumor and non-tumor regions. To quantify the inter-observer variability, a second gastrointestinal physician (G2) contoured 319 of the images independently.

The 550 images and annotations from G1 were divided into 408 training, 82 validation, and 60 testing sets. Three deep learning architectures were trained; a fully convolutional neural network (FCN32), a U-Net, and a SegNet. These architectures have been used for robust medical image segmentation in previous studies.

All models were trained on a CPU supercomputing cluster. Data augmentation in the form of random image transformations, including scaling, rotation, shearing, Gaussian blurring, and noise addition, was used to improve the models' robustness.

The neural networks' output went through a final layer of noise removal and hole filling before evaluation. Finally, the segmentations from G2 and the neural networks' predictions were compared against the ground truth labels from G1.

Results

The FCN32, U-Net, and SegNet had average segmentation times of 0.77, 0.48, and 0.43 seconds per image, respectively. The average segmentation time per image for G1 and G2 were 10 and 8 seconds, respectively.

All the ground truth labels contained tumors, but G2 and the deep learning models did not always find tumors in the images. The scores are based on the agreement of tumor contours with G1’s ground truth and were thus only computed for images in which tumor was found. The automated segmentation algorithms consistently achieved equal or better scores than G2's manual segmentations. G2's low F1/DICE and precision scores indicate poor agreement between the manual contours.

Conclusion

There is a need for robust and accurate segmentation algorithms for rectal tumor segmentation since manual segmentation of these tumors is susceptible to significant inter-observer variability. The deep learning-based segmentation algorithms proposed in this study are more efficient and achieved a higher agreement with our manual ground truth segmentations than a second expert annotator. Future studies will investigate how to train deep learning models on multiple ground truth annotations to prevent learning observer biases.},

howpublished = {ESTRO Annual meeting},

keywords = {},

pubstate = {published},

tppubtype = {presentation}

}

Purpose or Objective
The objective of this study was to develop an automated rectal tumor segmentation algorithm from endoscopy images. The algorithm will be used in a future multimodal treatment outcome prediction model. Currently, treatment outcome prediction models rely on manual segmentations of regions of interest, which are prone to inter-observer variability. To quantify this human error and demonstrate the feasibility of automated endoscopy image segmentation, we compare three deep learning architectures.

Material and Methods
A gastrointestinal physician (G1) segmented 550 endoscopy images of rectal tumors into tumor and non-tumor regions. To quantify the inter-observer variability, a second gastrointestinal physician (G2) contoured 319 of the images independently.

The 550 images and annotations from G1 were divided into 408 training, 82 validation, and 60 testing sets. Three deep learning architectures were trained; a fully convolutional neural network (FCN32), a U-Net, and a SegNet. These architectures have been used for robust medical image segmentation in previous studies.

All models were trained on a CPU supercomputing cluster. Data augmentation in the form of random image transformations, including scaling, rotation, shearing, Gaussian blurring, and noise addition, was used to improve the models' robustness.

The neural networks' output went through a final layer of noise removal and hole filling before evaluation. Finally, the segmentations from G2 and the neural networks' predictions were compared against the ground truth labels from G1.

Results
The FCN32, U-Net, and SegNet had average segmentation times of 0.77, 0.48, and 0.43 seconds per image, respectively. The average segmentation time per image for G1 and G2 were 10 and 8 seconds, respectively.

All the ground truth labels contained tumors, but G2 and the deep learning models did not always find tumors in the images. The scores are based on the agreement of tumor contours with G1’s ground truth and were thus only computed for images in which tumor was found. The automated segmentation algorithms consistently achieved equal or better scores than G2's manual segmentations. G2's low F1/DICE and precision scores indicate poor agreement between the manual contours.

Conclusion
There is a need for robust and accurate segmentation algorithms for rectal tumor segmentation since manual segmentation of these tumors is susceptible to significant inter-observer variability. The deep learning-based segmentation algorithms proposed in this study are more efficient and achieved a higher agreement with our manual ground truth segmentations than a second expert annotator. Future studies will investigate how to train deep learning models on multiple ground truth annotations to prevent learning observer biases.

Thibodeau-Antonacci, Alana; Jafarzadeh, Hossein; Carroll, Liam; Weishaupt, Luca L.

Mitacs Globalink Research Award award

2021.

Abstract | Links | BibTeX

Weishaupt, Luca L.; Thibodeau-Antonacci, Alana; Garant, Aurelie; Singh, Kelita; Miller, Corey; Vuong, Té; Enger, Shirin A.

Inter-Observer Variability and Deep Learning in Rectal Tumor Segmentation from Endoscopy Images Presentation

The COMP Annual Scientific Meeting 2021, 22.06.2021.

Abstract | BibTeX

@misc{Weishaupt2021c,

title = {Inter-Observer Variability and Deep Learning in Rectal Tumor Segmentation from Endoscopy Images},

author = {Luca L. Weishaupt and Alana Thibodeau-Antonacci and Aurelie Garant and Kelita Singh and Corey Miller and Té Vuong and Shirin A. Enger},

year  = {2021},

date = {2021-06-22},

urldate = {2021-06-22},

abstract = {Purpose

To develop an automated rectal tumor segmentation algorithm from endoscopy images.



Material/Methods

A gastrointestinal physician (G1) segmented 2005 endoscopy images into tumor and non-tumor

regions. To quantify the inter-observer variability, a second gastrointestinal physician (G2)

contoured the images independently.



Three deep-learning architectures used for robust medical image segmentation in previous

studies were trained: a fully convolutional neural network (FCN32), a U-Net, and a SegNet.

Since the majority of the images did not contain tumors, two methods were compared for

training. Models were trained using only tumor images (M1) and all images (M2). G1’s images

and annotations were divided into 408 training, 82 validation, and 60 testing sets for M1, 1181

training, 372 validation, and 452 testing sets for M2.

Finally, segmentations from G2 and neural networks' predictions were compared against ground

truth labels from G1, and F1 scores were computed for images where both physicians found

tumors.



Results

The deep-learning segmentation took less than 1 second, while manual segmentation took

approximately 10 seconds per image.

The M1’s models consistently achieved equal or better scores (SegNet F1:0.80±0.08) than G2's

manual segmentations (F1:0.68±0.25). G2's low F1/DICE and precision scores indicate poor

agreement between the manual contours. Models from M2 achieved lower scores than G2 and

M1’s models since they demonstrated a strong bias towards predicting no tumor for all images.



Conclusion

Future studies will investigate training on an equal number of images with/without tumor, using

ground truth contours from multiple experts simultaneously.},

howpublished = {The COMP Annual Scientific Meeting 2021},

keywords = {},

pubstate = {published},

tppubtype = {presentation}

}

Morcos, Marc; Antaki, Majd; Thibodeau-Antonacci, Alana; Kalinowski, Jonathan; Glickman, Harry; Enger, Shirin A.

RapidBrachyMCTPS: An open-source dose calculation and optimization tool for brachytherapy research Presentation

COMP, 01.06.2021.

BibTeX

Thibodeau-Antonacci, Alana; Vuong, Té; Bekerat, Hamed; Liang, Liheng; Enger, Shirin A.

Development of a Dynamic Shielding Intensity-Modulated Brachytherapy Applicator for the Treatment of Rectal Cancer award

2021.

Abstract | Links | BibTeX

Weishaupt, Luca L.; Torres, Jose; Camilleri-Broët, Sophie; Rayes, Roni F.; Spicer, Jonathan D.; Maldonado, Sabrina Côté; Enger, Shirin A.

Deep learning-based tumor segmentation on digital images of histopathology slides for microdosimetry applications Journal Article

In: arXiv:2105.01824 [physics], 2021, (arXiv: 2105.01824).

Abstract | Links | BibTeX

@article{weishaupt_deep_2021,

title = {Deep learning-based tumor segmentation on digital images of histopathology slides for microdosimetry applications},

author = {Luca L. Weishaupt and Jose Torres and Sophie Camilleri-Broët and Roni F. Rayes and Jonathan D. Spicer and Sabrina Côté Maldonado and Shirin A. Enger},

url = {http://arxiv.org/abs/2105.01824},

year  = {2021},

date = {2021-05-01},

urldate = {2021-09-08},

journal = {arXiv:2105.01824 [physics]},

abstract = {$textbackslashbfPurpose:$ The goal of this study was (i) to use artificial intelligence to automate the traditionally labor-intensive process of manual segmentation of tumor regions in pathology slides performed by a pathologist and (ii) to validate the use of a well-known and readily available deep learning architecture. Automation will reduce the human error involved in manual delineation, increase efficiency, and result in accurate and reproducible segmentation. This advancement will alleviate the bottleneck in the workflow in clinical and research applications due to a lack of pathologist time. Our application is patient-specific microdosimetry and radiobiological modeling, which builds on the contoured pathology slides. $textbackslashbfMethods:$ A U-Net architecture was used to segment tumor regions in pathology core biopsies of lung tissue with adenocarcinoma stained using hematoxylin and eosin. A pathologist manually contoured the tumor regions in 56 images with binary masks for training. Overlapping patch extraction with various patch sizes and image downsampling were investigated individually. Data augmentation and 8-fold cross-validation were used. $textbackslashbfResults:$ The U-Net achieved accuracy of 0.91$textbackslashpm$0.06, specificity of 0.90$textbackslashpm$0.08, sensitivity of 0.92$textbackslashpm$0.07, and precision of 0.8$textbackslashpm$0.1. The F1/DICE score was 0.85$textbackslashpm$0.07, with a segmentation time of 3.24$textbackslashpm$0.03 seconds per image, achieving a 370$textbackslashpm$3 times increased efficiency over manual segmentation. In some cases, the U-Net correctly delineated the tumor's stroma from its epithelial component in regions that were classified as tumor by the pathologist. $textbackslashbfConclusion:$ The U-Net architecture can segment images with a level of efficiency and accuracy that makes it suitable for tumor segmentation of histopathological images in fields such as radiotherapy dosimetry, specifically in the subfields of microdosimetry.},

note = {arXiv: 2105.01824},

keywords = {},

pubstate = {published},

tppubtype = {article}

}

$textbackslashbfPurpose:$ The goal of this study was (i) to use artificial intelligence to automate the traditionally labor-intensive process of manual segmentation of tumor regions in pathology slides performed by a pathologist and (ii) to validate the use of a well-known and readily available deep learning architecture. Automation will reduce the human error involved in manual delineation, increase efficiency, and result in accurate and reproducible segmentation. This advancement will alleviate the bottleneck in the workflow in clinical and research applications due to a lack of pathologist time. Our application is patient-specific microdosimetry and radiobiological modeling, which builds on the contoured pathology slides. $textbackslashbfMethods:$ A U-Net architecture was used to segment tumor regions in pathology core biopsies of lung tissue with adenocarcinoma stained using hematoxylin and eosin. A pathologist manually contoured the tumor regions in 56 images with binary masks for training. Overlapping patch extraction with various patch sizes and image downsampling were investigated individually. Data augmentation and 8-fold cross-validation were used. $textbackslashbfResults:$ The U-Net achieved accuracy of 0.91$textbackslashpm$0.06, specificity of 0.90$textbackslashpm$0.08, sensitivity of 0.92$textbackslashpm$0.07, and precision of 0.8$textbackslashpm$0.1. The F1/DICE score was 0.85$textbackslashpm$0.07, with a segmentation time of 3.24$textbackslashpm$0.03 seconds per image, achieving a 370$textbackslashpm$3 times increased efficiency over manual segmentation. In some cases, the U-Net correctly delineated the tumor's stroma from its epithelial component in regions that were classified as tumor by the pathologist. $textbackslashbfConclusion:$ The U-Net architecture can segment images with a level of efficiency and accuracy that makes it suitable for tumor segmentation of histopathological images in fields such as radiotherapy dosimetry, specifically in the subfields of microdosimetry.

Thibodeau-Antonacci, Alana; Vuong, Té; Bekerat, Hamed; Childress, Lilian; Enger, Shirin A.

OC-0112 development of a dynamic-shielding intensity modulated endorectal brachytherapy applicator Presentation

Radiotherapy and Oncology, 01.05.2021, ISBN: 0167-8140, 1879-0887.

Abstract | Links | BibTeX

Zou, Yujing; Lecavalier-Barsoum, Magali; Enger, Shirin A.

Treatment outcome Prediction for gynecological cancers patients with a machine learning model using pre/post diagnostic image modalities and digital histopathology images Presentation

CRUK RadNet Manchester AI for Optimising Radiotherapy Outcomes Workshop, 10.02.2021.

Abstract | BibTeX

Weishaupt, Luca L.

Fire-Up - Radiation Treatment Outcome Prediction Presentation

Fire-Up Presentation, 09.02.2021.

BibTeX

Weishaupt, Luca L.; Sayed, Hisham Kamal; Mao, Ximeng; Choo, Chunhee; Stish, Bradley; Enger, Shirin A.; Deufel, Christopher

Approaching automated applicator digitization from a new angle: using sagittal images to improve deep learning accuracy and robustness in high-dose-rate prostate brachytherapy Journal Article

In: 2021 ABS Annual Meeting, 2021, (Type: Journal Article).

BibTeX

Deufel, Christopher; Weishaupt, Luca L.; Sayed, Hisham Kamal; Choo, Chunhee; Stish, Bradley

Deep learning for automated applicator reconstruction in high-dose-rate prostate brachytherapy Journal Article

In: World Congress of Brachytherapy 2021, 2021, (Type: Journal Article).

Links | BibTeX

2020

Mao, Ximeng; Pineau, Joelle; Keyes, Roy; Enger, Shirin A.

RapidBrachyDL: Rapid Radiation Dose Calculations in Brachytherapy Via Deep Learning Journal Article

In: International Journal of Radiation Oncology, Biology, Physics, vol. 108, no. 3, pp. 802–812, 2020, ISSN: 1879-355X.

Abstract | Links | BibTeX

@article{mao_rapidbrachydl_2020,

title = {RapidBrachyDL: Rapid Radiation Dose Calculations in Brachytherapy Via Deep Learning},

author = {Ximeng Mao and Joelle Pineau and Roy Keyes and Shirin A. Enger},

doi = {10.1016/j.ijrobp.2020.04.045},

issn = {1879-355X},

year  = {2020},

date = {2020-11-01},

journal = {International Journal of Radiation Oncology, Biology, Physics},

volume = {108},

number = {3},

pages = {802--812},

abstract = {PURPOSE: Detailed and accurate absorbed dose calculations from radiation interactions with the human body can be obtained with the Monte Carlo (MC) method. However, the MC method can be slow for use in the time-sensitive clinical workflow. The aim of this study was to provide a solution to the accuracy-time trade-off for 192Ir-based high-dose-rate brachytherapy by using deep learning. 

METHODS AND MATERIALS: RapidBrachyDL, a 3-dimensional deep convolutional neural network (CNN) model, is proposed to predict dose distributions calculated with the MC method given a patient's computed tomography images, contours of clinical target volume (CTV) and organs at risk, and treatment plan. Sixty-one patients with prostate cancer and 10 patients with cervical cancer were included in this study, with data from 47 patients with prostate cancer being used to train the model. 

RESULTS: Compared with ground truth MC simulations, the predicted dose distributions by RapidBrachyDL showed a consistent shape in the dose-volume histograms (DVHs); comparable DVH dosimetric indices including 0.73% difference for prostate CTV D90, 1.1% for rectum D2cc, 1.45% for urethra D0.1cc, and 1.05% for bladder D2cc; and substantially smaller prediction time, acceleration by a factor of 300. RapidBrachyDL also demonstrated good generalization to cervical data with 1.73%, 2.46%, 1.68%, and 1.74% difference for CTV D90, rectum D2cc, sigmoid D2cc, and bladder D2cc, respectively, which was unseen during the training. 

CONCLUSION: Deep CNN-based dose estimation is a promising method for patient-specific brachytherapy dosimetry. Desired radiation quantities can be obtained with accuracies arbitrarily close to those of the source MC algorithm, but with much faster computation times. The idea behind deep CNN-based dose estimation can be safely extended to other radiation sources and tumor sites by following a similar training process.},

keywords = {},

pubstate = {published},

tppubtype = {article}

}

Zou, Yujing

Graduate Excellence Fellowship award

2020.

BibTeX

Weishaupt, Luca L.; Sayed, Hisham Kamal; Mao, Ximeng; Choo, Chunhee; Stish, Bradley; Enger, Shirin A.; Deufel, Christopher

Approaching automated applicator digitization from a new angle - Using sagittal images to improve deep learning accuracy and robustness in high-dose-rate prostate brachytherapy Presentation

12.06.2020, (Type: Journal Article).

Links | BibTeX

Weishaupt, Luca L.

Math And Physics Class Of 1965 Prize award

2020.

Abstract | BibTeX

Mao, Ximeng; Pineau, Joelle; Keyes, Roy; Enger, Shirin A.

RapidBrachyDL: Rapid Radiation Dose Calculations in Brachytherapy Via Deep Learning Journal Article

In: Int J Radiat Oncol Biol Phys, vol. 108, no. 3, pp. 802-812, 2020, ISSN: 0360-3016.

Links | BibTeX

Weishaupt, Luca L.; Torres, Jose; Camilleri-Broët, Sophie; Maldonado, Sabrina Côté; Enger, Shirin A.

Classification and Segmentation of Tumor Cells and Nuclei On Biopsy Slides Using Deep Learning for Microdosimetry Applications Journal Article

In: 2020 Joint AAPM textbar COMP Virtual Meeting, 2020, (Type: Journal Article).

Abstract | Links | BibTeX

@article{weishaupt_classification_2020,

title = {Classification and Segmentation of Tumor Cells and Nuclei On Biopsy Slides Using Deep Learning for Microdosimetry Applications},

author = {Luca L. Weishaupt and Jose Torres and Sophie Camilleri-Broët and Sabrina Côté Maldonado and Shirin A. Enger},

url = {https://w3.aapm.org/meetings/2020AM/programInfo/programAbs.php?sid=8797&aid=51830},

year = {2020},

date = {2020-01-01},

journal = {2020 Joint AAPM textbar COMP Virtual Meeting},

abstract = {Purpose:

To automate the classification and segmentation of tumor cells in images of biopsy slides using deep learning to minimize manual labor, the time required, and human error. The segmented tumor cells and nuclei will be used for patient-specific microdosimetry studies.

Methods:

A pathologist manually contoured images of 57 pathology core biopsies in TIFF format, each containing 3750x3750 pixels with a 248 nm per pixel resolution on a pixel by pixel basis. The contoured pixels were used as the ground truth for a three-dimensional deep convolutional neural network model based on a UNet architecture using Keras and Tensorflow. Forty-eight of the core images were used to train the model with data augmentation using binary cross-entropy as the loss function on a 120 GB GPU cluster for 12 hours. The remaining nine core images were used for testing. Testing was done by applying a 50% confidence threshold on the model’s prediction and comparing the results with the manual contours.

Results:

The average time for the pathologist to contour a core image was 20 minutes. The model was able to segment three images per minute with an accuracy of 90.9%, specificity of 91.2%, sensitivity of 90.0%, precision of 73.0%, and a dice coefficient of 80.6%. The model’s predictions were visually similar to the manual segmentation. The model’s predictions were more confident about the center of the tumor regions than the edges.

Conclusion:

The proposed model can closely and consistently replicate tumor cell contours made by a pathologist 60 times faster than manual contouring. It can autonomously and efficiently generate large amounts of contoured pathology data that can be used for further research, such as microdosimetry performed on patient-specific tumor nuclei and cells. Future studies will investigate the accuracy and consistency of the manually contoured data, which was used as the ground truth.},

note = {Type: Journal Article},

keywords = {},

pubstate = {published},

tppubtype = {article}

}