BEYOND THE SURFACE
Project Artist - Beyond the Surface
Mater Misercordiae Hospital and University College Dublin
Artificial Intelligence and Fluorescence Guided Surgery for Intraoperative Cancer Classification
I was brought on to the team in November 2021 to research a unit within the Mater Misercodiae Hospital, Dublin in conjunction with University College Dublin. My role was to create a piece of art in response to the groundbreaking use of Ai as a decision support system for surgeons during complex cancer operations.
As with all these projects, thus started a period of research before any creative elements were discussed.
Research and findings:
The system uses field-leading computer vision analytics, with novel biophysics inspired algorithms, to augment in real time human interpretation of tissue characteristics during operations.
It uses advanced near-infrared optical visualisation technology
with a ‘phase contrast’ approved fluorescent dye
It brings together experts in surgery, cancer biology, chemistry, optical physics, maths, computer science
All with the aim of better patient care.
THE BIG QUESTION - is this cancer or not?
To be able to answer this immediately, intra-surgery, without additional tests such as pathology and radiology would be great advance
The use of fluorescence within cancer detection is not a new technique, it has been used to reveal lesions’ biological nature. This is done through monitoring the way fluorescence moves through the lesion in comparison with other areas of normal healthy tissue. Blood flow moves much slower through cancerous areas, therefore the fluorescence will sit in a cancer longer than healthy tissue. Making it a literal beacon to surgeons where there is something not quite right.
BUT differences are subtle, happen rapidly, happen transiently, and are hard to study in all areas at once solely using observation.
Therefore by using software, the computer does the watching,.
motion stabilized tracking
quantification of dye kinetics
in different areas, at the same time, and over time
no human error, such as distraction
converts visual information into numbers, which are displayed as graphs
MOST IMPORTANTLY immediately mathematically analyses
Thus, it provides a ‘probabilistic determination of the legion’s nature’ allowing the continued flow of patient care.
IN THE THEATRE:
Data scientist is present
Surgical team identify area of abnormality
Boxes are drawn on the screen to track the changes with fluorescence
Anaesthetist gives dye
Within seconds the fluorescence signal enters the tissues
Computer determines the leison’s nature
After a couple minutes there is sufficient data to diagnose cancerous tissue
The team have been using fluorescence to teach an Ai to track the blood flow, and to give the results from biopsies, so that the Ai can determine what is health tissue and what is not.
Over time this will allow for the Ai to be as accurate as biopsies, without the wait time. A digital-biopsy. Just think about it… At the moment a patient will undergo fluorescence surgery so that surgeons can take a biopsy of the area of concern, send it off to the labs, receive the results, discuss potential treatment, potentially book the patient back in for surgery, undergo surgery, and then recover from surgery… all of which would take months. This new technique would allow for surgeons to be able to assess the concerning area, diagnose on the spot, and with pre-received permission, remove the cancerous tissue there and then. You are cutting down the process by months. Allowing faster treatments, less time for cancers to develop, faster recovery, and better patient care.
Similarly, the Ai will be able to detect the line between healthy and cancerous tissue down to the finest degree. This will enable surgeons to save as much healthy tissue as possible.
Therefore, what it could do?
Cut here lines
Greater chance of saving non-cancerous tissue and removing cancerous tissue
Better understanding of cancer and how it behaves
After completing my research, I realised that the in-real-time aspect of the fluorescence and
it’s movements was crucial in creating the work. I wanted to create a painting of an ambiguous
area of human tissue, with a projection over the top mimicking the fluorescence. As I am not a
digital artist myself I went looking for one, and with the help from the Mater, found Méabh
Hennelly, an Irish artist, who came on to collaborate with the project.
Bringing Méabh up to date with the research and visiting the Mater to view one of the surgeries
left us with a lot to discuss. I myself have a lot of experience in surgery, and have therefore
become desensitised to a lot of the ‘graphic’ aspects of it. With Méabh on board, we were able to
discuss what would be a happy medium, that explores the surgery but appeals to both surgeons
and the general public.
The collaborative project was a brilliant experience as the end result was not the mimicked
fluorescence that I had envisioned, nor did the painting turn out how I had originally planned it.
Working remotely from Ireland and Scotland, Méabh and I bounced projections and photos of the
painting back and forth, each adjusting our own in response to what the other had done. My
painting changed from being in colour to black and white, as the lack of colour made it less graphic
and also played with the projections better. Méabh’s projections became something more
meditative, more beautiful, complex and interesting than the fluorescence itself. The shapes and the
contrast within the painting changed to enhance these. Méabh’s projected videos were built around
a circular design, but more were developed to fill the square of the painting. This back and forth
meant the piece was constantly shifting, constantly developing, into the end result which was more
than either of us had dared hope it would be.