Project Artist - Beyond the Surface 

Mater Misercordiae Hospital and University College Dublin 

2021-2023

​

Artificial Intelligence and Fluorescence Guided Surgery for Intraoperative Cancer Classification

 

I was brought on to the team in November 2021 to research a unit within the Mater Misercodiae Hospital, Dublin in conjunction with University College Dublin. My role was to create a piece of art in response to the groundbreaking use of Ai as a decision support system for surgeons during complex cancer operations.

As with all these projects, thus started a period of research before any creative elements were discussed.

 

 

Research and findings:

The system uses field-leading computer vision analytics, with novel biophysics inspired algorithms, to augment in real time human interpretation of tissue characteristics during operations.

 

It uses advanced near-infrared optical visualisation technology

with a ‘phase contrast’ approved fluorescent dye

It brings together experts in surgery, cancer biology, chemistry, optical physics, maths, computer science

All with the aim of better patient care.

THE BIG QUESTION - is this cancer or not?

To be able to answer this immediately, intra-surgery, without additional tests such as pathology and radiology would be great advance

 

The use of fluorescence within cancer detection is not a new technique, it has been used to reveal lesions’ biological nature. This is done through monitoring the way fluorescence moves through the lesion in comparison with other areas of normal healthy tissue. Blood flow moves much slower through cancerous areas, therefore the fluorescence will sit in a cancer longer than healthy tissue. Making it a literal beacon to surgeons where there is something not quite right.

 

BUT differences are subtle, happen rapidly, happen transiently, and are hard to study in all areas at once solely using observation.

 

Therefore by using software, the computer does the watching,.

• motion stabilized tracking

• quantification of dye kinetics

• in different areas, at the same time, and over time

• no human error, such as distraction

• converts visual information into numbers, which are displayed as graphs

• MOST IMPORTANTLY immediately mathematically analyses

 

Thus, it provides a ‘probabilistic determination of the legion’s nature’ allowing the continued flow of patient care.

Efficiency!

 

IN THE THEATRE:

• Data scientist is present

• Surgical team identify area of abnormality

• Boxes are drawn on the screen to track the changes with fluorescence

• Anaesthetist gives dye

• Within seconds the fluorescence signal enters the tissues

• Computer determines the leison’s nature

• After a couple minutes there is sufficient data to diagnose cancerous tissue

 

 

Further applications:

The team have been using fluorescence to teach an Ai to track the blood flow, and to give the results from biopsies, so that the Ai can determine what is health tissue and what is not.

 

Over time this will allow for the Ai to be as accurate as biopsies, without the wait time. A digital-biopsy.  Just think about it… At the moment a patient will undergo fluorescence surgery so that surgeons can take a biopsy of the area of concern, send it off to the labs, receive the results, discuss potential treatment, potentially book the patient back in for surgery, undergo surgery, and then recover from surgery… all of which would take months. This new technique would allow for surgeons to be able to assess the concerning area, diagnose on the spot, and with pre-received permission, remove the cancerous tissue there and then. You are cutting down the process by months. Allowing faster treatments, less time for cancers to develop, faster recovery, and better patient care.

 

Similarly, the Ai will be able to detect the line between healthy and cancerous tissue down to the finest degree. This will enable surgeons to save as much healthy tissue as possible.

 

Therefore, what it could do?

• Digital biopsy

• Cut here lines

• Greater accuracy

• Greater chance of saving non-cancerous tissue and removing cancerous tissue

• Better understanding of cancer and how it behaves

 

The Art:

After completing my research, I realised that the in-real-time aspect of the fluorescence and

it’s movements was crucial in creating the work.  I wanted to create a painting of an ambiguous

area of human tissue, with a projection over the top mimicking the fluorescence. As I am not a

digital artist myself I went looking for one, and with the help from the Mater, found Méabh

Hennelly, an Irish artist, who came on to collaborate with the project.

 

Bringing Méabh up to date with the research and visiting the Mater to view one of the surgeries

left us with a lot to discuss. I myself have a lot of experience in surgery, and have therefore

become desensitised to a lot of the ‘graphic’ aspects of it. With Méabh on board, we were able to

discuss what would be a happy medium, that explores the surgery but appeals to both surgeons

and the general public.

 

The collaborative project was a brilliant experience as the end result was not the mimicked

fluorescence that I had envisioned, nor did the painting turn out how I had originally planned it.

 

Working remotely from Ireland and Scotland, Méabh and I bounced projections and photos of the

painting back and forth, each adjusting our own in response to what the other had done. My

painting changed from being in colour to black and white, as the lack of colour made it less graphic

and also played with the projections better. Méabh’s projections became something more

meditative, more beautiful, complex and interesting than the fluorescence itself. The shapes and the

contrast within the painting changed to enhance these. Méabh’s projected videos were built around

a circular design, but more were developed to fill the square of the painting. This back and forth

meant the piece was constantly shifting, constantly developing, into the end result which was more

than either of us had dared hope it would be.

 

 

The Exhibition: