As artificial intelligence platforms (AI) continue to increase in popularity, developing a special language that doctors can use to integrate platforms such as ChatGPT to work would be a worthwhile effort, according to Jharna M. Patel, MD.
During the 2024 SGO Annual Conference on Women’s CancerPatel presented findings from a study that examined the use of ChatGPT to answer genetic counseling questions for patients with gynecological cancer. Studies have shown that regarding the 20 genetic counseling questions, the evaluations of the gynecologists gave all the answers of ChatGPT points 1, which means that the answers were many and correct. In addition, 67% of the answers to the 20 questions about Lynch syndrome and the topics of hereditary breast cancer and ovarian disease received a score of 1. An additional 33% received a score of 2, which means that the answer is correct but incomplete.
“ChatGPT is a powerful tool, but it has its limitations. Much work is needed in this field before it can be used in the field of gynecologic oncology,” Patel said in an interview with OncLive®. “However, I hope this [study] it opens the door for us to think about this powerful tool that can be used as a patient tool.
In the interview, Patel discussed the rationale for using ChatGPT to answer genetic counseling questions in the field of gynecologic oncology, expanded on the findings of this study, and detailed the next steps in researching the platforms. of AI for potential use in clinical practice.
Patel is a gynecologic oncologist at New York University Langone Health’s Perlmutter Cancer Center.
OncLive: What were the reasons for researching the use of ChatGPT for genetic counseling in patients with gynecological cancer?
Patel: ChatGPT is being widely used by everyone, including our patients. ChatGPT has been subject to extensive research and has been found to have passed the United States Medical Licensing Examination. [USMLE]. Therefore, we wanted to see clearly what its application in gynecologic oncology could be.
We know that [ChatGPT] able to answer questions based on facts; however, we wanted to test its ability to answer various questions. We believe that genetic counseling is the team that should be tested for these specific questions.
What were some of the methods and designs of this study?
We created a list of 40 questions along with gynecologists and polls from professional community websites about the most common questions related to genetic counseling in gynecology. A list of questions [was] then he is divided into two parts. The first group was related to genetic counseling questions in general, and these were questions like, Will my insurance be affected if I find out that BRCA syndrome?’ The second question was related to questions related to death, such as Lynch syndrome and BRCA– questions related to breast cancer and ovarian cancer.
Questions were asked on ChatGPT. We used an eighth-grade reading level to model the average user. The questions were scored by 2 gynecologic oncologists on a scale of 1 to 4. A score of 1 would mean that the answer was completely accurate and comprehensive, and in clinical practice, the gynecologic oncologist was There will be nothing important to add to that answer. . A score of 4 would indicate a completely incorrect answer. If there was a discrepancy between the 2 reviewers, additional gynecologic oncologists were asked to provide questionnaires to resolve any discrepancies.
What were some of the key findings from this study, and did anything surprise you?
The most surprising finding was in a group of general genetic counseling questions. ChatGPT replied [those questions] with 100% sensitivity and accuracy. There were no statistical differences. All of the gynecologists we asked to provide feedback on the questionnaire agreed that the answers were comprehensive and accurate.
Another thing that was found is that in the specific questions of this disease, especially Lynch syndrome and BRCA-associated syndromes, although ChatGPT was positive [with a score of 1] 67% of the time, it gave the correct answers [that] they were incomprehensible [a score of 2], and the attendees thought there would be something more to add to that answer. It was really surprising and interesting for both sides.
Were there any notable deficiencies in this study?
There were limitations in our study. This was a scoring system that adhered to previously published literature, meaning our study was free of any potential scoring bias. We attempted to minimize this bias by ensuring that each assessor was blinded to the responses of the other assessors.
Another problem was our direct questions about Lynch’s death. We only included 3 questions in our study that specifically related to Lynch syndrome, which limited our ability to draw accurate conclusions about that group. The addition of Lynch syndrome-specific questions would allow us to compare that group of questions directly with breast and ovarian cancer genetic questions.
Are there any next steps for this research?
The next steps of [this] the research is really interesting. AI pipelines are being developed all over the place, and it would be interesting to see if a similar model of AI language could be developed with input from the doctor to help see if it produces more answers than the doctor can. which could provide it in their medical practice. Now [we could see] if that really influences the uptake of genetic testing in our patients.
Reference
Patel JM, Hermann CE, Growdon WB, Aviki E, Stasenko M. ChatGPT effectively performs genetic counseling for gynecologic cancer. Presented at: 2024 SGO Annual Meeting on Women’s Cancer; March 16-18, 2024; San Diego, CA.
#ChatGPT #Shows #Potential #Gene #Therapy #Gynecologic #Oncology
