Since OpenAI introduced ChatGPT I’ve studied Large Language Models (LLMs) and tested their evolving capabilities. ChatGPT created Monica as an enhancement to search engines. Others have produced competitors: Gemini (Google’s AI), Claude (Anthropic) and Grok (xAI, Elon Musk’s contribution).
When I read about two women in Brazil today dying from the Oropouche virus, one I had never heard of, and that it was mosquito-borne and spreading beyond the Amazon Basin to other areas of South America, I became somewhat concerned.
Having just read about Perplexity.ai as an AI search assistant, I decided to test it on a general question about pandemic threats. I asked:
“What viruses represent future pandemic threats?”
This is what Perplexity wrote:
Based on the search results, several viruses represent potential future pandemic threats:
The World Health Organization (WHO) maintains a list of priority pathogens for research and development efforts, which includes these and other potential threats. It’s important to note that while these viruses are known concerns, the next pandemic could also be caused by a currently unknown pathogen. This is why the concept of “Disease X” is included in pandemic preparedness efforts. Factors contributing to increased pandemic risk include environmental changes, human encroachment on natural habitats, and increased global travel and trade. To prepare for future pandemics, ongoing surveillance, research into these viral families, and development of vaccines and treatments are crucial.
This is a pretty comprehensive response and expressed well. What I didn’t understand was since Perplexity.ai was a search assistant, why wouldn’t it mention Oropouche when a Google search did?
I thought a fair test would be to pose the same question to other AIs. I asked ChatGPT’s Monica and it came back with two additions to the Perplexity.ai list. The one I couldn’t understand Perplexity.ai missing was Dengue, described by Monica as “A mosquito-borne virus that can cause severe illness and has been spreading to new regions due to climate change.” The other was unfamiliar to me called Hendra and was described as “similar to Nipah, it is also transmitted from bats and can infect horses and humans, leading to severe respiratory and neurological symptoms.”
I then asked Google’s Gemini. Gemini’s response grouped viruses into two categories: zoonotic and RNA-based. It also included “Disease X” as an unknown. Then it added a category neither of the previous two had mentioned: the growing threat of AMR (anti-microbial resistance) as a pandemic threat.
Gemini wasn’t finished in its response. Even though I hadn’t asked, Gemini added information about root causes including:
In fairness to Perplexity and Monica, both included suggested follow-up questions to provide root-cause answers.
I went through the same process with Claude. Claude had two more to add to the list: Filoviruses, a group of viruses similar to Ebola, and Henipaviruses, a group of bat-borne viruses related to Nipah. Claude then asked me if I wanted it “to elaborate on any of these viruses or discuss pandemic preparedness strategies?”
My final pursuit in this exercise was to try and ask Grok the same question But with Grok I couldn’t because I needed to be an X Premium+ subscriber in the United States to use it.
Getting back to Perplexity.ai, it wasn’t alone in not mentioning Oropouche. But as a true search AI assistant, I couldn’t understand why it didn’t get mentioned.
I knew why the others had missed Oropouche. That’s because Monica, Gemini and Claude are AIs built with Large Language Model (LLM) algorithms that train with datasets of materials from the Internet that are often several months old.
To prove the difference, I went through a final exercise involving current events at the Paris Olympics. I asked Gemini and Claude who won medals today on August 5, 2024, and here are the two answers respectively:
Gemini: “There seems to be a slight confusion. The Paris Olympics are scheduled for July 26 to August 11, 2024. As of now, we are in the year 2023.” Whoops!!
Claude: “I apologize, but I don’t have access to real-time information about Olympic events…The most recent Olympic Games I have detailed information about were the 2022 Winter Olympics in Beijing. Sorry!
Ask Perplexity.ai the same question and here is its answer:
“Here are the key medal winners from the Paris Olympics on August 5, 2024:
As the old saying goes, “One of these is not like the others” and it is Perplexity.ai. It knows what’s happening in the present. Not mentioning the Oropouche virus was the only surprise to me. Maybe the explanation is that Oropouche wasn’t seen as a pandemic threat as of yet.
Ignoring the Oropouche omission, it is pretty clear to me that as AIs go, the current frontrunners whose LLMs train on datasets from the past will soon be obsolete. If I were them, in the light of Perplexity.ai, I would be panicking.
Let’s hope Oropouche is right in its omission and that the virus doesn’t join other vector-borne diseases to spread across the planet.
]]>