Scientists have successfully employed artificial intelligence to construct entirely new viruses, a breakthrough that signals the potential for AI to engineer novel forms of life.
The recently identified viruses display enough variation from known strains to potentially be categorized as new species. These entities are bacteriophages, meaning they exclusively target bacteria and pose no threat to humans. Researchers behind the study took deliberate measures to guarantee their models could not produce viruses capable of infecting people, animals, or plants.
This week, a new study by a different group of scientists highlighted artificial intelligence’s troubling ability to readily circumvent existing safety measures put in place to prevent the development of bioweapons.
Microsoft researchers have demonstrated that artificial intelligence is capable of bypassing security measures intended to stop malicious individuals from obtaining dangerous materials, like toxic molecules, from supply companies. This discovery was detailed in a study published on Thursday, October 2nd, in the journal Science.
Following the identification of a critical vulnerability, a research team promptly developed software patches designed to substantially mitigate the associated risks. However, the deployment of these protective updates currently requires advanced technical expertise and access to specialized tools, resources that remain largely unavailable to the general public.
Emerging research collectively highlights a significant future risk: the potential for artificial intelligence to engineer new, dangerous lifeforms or bioweapons. In the most severe circumstances, such a development could trigger a global pandemic. While AI systems currently lack the sophisticated capability to design these threats, experts caution that this scenario may become a reality in the not-so-distant future.
Experts advocate for the implementation of robust, multi-layered safety systems to avert potential dangers posed by artificial intelligence. Such comprehensive frameworks would include enhanced screening tools and adaptable regulations specifically tailored to govern AI-driven biological synthesis.
A central concern with AI-developed biological innovations, such as synthetic viruses and proteins, is the “dual-use problem.” This critical issue characterizes any technology or research capable of offering significant benefits while simultaneously carrying the potential for intentional misuse to cause harm.
Scientific advancements often present a double-edged sword, where knowledge gained for positive outcomes can be perverted for destructive ends. A researcher studying infectious diseases, for example, might genetically alter a virus to uncover mechanisms of increased transmissibility, aiming to develop countermeasures. Yet, this very insight could be exploited to engineer a more potent pathogen, sparking a pandemic. Similarly, innovations in aerosol drug delivery, crucial for developing more effective inhalers for asthma sufferers, could also be adapted for the dissemination of chemical weapons.
Stanford University researchers Sam King, a doctoral student, and his faculty advisor, Assistant Professor of Chemical Engineering Brian Hie, recognized a significant challenge in developing new antimicrobial agents. Their objective was to engineer novel bacteriophages, often called phages, designed to specifically seek out and destroy bacteria within infected patients. The details of their work were released in September as a preprint on the bioRxiv database, and the findings have not yet undergone peer review.
Bacteriophages, viruses that specifically target and destroy bacteria, are currently under evaluation by scientists as a promising new tool in combating infectious diseases. Sourced from natural environments and cultivated in labs, these phages are being tested as potential additions or alternatives to conventional antibiotics. The goal is to address the critical challenge of antibiotic resistance and ultimately save lives. Nevertheless, a theoretical risk accompanies this research: given that phages are viruses and some viruses are pathogenic to humans, there is a possibility that development efforts could inadvertently yield a phage capable of causing harm to people.
Recognizing a potential risk, the research team proactively implemented measures to reduce it. Crucially, their AI models were not trained on viruses that infect humans or any other eukaryotic organisms—a biological domain encompassing plants, animals, and all life forms distinct from bacteria and archaea. Additionally, the models underwent rigorous testing to confirm they could not independently develop viruses similar to those known to infect plants or animals.
Operating under strict safeguards, an artificial intelligence system was directed to generate designs inspired by a bacteriophage commonly utilized in laboratory research. However, King noted that anyone intending to create a deadly virus would likely find older, more conventional methods to be an easier approach.
The method remains significantly challenging, demanding considerable expertise and time, King told Live Science. He indicated that this inherent difficulty currently acts as a safeguard, preventing its application for more dangerous uses.
The rapidly evolving landscape of AI-enabled biology necessitates the on-the-fly development of safety protocols, leaving definitive standards yet to be established. Researchers warn that future regulations must meticulously balance the inherent risks of AI applications in biology against their profound benefits. A critical challenge further lies in anticipating how these advanced AI models could potentially bypass or circumvent implemented safeguards.
AI models demonstrate significant intelligence, a point underscored by Tina Hernandez-Boussard, a professor of medicine at the Stanford University School of Medicine who consulted on safety for the models’ viral sequence benchmarks used in a recent preprint study. Hernandez-Boussard cautioned that these models are fundamentally engineered for peak performance. This design priority, she explained, means that once trained with data, they possess the capacity to override existing safeguards.
Meticulous curation of AI training data stands as a foundational step to avert future security challenges, an expert highlighted. Illustratively, in a recent phage study, researchers intentionally excluded data on viruses known to infect eukaryotes from their model. They further conducted tests to guarantee the models could not independently generate genetic sequences that would render their bacteriophages dangerous to humans. These assessments confirmed the models did not develop such a capacity.
The scope of AI safety extends to the critical process of translating artificial intelligence designs—essentially genetic blueprints—into tangible biological products such as proteins or viruses. While many leading biotech supply companies currently deploy software to ensure customers do not acquire toxic molecules, the implementation of this vital screening remains voluntary within the industry.
A recent study by Microsoft’s chief science officer Eric Horvitz and senior applied scientist Bruce Wittman has revealed a critical vulnerability in current genetic screening software: it can be circumvented by AI-designed genetic sequences. The existing programs operate by comparing novel genetic data to known sequences associated with the production of toxic proteins. However, artificial intelligence is capable of generating significantly different genetic sequences that nonetheless code for the identical toxic functions. As a result, these AI-generated sequences may not trigger the necessary alerts in the present screening systems.
Leveraging a cybersecurity-inspired approach, researchers identified a critical software vulnerability and promptly alerted trusted experts and professional organizations. This led to a collaborative initiative focused on developing necessary software fixes. According to Horvitz, speaking at a September 30 press conference, these patches were subsequently deployed globally months later, significantly enhancing biosecurity screening measures.
While security patches significantly reduced risks, an average of 3% of potentially dangerous gene sequences still eluded detection across four commonly used screening tools, Horvitz and colleagues reported. This presented a unique security challenge for the researchers, particularly concerning the publication of their findings. The fundamental scientific principle of replicability demands that papers provide enough data for other researchers to verify results. However, the team faced a paradox: publicly releasing all data regarding sequences and software could inadvertently offer malicious actors the precise information needed to bypass the very security patches designed to protect against such threats.
Horvitz reported a distinct unease among peer reviewers, who appeared to grapple with uncertainty regarding the correct methodology for their tasks.
A multi-tiered access system has been established, requiring researchers seeking sensitive data to apply to the International Biosecurity and Biosafety Initiative for Science (IBBIS). This organization will operate as a neutral third party, evaluating all requests for access. Microsoft has funded both this evaluation service and the data hosting through a dedicated endowment.
A prominent science journal has, for the first time, formally endorsed a specific data-sharing methodology. Tessa Alexanian, technical lead at Common Mechanism, described this “managed access program” as an experiment. She emphasized the team’s keen interest in “evolv[ing] our approach” as the program develops.
Regulatory frameworks governing artificial intelligence tools are currently limited, with safety protocols like those recently examined in the journal *Science* remaining largely voluntary. This absence of mandatory oversight creates a vulnerability: individuals with malicious intent could potentially utilize AI to design hazardous molecules and then produce them in a laboratory setting, effectively bypassing any external scrutiny or gatekeepers.
Biosecurity is increasingly benefiting from comprehensive guidance issued by both professional consortia and government bodies. A prime example is a 2023 U.S. presidential executive order, which places a strong emphasis on safety. This directive calls for “robust, reliable, repeatable, and standardized evaluations of AI systems,” alongside the development of policies and institutions aimed at mitigating potential risks. Furthermore, according to Diggans, the Trump Administration is actively working on a framework designed to limit federal research and development funding for companies that do not perform adequate safety screenings.
According to Alexanian, policymakers are demonstrating a growing willingness to implement incentives for screening.
The United Kingdom’s state-backed AI Security Institute is actively working to establish policies and standards aimed at mitigating the inherent risks of artificial intelligence. The organization funds crucial research projects focused on enhancing AI safety and risk reduction. These initiatives include developing defenses against the misuse of AI systems, protecting against external threats like the injection of corrupted data into AI training processes, and exploring methods to prevent public, open-use AI models from being exploited for malicious purposes.
The increasing complexity of AI-designed genetic sequences offers a significant advantage for safety protocols, providing screening tools with a more extensive dataset to analyze. This enhanced information simplifies the process of identifying potential dangers within comprehensive genetic constructs, making whole-genome designs—such as the bacteriophages engineered by King and Hie—relatively straightforward to assess for inherent risks.
Diggans highlighted that synthesis screening benefits significantly from a larger volume of data. This characteristic, he added, makes it an exceptionally potent and informative approach when applied across the entire genome.
Microsoft is partnering with government agencies to develop artificial intelligence systems aimed at identifying AI-related misconduct. One key application involves using AI to analyze extensive sewage and air-quality data, searching for evidence of the unauthorized manufacture of dangerous toxins, proteins, or viruses. Experts suggest this approach will expand screening beyond single sites of nucleic acid synthesis, broadening detection efforts across entire ecosystems.
Artificial intelligence theoretically holds the capacity to engineer entirely new genomes for species ranging from bacteria and archaea to more complex organisms. However, there is presently no straightforward laboratory method for AI to translate these digital instructions into living organisms, according to King. Consequently, while threats from AI-designed life are not immediate, their potential emergence in the future is considered a plausible, though not impossibly distant, prospect. Given the significant new frontiers AI is expected to unveil in the near future, Hernandez-Boussard stressed the vital importance of fostering creative solutions across various scientific fields.
A speaker highlighted the critical necessity for a broad, multidisciplinary community—encompassing funders, publishers, industry, and academics—to collectively mandate safety evaluations.
