What the US AI Safety Institute says still needs research
The US AI Safety Institute has named the research gaps it says still stand between current practice and effective mitigation of synthetic content risks.
What the AI Safety Institute says needs research
The US AI Safety Institute has published a call to action naming specific research gaps in mitigating synthetic content risk. The document identifies areas where current work does not yet provide adequate protection, and appeals to public and private funders, academic institutions, civil society and industry researchers to prioritise them.
The gaps fall into two categories: technical and social science.
Technical research gaps
The institute identifies four technical domains requiring frontier research.
- Digital watermarking robustness
- Content authentication mechanisms
- Detection of synthetic materials
- Safeguards preventing harmful model outputs
The call does not specify what makes current work in these areas insufficient, only that research is needed. It names them as gaps, implying that existing solutions are not yet adequate for the scale and nature of the risk.
Social science research gaps
The institute calls for social science research addressing systemic impacts of synthetic content. This includes three areas:
- How synthetic content affects information integrity and public trust
- Its effects in sensitive domains such as education and counselling
- Strategies to combat fraud and impersonation
These gaps reflect a concern that technical solutions alone are insufficient. The institute is calling for research into how synthetic content circulates, whom it harms, and how to respond socially and institutionally, not only technologically.
Why the institute is making this call
Synthetic content has legitimate uses. But the institute states that its widespread production risks damaging information integrity and enabling harmful activities including child exploitation, fraud, and intellectual property violations. It frames this as a frontier research challenge requiring coordinated effort across sectors.
Who the AI Safety Institute is asking to close the gaps
The institute appeals explicitly to all stakeholders: public and private funders as well as academic, civil society and industry scholars. This is framed as a collective responsibility rather than a problem for any single sector to solve.
The document is a statement that the research gaps it names are not yet closed by existing industry procurement, academic output, or government work. Whether a reader has been told detection is a solved problem, the institute's own assessment is that frontier research remains necessary.
- What Content Credentials establish, and what they do not , the provenance limits named in that gap list, in full
- Detector results on deepfakes found in the wild , the detection side of the same problem, measured
- What invisible watermarks can and cannot survive , the watermarking gap on that list, with the attack research behind it
Sources
- [1]US AI Safety Institute. Frontier Research on Mitigating Risks from Synthetic Content: A Call to Action.
Common questions on this topic
What technical research gaps does the US AI Safety Institute identify?
The institute identifies four technical research gaps: digital watermarking robustness, content authentication mechanisms, detection of synthetic materials, and safeguards preventing harmful model outputs. [1]
What social science research does the institute say is needed?
The institute calls for social science research on how synthetic content affects information integrity and public trust, its effects in sensitive domains such as education and counselling, and strategies to combat fraud and impersonation. [1]
Who is the call to action directed at?
The institute calls on all stakeholders to support this research: public and private funders as well as academic, civil society and industry scholars. [1]
What risks from synthetic content does the institute cite?
The institute states that widespread synthetic content production risks damaging information integrity and enabling harmful activities including child exploitation, fraud, and intellectual property violations. [1]