Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More
The National Institute of Standards and Technology (NIST) is facing an internal crisis as staff members and scientists have threatened to resign over the anticipated appointment of Paul Christiano to a crucial, though non-political, position at the agency’s newly-formed US AI Safety Institute (AISI), according to at least two sources with direct knowledge of the situation, who asked to remain anonymous.
NIST is an agency of the US Department of Commerce whose mission is “to promote US innovation and industrial competitiveness by advancing measurement science, standards, and technology in ways that enhance economic security and improve our quality of life.” According to the agency’s website, its core competencies are “measurement science,” “rigorous traceability” and “development and use of standards.” NIST also develops cybersecurity standards, guidelines and best practices, and released an AI security framework in January 2023.
Christiano, who is known for his ties to the effective altruism (EA) movement and its offshoot, longtermism (a view that prioritizes the long-term future of humanity, popularized by philosopher William MacAskill), was allegedly rushed through the hiring process without anyone knowing until today, one of the sources said.
The appointment of Christiano, which was said to come directly from Secretary of Commerce Gina Raimondo, has sparked outrage among NIST employees who fear that Christiano’s association with EA and longtermism could compromise the institute’s objectivity and integrity.
However, Divyansh Kaushik, associate director for emerging technologies and national security at the Federation of American Scientists, told VentureBeat that President Biden’s AI Executive Order, introduced in November 2023, specifically asks NIST and the AISI to focus on certain tasks — including CBRN (chemical, biological, radiological and nuclear materials) — for which Paul Christiano is “extremely qualified.”
Many say EA — defined by the Center for Effective Altruism as an “intellectual project using evidence and reason to figure out how to benefit others as much as possible” — has turned into a cult-like group of highly influential and wealthy adherents (made famous by FTX founder and jailbird Sam Bankman-Fried) whose paramount concern revolves around preventing a future AI catastrophe from destroying humanity. Critics of the EA focus on this existential risk, or “x-risk,” say it is happening to the detriment of a necessary focus on current, measurable AI risks — including bias, misinformation, high-risk applications and traditional cybersecurity.
The US AI Safety Institute was established in November 2023
The AISI was established in November 2023 to “support the responsibilities assigned to the Department of Commerce” under the AI Executive Order. Earlier today, US Senate Majority Leader Chuck Schumer (D-NY) announced that the NIST will receive up to $10 million to establish the US AI Safety Institute.
Last month, VentureBeat reported on criticism about NIST’s lack of transparency around the AISI: In mid-December, House Science Committee lawmakers from both parties sent a letter to NIST that Politico reported “chastised the agency for a lack of transparency and for failing to announce a competitive process for planned research grants related to the new U.S. AI Safety Institute.” That lack of transparency centered around a potential grant to RAND Corporation, “an influential think tank tied to tech billionaires, the AI industry and effective altruism.” (VentureBeat has also reported about a “widening web” of effective altruism adherents in AI ‘safety’ and security circles, including within RAND and leading LLM model company Anthropic).
‘Important to ensure that NIST’s mission is not affected’
Kaushik told VentureBeat that regardless of who is appointed to the AISI role, “it is important to ensure that NIST’s mission is not affected by it.”
NIST has always done “methodical measurement research that is highly grounded,” he said. “We’re in uncharted territory with trying to establish an evaluation program for hypothetical risks from general purpose models. I totally understand the apprehension NIST scientists would be having and it’s on Commerce leadership to ensure that any appointments stay true to science and true to NIST’s mission and methodology.”
If certain appointments cause NIST’s “superstar scientists” to quit, he added, “that would be a shame — in that case, I’d certainly hope the Secretary looks at what she will lose by appointing one person and reconsider her choice.”
Congressional committees, he pointed out, are already paying close attention to AISI’s work. “I hope they’re able to do good work with good people who are the best measurement scientists and can do this work in a way that leans on NIST’s strengths rather than trying to remake what NIST’s mission is and what it does,” he said.
That said, in a post on X Kaushik wrote that “If they’re able to hire someone quickly, that’s great! It should be celebrated not frowned upon. In fact the EO’s aggressive timelines require that to happen.”
VentureBeat has reached out to Secretary of Commerce Gina Raimondo, AISI director Elizabeth Kelly, and Paul Christiano. We will update if and when we hear back.
