Fireworks AI told customers their models were safe. Its second breach reached them

On October 10, Fireworks AI told its customers the thing every breached company wants to say. It had found no evidence that anyone touched customer models, datasets, training data, prompts, outputs or personal information. The damage, as it stood, was a pile of stolen secrets: API keys and tokens that customers had stored in their accounts for evaluation jobs.

On November 9 the company published an update. The bad news opened with one flat line: "Since then, we discovered impacts to account and billing information." Further down, it said that for "a limited set of accounts" the intruders had also reached secrets, datasets and incomplete model weights.

For an AI company, that is about the worst sentence you can write.

Two doors, one month

Fireworks sells inference. Companies hand it a model, open source or fine-tuned on their own data, and Fireworks runs it fast on GPUs it manages. Bloomberg's headline on the news calls it Nvidia-backed, and RuntimeWire notes it was co-founded by its chief executive, Lin Qiao. The whole pitch rests on customers trusting it with their models and their data.

The first break-in surfaced at 9am Pacific on October 6, when Fireworks spotted what it called "unauthorized use of internal Fireworks credentials". Over the next four days it notified customers, rotated employee API keys, revoked user keys and rotated service-account tokens. Its bulletin carried a warning that anyone who has cleaned up after a credential leak will recognise: "Removing a stored secret alone does not invalidate the underlying credential." Deleting a key from your Fireworks settings did nothing if the attacker had already copied it. Customers had to go and kill each key wherever it was issued.

Then a second door opened. According to the November bulletin, a third-party cloud provider that hosted Fireworks research and development machines had its own security incident on October 23. The next day, the attacker used that access to get onto Fireworks development machines and ran an unauthorized workload there, one the company says "led to additional customer data impact." Fireworks says it caught that the same day. RuntimeWire, summarising Bloomberg's reporting, called it an October 24 intrusion "through a cloud provider." Fireworks files the October 24 activity under the same incident. Bloomberg called it the company's second hack in weeks. Both descriptions fit the facts.

Who wants a stolen model

Fireworks says the evidence "indicates links to a cybercriminal threat actor group, which is financially motivated," and that the group's business is selling tokens from unauthorized access to large language models through illicit routing gateways. The industry has a name for this: LLMjacking. Google's Threat Intelligence Group and Mandiant published research on it on November 2, and the Fireworks bulletin links to that work.

The economics are simple. Running big models costs money for every token. Steal someone's API key and you can run those models on their bill. Steal a lot of keys and you can set up a proxy, sell cheap access to anyone who asks, and let the victims pay the compute invoice. The buyers get discount AI with no usage policy attached. The victims find out when the bill arrives.

Seen that way, the first phase of the Fireworks breach makes sense. Secrets stored for evaluation jobs can include keys for other services, so one compromised platform can leak access to several more.

The second phase is harder to read. Billing addresses, invoice histories and credit balances tell an attacker which accounts are big and worth another look. Datasets and model weights are a different kind of loot. A company's fine-tuning dataset is often the most proprietary thing it owns: support transcripts, internal documents, labelled examples that took months to build. Weights are the trained model itself. Fireworks says the weights accessed were incomplete, which suggests checkpoints from unfinished training runs. It has not said how incomplete, how many accounts sit in the "limited set," or whether anything was copied out.

Fireworks did draw some boundaries. "We have not detected any impact to Inference, Training SDK and Nexus," the update says, adding that those products keep no customer data by default. "Payment card numbers are not stored by Fireworks and were not impacted." The company says it has seen no further successful unauthorized access to the customer platform since October 25.

The list of fixes reads like a list of things that were looser than they should have been. Fireworks says it replaced credentials and service accounts, cut unnecessary privileges, restricted network paths, tightened access to data storage and improved logging. It has also switched off self-serve dataset uploads and downloads for Batch Inference and dataset-based training. Palo Alto Networks' Unit 42 is doing the forensics, and law enforcement has been told.

The October sentence

What will stick with customers is the gap between the two bulletins. In October, the company's best reading of the evidence was that their models and data were untouched. That reading lasted about two weeks, until an outside provider's breach opened a path into machines where customer datasets and training checkpoints apparently sat.

The October bulletin said: "We recognize the concern and work this creates for affected customers." The November one created a lot more. Any company that sent Fireworks a dataset now has to ask what was in it, who else might have it, and since when. The answers are in Fireworks' private notices to the affected accounts, and nobody else gets to read them.

Sources: Fireworks AI security bulletin, November update, Fireworks AI security bulletin, October, RuntimeWire, The Defiant.