At the AI Infra Summit, European AI chip startup Euclyd emerged from stealth mode with some details of an ambitious hardware architecture which the company said will offer lower power and lower cost per token than existing solutions.

Democratizing the technology for the world is part of the startup’s ambition, Ingolf Held, co-founder and vice president of product at Euclyd, told EE Times.

“After significant brainstorming, we decided we wanted to do something for data center inference at massive scale, but we had a few ground rules,” Held said. “Power is the ultimate operating cost, so the highest efficiency should provide the lowest operational cost, which would allow us to roll this out everywhere, not just in the US, to a hyperscaler that has billions of dollars.”

tarting with a blank slate allowed the fledgling company to consider the problem from first principles, Held said, offering the opportunity to tackle compute, memory bandwidth, and memory capacity all at once.

Euclyd’s proposed silicon, a huge many-chiplet SiP design it calls Craftwerk, will incorporate 16,384 SIMD processors and deliver up to 8 PFLOPS (FP16) or 32 PFLOPS (FP4). These processing elements will be designed from the ground up by Euclyd. The device will use the largest possible silicon interposer (around 100 by 100 mm) with 2.5D and 3D elements, Held said.

“We will build it ourselves – we won’t inherit anything from Arm or RISC-V, and it’s going to be fully programmable with in-house programming tools,” he said. “It will support today’s transformers, but it isn’t limited to that.”

The design will maintain programmability to ensure it can accelerate whatever the future holds, Held said, whether that’s multi-modal inference, reasoning, recurrent models, state space models or diffusion models.

Euclyd’s custom memory and performance

Euclyd will pair compute chiplets with a custom memory design it calls Ultra Bandwidth Memory (UBM). UBM will enable 1 TB of DRAM with 8000 TB/s bandwidth in a Craftwerk SiP.

SRAM is fast, but AI accelerators on the market that rely on it are forced to shard models across many pieces of silicon, Held said.

“This is problematic because it requires significant investment, lights up a lot of silicon, but it also forces you to have all this connectivity to scale inside the rack or across racks, and that’s a big problem,” he said. “Our idea is to make this as dense as possible.”

HBM addresses SRAM’s capacity weakness, but its bandwidth – despite its name – is not high enough for what Euclyd wants to achieve. A custom design offers differentiation from competitors using “the same HBM,” Held said, noting that while Euclyd’s UBM is a custom design, it won’t require an exotic process technology.

Craftwerk’s size will enable multi-agent workflows on a single piece of silicon in a 3 kW TPD, Held said.

Nvidia’s DGX-B200 (eight current-gen Blackwell GPUs) can achieve 1038 tokens per second for a single user for Llama4-Maverick (400B), per figures released in May. Cerebras offers 2554 tokens per second for a single user, per current figures from Artificial Analysis. A single Craftwerk SiP will offer 20,000 tokens per second for a single user.

Euclyd’s rack will include 16 host CPUs plus 32 Craftwerks in a liquid cooled chassis with a 125 kW TDP; in a typical multi-user scenario this system will offer 7.68 million tokens per second for Llama4-Maverick, according to the company’s projections.

Seed funding

Three private investors are funding Euclyd so far: Peter Wennink (former CEO of ASML), Federico Faggin (inventor of the microprocessor and founder of Zilog and Synaptics), and Steven Schuurman (founder of Elastic). The company will start fundraising for a VC-backed round shortly to productize and scale, but its seed funding should be sufficient to demonstrate working silicon, Held said.

Euclyd founder and advisor Atul Sinha told EE Times that the company is seeing plenty of investment interest despite being based in Europe.

“Everyone we have spoken to in Europe has said this is exactly what they are waiting for,” Sinha said. “Our take is that Europe has been waiting for someone to say: let’s do this! Even investors in Silicon Valley have been waiting for someone who can really shoot performance up for a given footprint and power. Because they know this is the world’s problem.”

Sinha said that for chip design talent, Euclyd is better placed in Europe than in Silicon Valley. The company plans to remain Dutch, he confirmed, based in Eindhoven’s high-tech campus in The Netherlands, which is also home to NXP.

“What people don’t understand is that there are certain places in Europe where you do have a significant collection of technologies and manpower base,” Sinha said. “For semiconductors, Eindhoven is absolutely on the top of the list.”

“There’s no better place, I would argue,” he added.

By: DocMemory
Copyright © 2023 CST, Inc. All Rights Reserved