Microsoft plans to deploy FPGA cards in its new servers to run its Bing search engine next year. The company might also use these cards as a standard part of the million servers in its data centres.

That's the goal for Andrew Putnam, part of a Microsoft Research that led to the current use of FPGAs for Bing. In talks at the annual Hot Chips event, Putnam and an engineer from China's search giant Baidu described their parallel efforts.

FPGAs are not a natural fit for cost-sensitive data centres, but it turns out they make sense. "We started out focusing on Bing which needed help because it was not fast enough, not that we wanted to get FPGAs in the data centre—we looked at software, then GPUs, and then FPGAs," said Putnam in an interview with press.

Putnam's team would up building a 10cm x10cm card using a Stratix V FPGA with 8GB memory. It slots into the standard server Microsoft's data centre uses, drawing 25W from its PCI Express Gen 3 slot.

The team also created special cabling to create a 6x8 Torus network to link all the FPGAs in a rack. It enables services run across multiple FPGAs and accessed without going through host server CPUs and their network.

In a pilot using 1,632 servers, the approach doubled throughput for Bing while adding less than 30 per cent of the cost of the server. "That means we could use half the number of servers to do Bing," said Putnam, a huge savings for cost-sensitive data centres.

That paved the way for use of the Catapult cards in Microsoft's new production servers running Bing, starting next year. "Retrofitting existing servers is really difficult and that's true of FPGAs or whatever," he said, noting the effort and risk of changing such massive underlying hardware.

The FPGAs accelerate the ranking of search terms, a new partition of the overall search job. "Bing and its 30,000 lines of gnarly code wasn't a natural fit for FPGAs," but his team has found other more natural Microsoft data centre apps for its cards.

"Getting FPGAs into the data centre is one thing but keeping them there is another," said Putnam.

FPGAs are getting there in terms of being more amenable for compute operations, but to keep FPGAs there we need to improve their amount of programmability. The FPGA tools are too slow, there are too many warnings and not great debugging—but that's not new.

We'd like to give developers a feeling of putting their C++ into compiler but [compilation] is a manual process now. There's a huge need for high productivity FPGA languages.

A custom multi-core processor that Putnam 's team designed and embedded into the FPGA helps. "We gave them a middle ground between C++ and RTL where people can program," he said.

The team's first prototype was a board using six Xilinx Virtex 6 chips, but the design was rejected by the data centre group for several reasons. The team's second prototype shifted to Altera, largely because the company provided better support for what started as a Microsoft research project.

"Altera was really interesting in helping with the development—the resources they were willing to throw our way were more significant than those from Xilinx which was willing to troubleshoot our design," he said.

Altera engineers got the Microsoft team's code up and running on Linux a PCIe 3.0 board within two days. "A day later they had a Windows driver—PCIe 3.0 has been a pain point for us," he said.

By: DocMemory
Copyright © 2023 CST, Inc. All Rights Reserved