Message Font: Serif | Sans-Serif
No. of Recommendations: 3
I was discussing data center applications. (...) In data centers where they already have beefy server CPU anyway...

"They" would love to *not* have a beefy CPU. Google is getting rid of those beefy CPUs by making their TPUs run without one. Reason is that they are expensive, consume a lot of power (for little return when paired with an accelerator) and consume space.

So you and I agree that the bottleneck is not in processing power and efficiency.

We do not. I merely observed that bandwidth from main memory to accelerator is not the bottleneck. Again, look at Google's TPUs. Would they help if the bottleneck is elsewhere?

It is easy to conjure up a well-known example of where CPUs quickly lost to GPUs which soon lost to ASIC accelerators: coin mining. I don't think coin mining will move back to CPUs, do you? And if you don't, then what makes ML so different from coin mining?

Also, ML is not some poorly understood technology with wildly changing algorithms that requires CPUs for flexibility. Inference is basically matrix convolution, where each data point participates in an FMA hundreds or thousands of times. The only reason to not use accelerators to do ML in a large-scale data center is if you think something new will come out soon and obliterate current designs. Which is like holding on to a 5 year-old server in 1990s because something faster was going to come out in two years. Something faster was coming out sooner than that, but that just meant you should buy a new system now and upgrade then. (assuming you had an actual need for speed)

Print the post  


What was Your Dumbest Investment?
Share it with us -- and learn from others' stories of flubs.
When Life Gives You Lemons
We all have had hardships and made poor decisions. The important thing is how we respond and grow. Read the story of a Fool who started from nothing, and looks to gain everything.
Contact Us
Contact Customer Service and other Fool departments here.
Work for Fools?
Winner of the Washingtonian great places to work, and Glassdoor #1 Company to Work For 2015! Have access to all of TMF's online and email products for FREE, and be paid for your contributions to TMF! Click the link and start your Fool career.