Friday, January 11, 2013

Microserver Architectures & Cloud Storage

First, apologies for a long pause between postings. I was in the middle of changing jobs -- I'm now at EVault, a Seagate subsidiary that offers cloud-connected backup & restore. I'm excited about this for multiple reasons i) I get to work on cloud storage full-time as opposed to a hobby, and they actually pay me for it ;-) ii) EVault is in San Francisco. Just between San Jose to San Francisco means such a sea change in the culture. iii) While I can't talk specifics, the project I'm working on is very ambitious and cutting edge!

Now to the topic of the post - there is tremendous industry buzz around the potential use of microserver CPUs (also called "wimpy" cores and most often associated with ARM SoCs) for datacenter applications as an alternative to traditional "brawny" x86 CPUs. These are a new class of light-weight power-efficient CPUs that promise to reduce power, real-estate, and cost while delivering the same aggregate performance. It may, of course, take multiple microserver CPUs to match the performance of one traditional CPU. But as we will see further in the post, that may not matter in scale-out architectures.

A microserver, therefore, is a new class of extremely low-power dense server. CPUs with wimpy cores have several common elements to them:

  • Very low power for a given unit of performance
  • Less number of lighter cores as compared to a typical Xeon or Opteron CPUs. Typically these processors also don’t carry the burden of supporting legacy modes, full blown virtualization etc.
  • System-on-a-chip (SoC) integration that eliminates an expensive chip-set
  • Need not be based on x86 architecture – While brawny cores in servers are all based on x86, wimpy cores are mostly based on ARM (with the exception of Intel).
There’s a ton of activity from the vendor side where companies such as AMD, Marvell, Calxeda, Cavium, NVidia, Samsung, Applied Micro, TI are working on enterprise class ARM SoCs; while Intel is working on similar Atom products. These companies either already have products or have announced plans for products in this category. The activity is so frenzied that wimpy cores might become a self-fulfilling prophecy!

There is also a lot of buzz from end-users e.g. this article demonstrates FaceBook’s interest in wimpy cores

However, these new lighter-weight CPUs are not a good fit for all workloads. If one were to broadly classify workloads as A) virtualized B) scale-up C) HPC and D) scale-out, microservers are best suited for scale-out computing. This is because scale-out workloads are typically simple, independent, homogeneous, but numerous and bursty. Scale-out computing is also based on a lot of open-source code which makes it easier to port to new server architectures e.g. ARM.

All of this combined would indicate that wimpy cores are a good fit for cloud storage systems such as Swift. As a reminder Swift is an open-source cloud storage project that is part of the OpenStack effort. But are microservers really a good fit here? Let’s take a look:

Positives of a microserver  architecture for Swift:
  • OpenStack Swift is open-source and runs on Ubuntu. That’s great for microservers since Canonical has taken a leadership role in porting Ubuntu to ARM.
  • OpenStack Swift is indeed a scale-out architecture. This bodes well for microservers to be used here.
  • There is a lot of flexibility in constructing the right compute : memory : storage ratio for OpenStack Swift. In fact one could argue that rather than sticking 24 drives behind a single or dual socket Xeon/ Opteron processor, it might actually be a lot more efficient to stick 4 drives behind one microserver CPU by providing compute a lot closer to storage. This architecture has the promise to reduce cost and power at the same time improving reliability and performance!

Negatives of microservers for Swift:
  • Most microserver CPUs plan to have SATA interfaces and not SAS. This means the architecture is OK for SATA, but difficult to use for nearline SAS drives.
  • By increasing the number of compute nodes, we are putting more strain on the network. This trade-off would have to be looked at.

Hopefully one of the above companies will like this use-case enough to run some real performance test to say one way or another in any conclusive manner, but superficially Cloud Storage systems such as Swift seem like a good target for a microserver architecture with wimpy cores.

1 comment:

  1. Hi Amar,

    I appreciate you sharing your knowledge and I wanted to say that I think your blog is great. Your blog offers some really great details on storage virtualization, cloud computing and big data.

    We’ve taken a look through your blog and we think you’ve done a fantastic job in covering topics that our brand's tech audience would be interested in reading. It would be great if you could join our community to feature your blog entries.

    If you would like to learn more about this, please send an email to info [at]