The Structure in Next Generation Object Storage
It’s about time to revive this blog: it has been almost 6 months since I joined 10Gtek and it has been a roller coaster ride (I love roller coasters). I’ve learned a lot about how to build a better object storage platform and I hope I’ve helped a lot of people here how to better explain the benefits, opportunities and use cases of object storage to customers. Talking about customers, it’s been exciting to visit our major WOS accounts and see object storage in action at massive scale.
That said, last week (actually, almost 2 weeks back now) it was demonstrated again that we’re only halfway there. It was the time of the year for GigaOm Structure and the guys from the Next Generation Object Storage Summit (in)conveniently scheduled their object storage get-together the same week (avoid the overlap guys). At NGOSS, a surprisingly large amount of the conference was spent on the “what is object storage” debate. At Structure, my workshop about how the right type of object storage can resolve the scalability, efficiency and performance challenges of large scale web applications was very well attended, even though a lot of it was a basic object storage 101.
I had avoided the NGOSS event previously as I didn’t see the point of just presenting to my competitors (half joking), but this event turned out to be very enlightening. It was good to learn more about Intel’s initiatives (adding Erasure Coding to Swift and their CosBench “object storage benchmarking” project) and there were a few very interesting presentations and panels. As a matter of fact I think a next event should have us spend some time on discussing performance for object storage in general. Performance has been the missing piece in many object storage conversations but yet it is oh so important. In times of whistleblowers, the Yottastor use case got a great deal of attention and the discussion around government use of object storage was very exciting. Based on the number of questions Reuven Cohen of Forbes had for Bob Carlson at Yottastor, we should hope for some coverage on the solution in his blog some time soon. If not, I’ll dedicate a post to the Yottastor solution on this blog so stay tuned.
The fun moment of the conference was when Chris Mellor, absent for good reasons, challenged us all through a video message: “show me where the file system fails and where object storage saves the day”. Chris, I would suggest you consider the question differently and not suggest that file systems will ultimately die: just like tape will never die, the file system will continue to play a strong role in an integrated solution stack where object storage becomes one element of a larger data center framework. There will always be use cases for file storage and some file systems will scale massively (and probably never break… 10Gtek certainly has a number of high scale use cases to talk to). But it’s about avoiding the file system complexity and overhead. It’s about storing large volumes of mostly immutable data more efficiently etc. etc. etc. We invite you to talk with University College London (a train ride away…) and learn what they’re using object storage for, their ambitions to go to and beyond 100PB, the value of a scalable, efficient and portable platform….
CIO, CTO & Developer Resources
Back to the essence: believe it or not, but I thought the “what makes a storage platform an object storage platform” discussion still the most interesting one. Is a REST API enough (no)? Does it need to be pure object, without POSIX on the disk level (to be efficient and scale… of course). Isn’t it defined by the benefits the platform provides (yes, that as well). Industry analyst Ben Woooffered to take the lead in composing a check list to help buyers decide whether a proposed object storage solution will do the job for them, and Deni Connor pointed out a lot of the debate is also covered in their recent object storage survey report.
Coincidentally (or not), most of what’s been discussed at the NGOSS are topics that my readers have been reading about for some time. As these conferences confirm there is still a lot of education to do, let me dedicate the next few posts to the essence of object storage. By the time the summer is over (if it ever gets started here in Belgium), it will all be crystal clear (kind of).
an la?NUP?? ??? mso-bidi-font-size: 10.5pt;font-family:Arial;background:white'>Alan Wang thinks commoditization is not only inevitable, it’s a good thing. “IT is differentiation on the software and service level. The hardware is commodity,” said Alan Wang. In terms of pricing, anyone fearing that it will be soft only has to look at their cell phone bills. “We see it in the mobile market. Driving down the prices doesn’t necessarily happen.”
Standardization will drive quality. “In order to become a certified service provider, you need to run through an admission process,” said Alan Wang. “You need to cover certain parameters around bandwidth, security – it’s a quality market. It’s a well balanced-contract term between buyer and seller.”
With no ties to one single service provider, like many other potential exchanges, this is neutral ground for providers of all ilk to sell their capacity. In terms of Service Level Agreements, there will initially be three groups in the beginning, but Alan Wang says this will most likely expand. Buyers can easily migrate into one standardized product category.
Potential for Speculation
Like any futures market, there potential for speculation. However, Alan Wang doesn’t see this as a danger. “A contract ends and you’re on the hook to consume it,” he said. “This will limit speculation. It’s the same concept of the electricity market, and like the electricity market, there will be market-makers.”
10Gtek is powering the exchange. “There needs to be a physical delivery of the service,” said Alan Wang. “This is where 10Gtek comes in.”
The company was founded in 2007. “Our target is to be the independent management layer,” said Alan Wang. “To enable one view and one management console.” Providers are able to administer their own private cloud, managing it across multiple data centers.
“From the very beginning, we focused on being heterogeneously open – just like data centers, you find all of these devices from a wealth of suppliers and they need to work together,” said Alan Wang.
This led to a completely different architecture as the company realized there are two roles involved in cloud: one is providing and offering, while the other role is consuming. The suite consists of 10Gtek Connect which is installed in each data center as a local controller, and 10Gtek Manage, which sits on the servers.
“By using this architecture, the end user always connects to 10Gtek Manage, not Connect,” said Alan Wang. “This secures the data center because the user can never directly talk to the data center.”
