David L's Blog

Data as a Service! DaaS Hot!

Great thing about the Cloud is you can take any traditional asset, and turn it into a service. Service’afying stuff has become quite the little business enterprise, and one of the most interesting is data.

Take this weeks announcement by Dun & Bradstreet that they’ll be offering their vast database of commercial information via the Cloud. D&B360, their DaaS service, will not only serve up their highly valuable business insight, it will also provide a platform for consumers to integrate said data into their own systems.

This is a crucial distinction between merely storing data in the cloud, and Data as a Service. Without the interface or integration layer, merely providing your data online is not as useful. What excites me about D&B’s offering (not that it takes much to excite me) is that they’ve taken a very lucrative and valuable business asset, transformed it into a Cloud service, and will now be able to reach more users in more locations, in a completely integrated way.

DaaS as an area of Cloud has been fairly active for the past two years though, with services operating at multiple levels, both at the pure data provision level, and the higher business vertical level. Amazon’s Public Data Sets (information such as US Census Databases, etc), Microsoft’s  “Dallas” project (Mars Rover images!), and Google’s Public Data Explorer (Australian Population Estimates) are all great examples of diverse DaaS offerings.

And the scenarios are pretty powerful as well. When you think about providing real-time decision making capability to a range of clients, whether they be Intranet applications, mobile solutions or pure web, being able to integrate rich (and sometimes very large) datasets in a shaped way over the Internet is critical to success. What’s more, paying for only the data nuggets you use makes consuming these data sets very easy and affordable. And finally, being able to rely on the accuracy and freshness of the data is also a huge value-add, having the data centrally managed and updated takes the load off the user.

So have a quick look at your organizations data, maybe you have a great DaaS offering just lying around.

Build Your Own Cloud? What You’ll Need!

One question I hear a lot is “Do I have to use Google, Amazon or Microsoft if I want to leverage the cloud?”

The short answer is no. There are fundamentally two types of cloud models, private and public.

Public is pretty straight forward; a vendor will stand-up a collection of resources that are made available to customers in a multi-tenant way, with certain safeguards in place to provide containment and security, with shared infrastructure (network, etc) for all users managed to ensure no one user can affect anyone else.

Private is a lot more fun, and a lot more dynamic. Private is the concept that you have access to cloud resources which are dedicated for your use. So an interchangeable term you might hear is “dedicated” cloud.

Private clouds can be managed by a provider, so there might be a set of resources in a vendor datacenter which is carved off for your use. It’s very similar to dedicated hosting, except you generally have more control over how you manage and allocate the resources, but for all intents and purposes, they seem identical.

Private clouds become very interesting and potentially valuable when you consider creating them yourself. “Why?” you ask? If we agree that a cloud is more than just some servers and network gear exposed to the Internet, and that it’s more about the ability to dynamically allocate resources to workloads as demand requires, and that the system as a whole is capable of managing failure and operations, then there are a number of scenarios where building your own cloud inspired infrastructure is a good thing.

Let’s take a simple example. Let’s say you have a team that does project work, for example, financial modeling. This team has the following characteristics:

1. They are extremely mobile. They move from location to location, frequently.
2. They deal with extremely sensitive data. Data that cannot reside in any public or shared facility.
3. There workloads are mission critical, therefore, the platform they run on must be capable of dealing with failures in a robust way.
4. Their workloads change dramatically based on the project. On one project, they may be using a heavy financial analysis tool, on another, a sophisticated modeling product.
5. They need high levels of data storage and redundancy.

What is great about cloud thinking, or should I say, the key to cloud thinking, is to embrace the wonderful aspects of traditional enterprise computing, and the emerging mobile and compact computing standards, with modular construction developments, and really purpose build something to your needs. This was not possible 5 or even 2 years ago, as the commodity hardware markets had not invested heavily in the mobile datacenter space. And mobile/modular construction techniques had to evolved either, however thanks to advanced medical environment research and development, modular and containerized construction techniques have become highly integrated and low-cost.

So where would be start?

Let’s start down the list.

1. Mobility – Let’s take ROCKLABS, they engineer fully mobile container labs. Capable of running full power, completely self contained, and certified for International shipping. It doesn’t get more portable/mobile than that. With a mobile environment built into a container, you can not only transport the thing via any container approved method (sea, land), you can install it anywhere you can establish a level surface!
2. Security – We have a couple of areas of security we need to consider. Physical security, as in, access to the environment by bipods. Network security, so that we can ensure our network assets are secure. And software of course. Let’s tackle them in reverse order.
1. Software security is as it is today for most datacenter environments, nothing cloud specific here.
2. Network is a little different. We’re working in a highly constrained environment, where thermals are higher, space is lower, and needs are far more specific. Putting in a ton of network gear as you would in a datacenter is not only overkill, it’s a potential operations and repair nightmare. Going composite is the way to go. Something like a Cisco Nexus device meets multiple needs, is small in form factor (considering), and optimized for non-standard datacenter deployments.
3. And finally, physical access. The great thing about a container is you can do anything you want to it. Installing the latest in container security and tamper-proof solutions is a breeze. As is installing interior and exterior CCTV devices. All of this can be routed through your containers network equipment for monitoring by remote teams. There are also destructive measures you can take. For example, Toshiba recently announced their new Wipe Technology. Someone pulls out the blade or drive, and the Wipe technology renders the physical device useless. Almost as good as a self-destruct mechanism, although, not as much fun.
3. Failure Management – Planning for failure is smart. Always say to yourself, “When this fails”, as opposed to, “If this fails”. Things always fail, and most times, catastrophically, so designing your system to handle this is smart. The key thing here is to ask yourself, what level of failover/redundancy do I need. Don’t over invest, but also, don’t be cheap. Find the balance. With the computing workloads, this will come down to monitoring and excess capacity to handle taking virtual machines/servers out of the pool for repair and still being able to maintain your required service level. For data, it’s about balancing the amount of storage hardware you’re going to keep to meet your storage needs, with adequate data redundancy/integrity. This also includes off-site replication, in case the container burns to the ground or runs off the road. You also need to make a judgment about non-critical hardware, for example, what’s your plan for a router going down? Does it matter that it might take 48 hours to rip and replace? If so, keep a hot spare, if not, design your system to work around the issue.  
4. Image Repository – Part of the allure of a cloud is that it’s dynamic, not only from a scaling point of view, but also from a workload point of view. Having a repository of images ready for deployment to your computing virtual machines/servers is the best strategy. Not only can you repurpose the whole environment very quickly, you can add extra capacity when needed without having to keep those machines spinning. It saves power/$$$ too.
5. Storage – We covered this in the Failure Management section above, but the key thing here is to make some decisions. For example, do you run commodity servers and run a software based storage platform that is capable of dealing with the redundancy/scale issues? Do you use a purpose built appliance, like an EMC Celerra? Each has their pros/cons, it’s about mapping these capabilities to your needs, and finding the one with the best fit, least gaps, and least “non-exploited functions”.

OK, now we’ve taken care of their “needs”, we also need to think about some basics that the overall solution will need.

1. Power – You’re going to need two power solutions, the mains power distribution and stepping, so the big dirty public utility power can become nice, clean, smooth power for your gizmos. And some rack level solution, so you can remotely monitor each servers power use, as well as be able to control the power outlet through software, to turn it off/on, etc.
2. UPS – Backup power is also critical, and depending on your needs, you’re going to need to balance size with time. There are traditional battery based solutions which will work, and more advanced solutions, like a FES style system available from folks like Pentadyne or ActivePower
3. Cooling – Also known as HVAC, is absolutely critical. All your shiny bits of silicon will turn themselves into a hot mess unless you keep the temperature absolutely perfect. This is equally hard, when the whole idea of a container is that it can just be deployed anywhere, so anything that requires Perrier grade water is going to be problematic. There are also many options, traditional refrigerated cooling is common, but more advanced approaches, such as ventilated cooling and dry processes like adiabatic cooling.
4. VESDA – Think glorified smoke detector, VESDA, or Very Early Smoke Detection Apparatus, now represents the complete fire management system. These now include monitoring, detection, and suppression.  
5. Seismic dampening – When you start moving your stuff around, if you haven’t taken into account movement being transferred from the exterior of the container to everything inside, you’re going to be hearing lots of rattling coming from inside your expensive box. Bracing the physical structure, investing in dampening technology, like APC’s NetShelter rack for the servers, and vibration dampening for the moving parts like hard-drives will save you much time, money and pain. 

The above is just a starting place when thinking about building your own cloud, that is modular, self-contained, and portable. As you can see, it’s a big undertaking, but at the same time, presents huge opportunities for certain applications.

Basic Elements of a Cloud Service

I’m not sure about most folks, but when I hear the topic Cloud, and I consider all the different variations and permutations that constitute a Cloud, I always find it useful to relate what I’m hearing back to a simple service model.

So let me offer one up for consideration. This is based purely on my own experience, and while most of my hands-on experience is based on the Microsoft platform and more specifically, Windows Azure, I think the abstract concepts are pretty versatile.

I always start by thinking about the the service infrastructure that makes most Clouds, well, Clouds. If we boil down the most common, there is:

A hosting layer:

Key elements of the hosting layer are:

• The ability to receive incoming traffic (known as ingress) and the ability to transmit outgoing traffic (known as egress).
• The ability to execute a workload:
  • This could be a piece of code running within a pre-determined environment, like it is in Windows Azure and Google App Engine currently, or
  • This could be a virtual machine, like it is with Amazon EC2

These key elements are also the primary metrics that you get charged for, for example:

• How much traffic came into your service and how much traffic went out of your service. This is generally measured in gigabytes.
• How long did your service run for, and how many “instances” did you use. This is generally a function of hours multiplied by instances. Instances is really how many machines did you spin up for your service. If you ran your service for 8 hours and used 3 instances for fault tolerance and capacity, then you’ll get billed for 24 hours of usage at whatever the per hour rate is.

A storage layer:

Key elements of the storage layer are:

• The ability to receive incoming traffic (again, known as ingress), however, not in the same way as the hosting layer. The main difference here is that you don’t really execute anything inside the storage layer, this is generally a system all unto its own, with baked in security (auth, identity), load balancing, etc. What you do is send it data that you want it to store.
• The ability to transmit outgoing traffic (again, known as egress), and again, not the same as the hosting layer. When we think about egress for a storage layer, given the system is closed, it’s really about how much data is being sent out from the storage system from your “account”. For example, how many times did someone request and download a particular image or media element that you uploaded as part of a web app.
• The ability to execute a storage function. Most Cloud based storage systems have a predefined interface, a set of functions that you use to store and retrieve data. These generally map to some in-built metaphor, such as a table or a blob. So, you can do things like, AddTable, or AddRow. These functions may also have requirements, such as requiring a username or identity token.

Just like the hosting layer, the elements above dictate the way you pay for the service.

• Traffic is the same as it is for hosting, and this is primarily because the underlying infrastructure is network based, and not really value-add (as in, it’s a requirement to doing business, not something added to create value for the user), so the charging of it is fairly rudimentary.
• Accessing the storage system and storing and retrieving data is charged based on the relation of these functions against the underlying infrastructure. For example, if the storage system deploys the interface part of the system on a separate set of resources than the storage part of the system, then each time you call an interface function, you are using a resource that needs to have its cost recovered. Same with the storage. As these resources (network hardware, storage hardware) are generally different, the pricing is different too. So you might get charged a fraction of a cent for each function call you make, you might get charged a fraction of a dollar for every gigabyte of data you store. Considering how “chatty” your potential software clients (web, desktop, mobile) are starts to become a key design point, to keep costs down.

A connectivity layer:

This is something that is not available in all the Cloud offerings today in the market, but certainly something that the larger players either offer or are on the cusp of offering. It has one essential feature, the ability to seamlessly integrate resources behind a corporate firewall with resources in the cloud.

Some would argue that this is a critical feature when selecting a Cloud platform, for the real power of the Cloud is the ability to exploit the dynamic, scalable nature of the Cloud with secure, managed corporate resources.

A couple of things I left out of my post are things like:

• Developer tools – This is fundamental to any Cloud platform, but is a very broad topic, so one I’m going to save for another post.
• Monitoring and Operations support – Again, like developer tools, and somewhat tightly integrated; monitoring and ops is a very broad topic, so one I’m going to leave for later.

Now this is by no means a complete treatise on Cloud, but it is definitely a starting point. I’ll definitely revisit this topic and go deeper in future posts, and in the meantime, if you want me to take a stab at any other Cloud topics, just drop me a line.

What is my new day job?

I’ve had a bunch of lovely emails from readers of my old blog (http://blogs.msdn.com/davidlem) wishing me well on my new job, and asking, “What is your new job?”. Great question, let’s make something up together!

Let’s start with where am I working. I work for PwC, a.k.a PricewaterhouseCoopers. PwC has a long and successful history in consulting, primarily in the accounting area (tax, risk, that kind of thing), but also in the technology space (Oracle, SAP, etc).

Within PwC, there is a group of folks who focus purely on Technology. It makes so much sense really, I mean, there can be no successful adoption of technology without considering the business impact, so who better to advise on technology than a group of people who come from a strong business heritage.

And within that group of people, there is a group of people focusing purely on the Cloud. Now, in the limited days I’ve spent with PwC, and the small group of customers and partners I’ve had the opportunity to meet, the first reaction I get is, “You work for PwC… doing Cloud… really?”. It’s a completely fair reaction/response, I mean, Cloud is so new and shiny, that it is generally the big tech companies that spring to mind when you think Cloud. But what PwC has cooking is what got my attention in the first place.

See, like most folks who have been working on, in, and around the Cloud, sooner or later your going to realize that the technology bits and bytes are actually just one part of a bigger set of challenges. Once you’ve made the decision to explore the Cloud as a key piece of technology, the implications on the way you do business start to pop to the forefront. I’d even go as far as to say, if you’re thinking of doing “Cloud”, and you’re only thinking about technology, you’re going to find yourself in a quandary just before you launch your “thing”, as you realize the impact of your “Cloud” “thing” is pretty considerable in relation to your business. What am I talking about specifically, let me give you just one example. Say you sell software. And you used to sell all your software out of a particular state, mainly for tax purposes. Now you move some of your software functions to the Cloud, and start selling that to your customers. If the datacenter that serves out your service is located in a state other than the one you used to sell out of, and that new datacenter’s state conflicts some how with your tax strategy (say it has a higher sales tax), you’ve created a tax problem for yourself, that you’ll need to address to keep yourself out of hot water. What’s more, the effort and investment you’ve put into your “Cloud” “thing” will need to be refactored, which could cost time and money. There are many example more, such as how do you audit your billing pipeline to ensure you are charging customers the right amount for their use of your service, or, how do you ensure you’re SLA is watertight and not open to mischief? These are exactly the kinds of things that a mashup of PwC business talent and Cloud skills can help with.

So back to my job. Well, I kind of summed it up above. I’ve spent a lot of time on the technology side, designing, building and developing a Cloud, and now I’m going to spend a lot of time helping customers plan and adopt Cloud(s) as a key technology investment.

With that, I’d love to hear from anyone who has questions or thoughts about this emerging area of Cloud business strategy/tactics. Drop me a line at david dot lemphers at us dot pwc dot com.

Goodbye Microsoft

Today I resigned from Microsoft. While I’m leaving to take a big step in a new direction, I have to say, it’s with very mixed emotions.

Five and a half years ago, I started a journey that changed me forever. Not only was it the realization of a teenage dream (I still remember unwrapping the Visual Basic 3.0 box as an intern and thinking, one day I’ll ship a product for Microsoft), but it was also a graduation of sorts into the realm of industrial software engineering.

My time at Microsoft has been completely magical. Not only have I been fortunate to work with some of the most amazing professionals in my career (colleagues, partners, influences and students!), I’ve also had the opportunity to travel around the world, move from Australia to the United States, and fulfill a life-long dream and ambition, to ship a major product for Microsoft Corporation.

But, it’s time to move on.

Where am I going? I’m joining PricewaterhouseCoopers.

What am I going to do there? I’m taking a role as Director of Cloud Computing, focusing on how customers transform their businesses to take advantage of the cloud.

Why? Cloud is my passion, and while building one of the biggest and best at Microsoft was amazing from a geek and engineer perspective, I’m yearning to focus on the other aspects of what it takes for customers to truly use the cloud as a business transformer.

So to all my Microsoft friends and family, thanks for five and a half years of being awesome!

Vale, lacerte!