Understanding the Hidden Costs of Cloud Automation solutions – Part2

Posted on 12/05/2011 by Bhaskar Krishnamsetty

The Holy Grail Solution – Extensible and turnkey Solution

Organizations require a turnkey solution to address the above hidden aspects and costs. Rather than dealing with the costly professional services and lengthy implementation cycles, organization require a true integrated ready to use turnkey cloud solution. The current traditional IT cannot be transformed to full agile and adaptive IT if some parts of the infrastructure management are still manual. One must have orchestration engine and end-to-end automation in place to move to an agile IT.

A holistic deployment of entire server, storage, network, and virtualization components of infrastructure is critical to build or manage large-scale data centers. Self-service creation of entire application environments requires automated and unified provisioning of both physical and virtual components. A seamless approach to linking and automating management processes and tasks across physical and virtual infrastructures is needed.

Organizations require an extensible platform by design to easily add and support additional components.  IT administrators should be able to customize and extend any infrastructure administrative and operational tasks that are specific to their environments. Any orchestration software that offers comprehensive open APIs and rich set of pre-integrated workflows will expedite the Clouds transformation journey.

So, I strongly recommend to ask the below questions when considering cloud management and orchestration solution.

Questions to ask?

  • Can we setup a proof of concept or trial demo in hours?
  • Can we deploy the solution in a day or two?
  • Can we build or manage a base cloud in matter of days?
  • Can we customize the offering with minimum to no help from the vendor?
  • Does it require extensive hidden professional services?
  • Can we build and execute repeatable physical and virtual infrastructure management and provisioning workflows without complex custom scripts and expensive system integration engagements?

Tail piece: Check out the iceberg under the waters -:)

Understanding the Hidden Costs of Cloud Automation solutions – Part1

Posted on 11/27/2011 by Bhaskar Krishnamsetty

There are many cloud and data center automation solutions in the market. I would like to provide some insights into the hidden costs with most of these automation solutions. This blog will go over some of these hidden costs and gotchas, the “Holy Grail Solution – The Turnkey and Extensible Solution”, and most importantly what questions should be asked to identify the right solution.

Multiple point products and integration overhead: Organizations typically use a disjointed set of vendor-specific tools to manage virtualized data centers. Virtualized data centers cannot be operated efficiently without integrated infrastructure provisioning and management tools. Most of the cloud platforms in the market today, lack a management layer which can handle physical, virtual and cloud based resources. Having a diverse set of management tools not only adds to the complexity but also increases the failure rates. Data center admins spend a lot of time correlating data from different sets of monitoring and management tools. Organizations are looking for a pre-integrated ,out-of-box orchestration solution with low footprint and faster time to implementation.

Expensive Professional services engagements: Typical cloud management and automation solutions require additional extensive professional services. After spending months of time and money, you might be lucky enough to get reports and customizations suited to your specific cloud. The premise of the cloud is that there is almost no penalty for trying something and failing. These expensive professional services engagements will take away that cloud promise – pay as you go, agility, self-service management and upfront expensive investments.

Distributed operations: Enterprises have undergone a substantial change over the last decade and geographically distributed operations are very common. There is great pressure to simplify the tools to empower self-service at the local level and to provide the robust functionality needed to support local languages and currencies. Organizations need the ability to manage across distributed data centers and to have a single pane of visibility across their entire operation.

Lack of integrated policy management and governance: Most of the solutions offer some aspects of management automation – I would say the easy/simple parts of infrastructure management. Many organizations still end up manually doing policy management, governance, auditing and many IT operational processes. These processes still take majority of admin’s time. The true potential of any automation solution cannot be fully realized if the parts of the process are not, need a push button or single-click automation capabilities. For instance, the automated provisioning process requires service catalog, self-service portal, resource checks, dynamic resource allocation, policy enforcement, capacity planning and the approval process. If any of these functions cannot be automated and are not part of a provisioning solution, then it is not a true automated provisioning solution.

Proprietary and closed solutions: Each organization’s environment varies and requires customization to some extent. If the orchestration solution doesn’t offer the customization and extensibility capabilities, organizations depend on the vendor to provide the customizations and will be vendor-locked. Providing north bound APIs is an essential part of any true management and orchestration solution. IT admins need out-of-box workflows for majority of typical administrative and operational processes, and then they should be able to customize and automate remaining workflows specific to their environment using easy to use workflow designers.


The Holy Grail Solution – Extensible and turnkey Solution (To be Continued…)

Pods2Clouds

Posted on 07/05/2011 by Bhaskar Krishnamsetty

Let me start off this blog with a simple question – “Is the lack of pre-integrated and out-of-box orchestration software to automate converged infrastructure slowing down the transition to private cloud?” I would say the answer is yes. If organizations are serious about using the converged infrastructure to build private clouds, they would realize this potential bottleneck. Otherwise, organizations will be limited to scale the pod based data centers and also cannot realize the private clouds promise – operational efficiencies.

Today’s datacenter architectures are getting complex with a mix of data center infrastructure, including servers, storage, networks and virtualization. These pre-built systems minimize the amount of up-front work and provide self-contained “pods” that include everything necessary to run a system. The advantages of converged systems are – “pre-integrated solution, simplified and less expensive”.  The complex and high cost data center orchestration software defeats the purpose of the converged infrastructure. The converged infrastructure systems need unified and integrated automation software to expedite the cloud transition – “pods to  clouds”.

The best of breed multi-vendor stacks provide the best features and performance according to Joe Onisick, Cloud Solutions Architect at WWT. These multi-vendor stacks provide organizations the  flexibility to build pod based next generation data centers organically on top of existing systems. FlexPod is one such multi-vendor stack that is a predesigned, simple, validated data center solution capable of running a variety of application workloads. The orchestration software that supports multi-vendors in both physical and virtual layers complements well with these best of breed multi-vendor stacks.

Organizations need a pre-integrated and out-of-box orchestrator with the following capabilities to fully leverage converged infrastructures for building private clouds

Speed of deployment: A very good point made by Joe Onisick in one of his blogs was “automation/orchestration software is included in a product offering doesn’t mean it’s ready to go ‘out-of-the-box.’ Orchestration and automation are very custom software sets that typically require significant service hours to integrate and tailor to each individual environment.” Organizations are looking for a pre-integrated and out-of-box orchestration solution with low footprint and faster time to implementation.

Unified orchestration: Having a diverse set of management tools not only adds to the complexity but also increases the failure rates. Data center admins spend a lot of time correlating data from different sets of monitoring and management tools. Integrated management not only streamlines the creation and management of Virtual Data Centers but also offers a complete visibility across the entire IT stack, thereby, helping in performance analysis and troubleshooting. This vastly reduces the complexity and gives IT a seamless way to manage the existing infrastructure as well as the new cloud based infrastructure.

Private cloud enablement: The private clouds can make internal enterprise data centers run as efficient as public clouds. Organizations need cloud management and automation capabilities such as self-service portal, service catalog, charge back and multi-tenant security in the integrated stack to enable private clouds. To really optimize the resource usage, enterprise managers need an orchestrator that can automate the management and provisioning of bare metal, virtualized and cloud resources.

Model-based orchestration: IT teams are not looking for tools that make scripting easier. Instead they are looking for a tool that packages task libraries, best practices and pre-built workflows based on infrastructure component models. IT teams should be able to build and execute repeatable physical and virtual infrastructure provisioning workflows without complex custom scripts and expensive system integration engagements. So a tool based on model-driven automation with a rich set of prebuilt workflow solutions for solving complex, repetitive and resource consuming IT tasks is “the need of the hour”.

Open Integration: Vaughn Stewart on his blog brought up a great point that “the multi-vendor pod architectures such as FlexPod provides a set of open APIs that enable third-party infrastructure and systems management solutions.” These open architectures truly enable partner eco systems to offer integrated stacks. Customers can also choose from a broad network of world-class solution delivery partners. So any orchestration software that leverages these open APIs and offers rich set of pre-integrated workflows will expedite the Pods to Clouds journey.

Why do you need physical infrastructure automation?

Posted on 06/15/2011 by Bhaskar Krishnamsetty

Building your own internal private cloud is actually lot harder than just creating and managing virtual machines. To fully realize the benefits of private cloud, you must have a full self-service catalog that includes both virtual and physical infrastructure. You must be able to automate all the things that take you long time to move to an agile IT. The current traditional IT cannot be transformed to full agile and adaptive IT if some parts of the infrastructure management are still manual. You must have an orchestration engine and end-to-end automated processes in place to move to an agile IT – the real promise of private clouds.

Most of the cloud platforms lack a management layer which can handle physical, virtual and cloud based resources. They are worried that they will be forced to use multiple tools to manage different sets of environments. Having a diverse set of management tools not only adds to the complexity but also increases the failure rate. IT leaders expect a single set of integrated solution that offers them an end to end solution to manage the entire lifecycle of the complex IT environment. You can reduce operational overhead by automating entire IT stack.

Following points highlight why you need to automate physical infrastructure as well.

Self-service creation of end-to-end application environments: Self-service creation of entire application environment requires automated provisioning and configuration underlying physical and virtual components. A seamless approach to linking and automating management processes and tasks across physical and virtual infrastructures is needed.

Physical Infrastructure Service Catalog: Organizations can extend the current service catalog offerings of virtual machines and virtual desktops to physical infrastructure services. It really opens up new market opportunities for service providers to offer physical infrastructure services.

Virtual data center explosive growth: Data centers of today are facing explosive growth with the advent of virtual environments. The new generation virtualized data centers require the speed and agility of underlying physical infrastructure provisioning and configuration. Organizations cannot expand and keep up with the pace of virtual environment expansion without physical infrastructure automation capabilities.

Single management interface: Giving organizations the ability to manage and orchestrate their entire infrastructure from a single management console yield considerable operational and administrative cost savings.

End-to-end service performance: Understanding the application layer and underlying supporting infrastructure is critical to deliver on SLAs, End-to-end service monitoring and performance can be achieved by integrating and automating both physical and virtual data collection.

End-to-end service visibility: Bringing together data from across disparate parts of the virtual and physical infrastructures can improve the visibility from a service perspective. You need datacenter management solutions that automate cross-silo virtualization capabilities to deliver end-to-end service visibility across both physical and virtual infrastructures.

Cloud Automation using Cloupia Orchestration Engine

Posted on 04/05/2011 by Bhaskar Krishnamsetty

Cloupia’s cloud automation using cloupia orchestrator helps standardize IT services, reduces downtime and increase compliance.  IT administrators can customize and automate many infrastructure administrative and operational tasks using easy to use workflow UI designer. Cloupia provides built-in task library where admins can easily build complex workflows. IT teams can build and execute repeatable physical and virtual infrastructure provisioning workflows without complex custom scripts and expensive system integration engagements. Application teams can also leverage these orchestration workflows to accelerate application deployment from development to production. With Cloupia’s orchestrator, IT teams can

  • Visually design infrastructure automation workflows using an easy-to-use UI designer
  • Automate manual & labor intensive operational and administrative tasks
  • Monitor for alerts on the systems using predefined or custom workflows
  • Ease management of complex IT infrastructures

Example Workflows:

  • Customized provisioning workflows
  • Pre & post application provisioning workflows
  • Automating server backup and recovery workflows
  • Scaling up and down upon workload demand
  • Alerts and actions upon performance thresholds

Who needs Cloud in a Box solution?

Posted on 01/20/2011 by Bhaskar Krishnamsetty

The top technology hardware vendors that are used to sell servers and storage boxes are kind of worried for a while with the cloud phenomenon and of course want to be part of cloud. So, they invented Cloud in a box to sell bigger boxes-:) It certainly doesn’t make sense from Cloud users point of view because it goes against typical cloud characteristics such as elasticity and fixed hardware costs. But it does make sense from cloud IaaS Providers or Enterprise IT teams point of view that are planning to build new dynamic data centers. It does provide higher level of abstraction unit to build next generation data centers. It takes care of lot of IT dirty work of connecting servers, storage and networking pieces along with back chassis and channels and certainly offer lot of OpEx savings. Additional processors and storage can be added relatively with ease using these pre-integrated and pre-configured bundles.

Cloupia’s Cloud-in-a-Box using Cisco UCS, NetApp & VMware: The Cloupia’s
Cloud-in-a-Box solution consists of Cisco UCS platform, NetApp Unified storage, VMware ESX Hypervisor Server & Cloupia’s Unified Infrastructure Controller. It comes with pre-integrated and pre-configured Compute, Storage and Network resource bundles. The Cloupia’s Cloud-in-a-Box solution offers self-service portal, multi-tenant security, monitoring and provisioning capabilities. The Cloupia’s software automates and orchestrates the deployment and management of compute, storage and network resources with-in the “Cloud-in-a-Box”. Organizations can jumpstart the private cloud implementation using Cloupia’s Cloud-in-a-Box solution. This solution can be used as a building block for new private cloud and dynamic data center build-outs by Service Providers and Enterprises.

There are several definitions for Cloud-in-a-box from “integrated cloud software and hardware” to “appliance with software, hardware and associated services” to “converged infrastructure platform”. Our definition is – a turnkey cloud solution to build private clouds with pre-configured and bundled compute, storage and network resources along with management and orchestration software. The cloud can be built fairly easily out of the box, I mean out of “Cloud-in-a-Box”-:)

2011 Virtualization and Cloud Management Predictions by Cloupia

Posted on 12/27/2010 by rpenmetsa

1. Virtualization and cloud management enters the mainstream – Organizations have well understood about the cost benefits and CapEx savings due to virtualization. With the increasing virtualization footprint in enterprises, the adoption of hybrid clouds will continue to grow in 2011. Organizations will realize true operational efficiencies and OpEx savings with comprehensive virtualization and cloud management solutions.

2. IT Automation will be key focus of Virtualization and Cloud solutions – Any company that has reached virtualization maturity needs automation tools for the virtual environment to gain any operational efficiency. Self-service provisioning, self-service management, automated monitoring, and automated dynamic reporting are some of the automated virtualization and cloud management capabilities organizations will be looking for.

3. Cost-effective, comprehensive and integrated cloud management capabilities will be key business solution differentiators. Large Enterprise IT teams are hesitant to move production applications into virtualization or cloud environments due to lack of visibility into performance and management issues.  In 2011, Organizations will be looking for comprehensive management tools with broad functionality to support multiple virtual infrastructures. Customers will be looking for end-to-end comprehensive cloud infrastructure management with pre-integrated cloud suites.

4. Virtual storage and virtual network management will start catching up with virtual server management. Organizations will look for integrated infrastructure management encompassing virtual compute, virtual storage and virtual network resources. Application deployments in virtual or cloud environments must consider all three pillars of virtual infrastructure during the provisioning – compute, storage and network.

5.  Multi-vendor hypervisor management solutions will prevail – Most organizations may need multiple virtualization vendors or multiple cloud providers to cover the entire application portfolio. So, organizations need flexibility to choose the virtualization vendor they want and a management solution to manage multiple virtualization vendors to avoid vendor lock-in risks.

6. Public clouds require the same security, governance and management tools as private clouds. The complexity and speed of technology will be key drivers for public cloud adoption. The single pane-of-glass and hybrid cloud management will be key desired capabilities to provide visibility, avoid vendor lock-ins, avoid integration overhead and to limit cloud sprawl.

7. Cloud compliance and governance will be key – Deployments that enter production environments without adequate policy adherence often do not meet the organization’s requirements. Organizations will look for advanced policy based provisioning capabilities to set and enforce policies easily to ensure compliance and governance.

8.  Hosted private clouds will gain momentum – as most enterprises look towards managed hosting providers for cloud expansion and disaster recovery. Most organizations take time to transform their existing data centers to private clouds. Hosted private clouds will be good option to move into clouds without disturbing the existing data center architectures and infrastructure immediately.

9. Advanced capacity and cost management plays key role in managing and planning dynamic virtual and cloud environments - Organizations need comprehensive visibility into infrastructure resources consumption in real time for capacity planning. Business units must get the visibility about how much they are paying for resources and how much goes for unused, allowing them to optimize resource consumption and costs with chargeback capabilities.

10. Connected IT processes – The true barrier for the cloud adoption is not the technology, but the cultural shift in the way IT operates in traditional environments. The lines between IT silos will become blurry. IT can respond to business needs in real time by streamlining the organization’s entrenched silos and automating end to end service delivery processes.

The Building Blocks of Private Clouds

Posted on 12/03/2010 by Bhaskar Krishnamsetty

Private clouds enable the power of sharing by maximizing the efficiency of computing, networking and storage resources. So, organizations can achieve rapid and efficient delivery of IT services, faster time to market, and reduced IT capital and operating expenditures with private clouds. Private clouds enable organizations to maximize the existing infrastructure investments and take advantage of cloud benefits without compromising the security and internal policies. The private clouds can make internal enterprise data centers act like public cloud providers. It can significantly improve the VMs to admin ratios up to 1000:1. Large enterprises are prioritising private clouds, according to a survey by Forrester Research. Forrester principal analyst James Staten said “Larger companies sink more investment into their own data centres, operations personnel, and IT processes. Thus, they have more to lose if the business goes to the public clouds. These organisations also have more in-house capabilities with which to build and operate a private cloud.” Let us examine the key building blocks of private clouds.

Virtualization: Virtualization provides significant cost savings by server consolidation but also provides far reaching benefits by enabling workload portability, encapsulation, and automated administration. Virtualization is a key building block to a complete cloud implementation, enabling flexibility, isolation and mobility within a cloud environment. Organizations which have deployed virtualization are already receiving some of the benefits that a complete private cloud solution will provide. The virtualization provides the capability of decoupling workloads from the servers they run on. System administrators can quickly move applications and services from one physical computer to other based on changing resource needs. New applications inside the virtual machines can be deployed with-in matter of minutes thus decreasing the IT response time.

Automation: Any company that has reached virtualization maturity needs automation of the virtual environment to gain any operational efficiency. Automation is crucial for enabling rapid provisioning and deployment. By automating tedious and time-consuming tasks such as configuration, setups, capacity modeling & real-time monitoring, organizations can focus more on strategic initiatives. Organizations can drastically reduce overall provisioning times and recurring provisioning efforts with automated self-service capabilities of private clouds. It must contain policy engine, work flow engine, provisioning engine to enable policy driven automated provisioning. The automated decision making and automated provisioning align IT resources to business goals through intelligent, policy-based IT resource allocation. The automated provisioning improves productivity of IT staff by automating and orchestrating manual repetitive tasks to decrease human errors during installation and configuration processes. IT can respond to business needs in real time by streamlining the organization’s entrenched silos and automating end to end service delivery processes.

Self-service management: Many analysts estimate that over 70% of outages in a data center are caused by mis-configuration while provisioning servers. The automation of server provisioning & configuration will make it easy to reproduce or repurpose servers, an otherwise very time consuming process. The self service capabilities using self-service portal, service catalogs and automated application provisioning reduces administrative burden and infrastructure management costs for IT. The users can provision and deploy applications and services in a matter of minutes with self-service capabilities. The users can rapidly build or tear down the complex environments using self-service provisioning capabilities. A complete end-to-end service delivery management that covers service creation, publishing, governance and management is required to enable a true self-service management. The service delivery orchestration and automation engine connects IT processes and coordinates siloed teams with pre-defined service delivery workflows. It also standardizes how IT services are delivered across organization and orchestrates IT processes that span across multiple groups.

Measured service: Private clouds must automatically control and optimize resource use by leveraging a metering capability. To be charged only for what you use, your usage must be measured. Service resource usage must be monitored, controlled, and reported to provide the transparency for both the provider and consumer. The capacity and cost management components are key to measure cloud services. The chargeback component must provide cost management, monitoring and reporting capabilities. The capacity management component must provide resource utilization, monitoring and reporting capabilities. The capacity and chargeback components enable business units to account, monitor and report resource usage and associated costs. Business units get the visibility about how much they are paying for resources and how much goes for unused, allowing them to optimize resource consumption. It enables to make better informed decisions by business units with a clear view into resources consumed and their associated costs. Giving organizations in-depth information about capacity and cost drivers help them make smarter decisions during budgeting processes, cost reduction analysis, and capacity forecasting analysis. Business and IT leaders can also use the data to make better projections about future resource needs.

Cloupia Announces The Public Beta Release Of Its Unified Cloud Platform API – “CloudSense API”

Posted on 10/26/2010 by admin

Cloupia today announced the public beta release of its unified cloud platform API – CloudSense API. This allows any cloud computing provider to build and interact with Cloupia platform’s capabilities using REST based CloudSense API. Using the CloudSense API, IT administrators and developers will be able to manage their cloud infrastructure with more control and flexibility. The CloudSense API allows Cloupia’s platform users to programmatically control their cloud environments. Organizations can also extend their existing workflows and provisioning automation to virtual and cloud environments with CloudSense API. The API keys are easily generated and managed. The CloudSense API empowers developers to use one interface to interact with a variety of cloud application services, enabling them to more easily access new technologies from cloud vendors using REST-like Query interface with JSON & XML encoding.  As a result, developers can deploy software applications to access services in these cloud environments without making time consuming and expensive changes to their source code. CloudSense API provides the ability to:

  • build interactive clients for Cloupia unified cloud platform – Cloupia Unified Infrastructure Controller
  • access to wide-range of self-service data center operations
  • instantly query cloud environments to retrieve virtual and cloud infrastructure resource data sets
  • retrieve real-time resource usage information across multiple virtual and cloud environments
  • integrate rich resource data sets and reports into their own Cloud portal
  • programmatically access and control to private, public and hybrid cloud environments
  • enable interoperability between clouds

What’s new in Cloupia Unified Infrastructure Controller V2.3?

Posted on 10/11/2010 by admin
  • Virtual Data Center Management – With virtual data centers, organizations can create multi-tenant isolated environments to combine virtual resources, operational details, rules and policies to manage specific group requirements. A group can have and manage multiple virtual data centers. Images, templates and policies can be further customized at virtual data center level. Organizations can allocate quotas and assign resource limits for individual groups at virtual data center level.
  • Cisco UCS support – CUIC provides auto-discovery, monitoring and complete visibility to manage all Cisco UCS components – chassis, servers, fans, modules, memory units and disks. Provides ability to configure and manage service profiles for dynamic load balancing and power management.
  • VM snapshots: End users can create and manage snapshots for restore points during recovery from disasters and malfunctions. Users can take the snapshot of the VMs at any time and revert to that snapshot at any time. Provides VM snapshot summary reports and ability to mark Gold snapshots.
  • VM access and credentials management – End user can configure unique credentials and self-manage secure access to VMs via web access or remote desktop. Users can lock and un-lock the VMs.
  • Restful API access – Access, authenticate, configure and manage Cloupia managed clouds and exposed functionality in CUIC using Restful API.
  • White labeled self-service portals: Organizations can customize and white-label self-service portals for its customers.
  • Distributed virtual switch support – Provides administration, monitoring and provisioning of distributed virtual switch.
  • Service Request Management Enhancements – Added provisioning service request cancellation, resubmission and logging capabilities.
  • Group administration – Access, create, manage and administer group level resources, VMs, policies, catalog items and users by Group Admins.
  • Added more capacity and inventory management reports
    • VM density report
    • Resource pool report
    • Inactive VMs report
    • Over-utilized and under-utilized resources report
    • Top 5 reports (Groups, vDCs, Hosts)
  •  Customizable dashboards using drag and drop widgets by users and admins
  •  End-user self-service password reset
  •  VM lease extensions
  •  Admins can configure auto deletion of inactive VMs to control the VM sprawl