Stuff for beginners
P2V conversions/migrations
3. P2V Primer
Economics of Cloud Computing
3. Cloudonomics
All the brilliance of AI on minimalist platforms
2 years ago
Showing posts with label cloud computing. Show all posts
Showing posts with label cloud computing. Show all posts
Friday, August 20, 2010
Interesting articles on Virtualization and Cloud Computing
In this article, I shall point you to some really interesting articles on Virtualization & Cloud Computing I came across during the literature survey for my research. These articles are indeed very stimulating and have been useful in my research in some way or the other. The texts mentioned here are categorized under different areas like Resources for beginners, Physical-to-Virtual (P2V) conversions, Server consolidation, Economics of cloud computing.
Saturday, July 3, 2010
Virtualization Lifecycle in the Context of Cloud Computing
Virtualization is a key technology for enabling cloud computing. As we have read before, virtualizing applications and consolidating hardware reduces IT Infrastructure costs (which includes purchasing hardware and maintaining it), allowing easier management of resources and on-demand provisioning of resources in the data center. The Infrastructure as a Service (IaaS) model of the Cloud deals with provisioning of hardware, storage and networking resources. Virtualizing applications via the use of Virtual machines holds great significance in this model to deliver and manage IT services. The Virtualization lifecycle comprises a set of technical assessment activities which are governed by business and operational decisions. Technical assessment for virtualizing candidates revolves around meeting end-user Service Level Agreements (SLA's), reducing IT costs, and designing an optimized data center. Every phase in the virtualization lifecycle for cloud computing is highly challenging with a wide variety of complex open problems which are currently being tackled.
Analysis & Discovery : For the process of moving from Physical environments to Virtualized environments (P2V), solid analysis of the virtualization candidates must be performed. This stage involves discovering the data center entities (servers, networks, storage devices), collecting utilization profile data of those entities along the different dimensions (CPU, memory, network i/o, disk i/o). The main theme of P2V is to move applications from an under-utilized bare metal environment to a virtualized / hypervisor environment to enable optimum utilization of hardware. In addition to discovering the heavy artillery in the data centers, it is important to assess the applications deployed on them. The OS characteristics (scheduling policies, caching strategies etc), application characteristics and configurations (Tomcat no. of starting threads etc). Once the performance and application characteristics are assessed, capacity management models for the need to be developed to host those applications in the virtual environments. For more information on the technical assessment for virtualization, see 'Conduct a Technical Assessment for Server Virtualization'.
Implementing Models : Developing capacity models for a virtual environment is a tricky task since it is governed by other business and operational factors. Target SLA's (performance, availability ), power consumption levels are to be kept in mind along with the possible impacts of virtualization (hardware normalization, hypervisor overheads). The idea is to come up with a 'pre-VM placement' strategy which describes the 'footprints' of VM's.
VM Placement & Management : Allocating Virtual Machines to Physical Machines dynamically and optimally would be the goal of IT Enterprises. VM's can be scaled out/up on demand. Provisioning of VM's goes hand-in-hand with capacity management monitoring to track the desired Service Levels of applications. Server consolidation is an important and inherent part of this phase, as optimal placement of VM's across physical architectures is the key to meet the business goals. Re-shaping and re-sizing the footprints of VM's dynamically in real time is also a hot research topic. Management of VM's also involve migration to other physical hosts, monitoring performance which can be done centrally. Many open issues also exists in VM Migration, such as synchronization problems and security issues.
Thus, the virtualization life cycle poses many challenges and many groups in the Industry and Academia are grappling to solve these issues. If executed well, it has a promising future in the context of cloud computing. For more resources on Virtualization and Cloud Computing, see 'Virtualization / Cloud Computing Blogs, Websites, Resources, Articles'.
Friday, January 29, 2010
Non-Relational Storage/Databases and Consistency
With the explosion of cloud computing, non-relational storage/databases (e.g. Google's BigTable) have gained attention in order to scale systems that serve thousands of clients/custormers (I've found this web page that collects projects related to non-relational databases). However, most of the frameworks/PaaS for developing applications using this new logical storage force the developer to use a shared-nothing approach when developing applications (e.g. Google App. Engine and Mircrosoft's Azure). This means that no state can be stored in the application between two invocations from the same client.
Part of the work done on my PhD thesis was related to consistently scale stateful applications running in clusters of servers based on multi-tier architectures (in a LAN). We use transparent replication -in order to hide the clients from the complexity of the replicated architecture- and guarantee snapshot isolation for the data being accessed (stored in relational DBs). In his blog, Werner Vogels (Amazon's CTO), goes one step further and discusses about consistency in the context of stateful applications and the requirements of cloud computing. The Vogel's view on this topic has been applied in projects at Amazon such as Dynamo. Whilst our approach ensures strong consistency, Amazon's approach is eventually consistent.
Part of the work done on my PhD thesis was related to consistently scale stateful applications running in clusters of servers based on multi-tier architectures (in a LAN). We use transparent replication -in order to hide the clients from the complexity of the replicated architecture- and guarantee snapshot isolation for the data being accessed (stored in relational DBs). In his blog, Werner Vogels (Amazon's CTO), goes one step further and discusses about consistency in the context of stateful applications and the requirements of cloud computing. The Vogel's view on this topic has been applied in projects at Amazon such as Dynamo. Whilst our approach ensures strong consistency, Amazon's approach is eventually consistent.
Tuesday, December 22, 2009
The Emperor's New Clothes: Internet of Services (and Cloud Computing)
In a recent post I talked about cloud computing and I said that, at least in the European Union (EU), this trend is integrated in the Internet of Services (IoS) vision for Future Internet. But what is the Internet of Services?
For the time being, there is no specific entry for IoS in the Wikipedia. However, IoS appears embedded in the definition of Service-Oriented Architecture. The term appears unreferenced in a sentence included in the Web 2.0 section: "In an Internet of Services, all people, machines, and goods will have access via the network infrastructure of tomorrow." Wow! Now I understand why this sentence is unreferenced... From this sentence one can come to the conclusion that "some people" is interested in changing (maybe radically) the structure of the current Internet to integrate (almost) everything in this world. What a challenging endeavour!!! I'm jokin of course. What I want to mean is that peacock terms and this kind of vague sentences should be removed from the entries of the Wikipedia because they may be a source of confusions. They should be moved to blogs like this :-)
But, wait! Who would be brave enough to accomplish this kind of mission? Isn't Google acting on the IoS through Chome OS, Android or App. Engine and the plethora of new services they are offering? Amazon through EC2, giant datacenters and Kindle? Apple through iPhone/iTunes? Microsoft through Azure and XBox Live/Natal? Is the underlying technology and decissions made on these works radical enough to provide users and devices "the access via the network infrastructure of tomorrow"? Or all this stuff are just step-by-step refinements and improvements of already well-known concepts? Of course I think that these actors are providing new services (that can be either categorized under IoS or not, who knows...) but I bet for the second option. Up to now, nobody is offering services based on a radically different Internet infrastructure. But stop again! What about researchers proposing innovative Internet architectures? I think that radical changes are not allowed in such a big infrastructure controlled by nobody in particular and used by millions of people around the globe. I also bet here for incremental improvements in the network infrastructure. An analogy can be established when some time ago, in the early years of SOA, many people claimed that application servers were not ready for the web services revolution and new software systems would be needed. However, application servers still are here, alive and kicking, adapted to the new requirements of SOAs and supporting current many service-oriented applications...
For what I've found in the web, SAP is behind many of the references to the term "Internet of Services". In fact, I found the sentence discussed above in an early vision from SAP exposed in the IEEE WETICE workshop in 2007. During the last 3 years, SAP has been producing a lot of content regarding the Internet of Services concept through the Theseus research program and more specifically through the TEXO project. The SAP's vision of the IoS in Theseus seems an evolution of the enterprise SOA, "making services easy to implement, consume, and trade." To SAP, IoS also embeds Web 2.0 technologies (See also this article in IEEE IT Professional). But, what's new in this approach??? Once again, it seems that there is nothing new under the sun, at least at the conceptual level. In a more recent white paper presented to the EU, this view has been refined and includes the cloud computing paradigm, probably to accomodate it in the Future Internet initiative where the IoS is included. SAP also drives the Internet of Services Community. From this community, I've extracted some of the key points/requirements of IoS:
The second point is maybe the one that is more different from the current way of offering services, because it implies the alignment and collaboration of different items and actors (now taken into account or acting almost independently) in order to succeed when providing the different services.
Finally, the third and fourth points, reflect also practices that can be found in current software developments, namely, the seamless integration and accessibility to the information placed in any element of the IT world (services, applications, devices, etc.). This of course includes some of the ideas behind the Internet of Things and virtualization techniques.
In a research paper published in the Social Science Research Network, Man Sze-Li et al. describe IoS as "a fusion of ideas, technologies, practices and communities". The following list summarizes the main requirements for IoS found in this paper:
The second point, has to do again with the "radical" changes on infrastructures, architectures and platforms. But these radical changes seem to imply the use of technology-agnostic infrastructures, open environments, transparent discovery techniques, etc. which already were promoted by SOAs. This means that the changes may be not so radical. In fact, many of these requirements can be found in system architectures already used by Amazon, Facebook, etc.
On the other side of the Atlantic ocean, the IoS concept seems to be driven by the Services Science initiative. I've not explored in depth the concept of Service Science and its relationship with the IoS yet, but it seems that conceptually arose earlier (as usual) in the U.S than in Europe, promoted by IBM.
To summarize all I've written above, to me, Internet of Services (and this can also be applied to the Internet of Things) are the emperor's new clothes/buzzwords that not so technical people in Europe (e.g. politicians, stakeholders, high level gurus...) are wearing/using regarding the new services that will be deployed in the so-called (and vague concept of) Future Internet whilst cloud computing are the new clothes/buzzwords that techies are spreading everywhere they go to refer to the architecture at the core of IoS that will support the execution of those services. Maybe the use of the future form can be changed to the present continuous form in order to refer to the services that many companies in the U.S. are providing right now.
For the time being, there is no specific entry for IoS in the Wikipedia. However, IoS appears embedded in the definition of Service-Oriented Architecture. The term appears unreferenced in a sentence included in the Web 2.0 section: "In an Internet of Services, all people, machines, and goods will have access via the network infrastructure of tomorrow." Wow! Now I understand why this sentence is unreferenced... From this sentence one can come to the conclusion that "some people" is interested in changing (maybe radically) the structure of the current Internet to integrate (almost) everything in this world. What a challenging endeavour!!! I'm jokin of course. What I want to mean is that peacock terms and this kind of vague sentences should be removed from the entries of the Wikipedia because they may be a source of confusions. They should be moved to blogs like this :-)
But, wait! Who would be brave enough to accomplish this kind of mission? Isn't Google acting on the IoS through Chome OS, Android or App. Engine and the plethora of new services they are offering? Amazon through EC2, giant datacenters and Kindle? Apple through iPhone/iTunes? Microsoft through Azure and XBox Live/Natal? Is the underlying technology and decissions made on these works radical enough to provide users and devices "the access via the network infrastructure of tomorrow"? Or all this stuff are just step-by-step refinements and improvements of already well-known concepts? Of course I think that these actors are providing new services (that can be either categorized under IoS or not, who knows...) but I bet for the second option. Up to now, nobody is offering services based on a radically different Internet infrastructure. But stop again! What about researchers proposing innovative Internet architectures? I think that radical changes are not allowed in such a big infrastructure controlled by nobody in particular and used by millions of people around the globe. I also bet here for incremental improvements in the network infrastructure. An analogy can be established when some time ago, in the early years of SOA, many people claimed that application servers were not ready for the web services revolution and new software systems would be needed. However, application servers still are here, alive and kicking, adapted to the new requirements of SOAs and supporting current many service-oriented applications...
For what I've found in the web, SAP is behind many of the references to the term "Internet of Services". In fact, I found the sentence discussed above in an early vision from SAP exposed in the IEEE WETICE workshop in 2007. During the last 3 years, SAP has been producing a lot of content regarding the Internet of Services concept through the Theseus research program and more specifically through the TEXO project. The SAP's vision of the IoS in Theseus seems an evolution of the enterprise SOA, "making services easy to implement, consume, and trade." To SAP, IoS also embeds Web 2.0 technologies (See also this article in IEEE IT Professional). But, what's new in this approach??? Once again, it seems that there is nothing new under the sun, at least at the conceptual level. In a more recent white paper presented to the EU, this view has been refined and includes the cloud computing paradigm, probably to accomodate it in the Future Internet initiative where the IoS is included. SAP also drives the Internet of Services Community. From this community, I've extracted some of the key points/requirements of IoS:
- Allow to access services at Internet scale.
- Requires different perspectives (technical, economical…), stakeholders and interests to succeed.
- Current services, business processes and applications must be accessed seamlessly.
- Also seamless access to devices and materials is required.
The second point is maybe the one that is more different from the current way of offering services, because it implies the alignment and collaboration of different items and actors (now taken into account or acting almost independently) in order to succeed when providing the different services.
Finally, the third and fourth points, reflect also practices that can be found in current software developments, namely, the seamless integration and accessibility to the information placed in any element of the IT world (services, applications, devices, etc.). This of course includes some of the ideas behind the Internet of Things and virtualization techniques.
In a research paper published in the Social Science Research Network, Man Sze-Li et al. describe IoS as "a fusion of ideas, technologies, practices and communities". The following list summarizes the main requirements for IoS found in this paper:
- Accommodate billions of users, organization, software-based services and devices, impacting positively on them.
- IoS technology and services as a means to server the user needs (enablers of provision, consumption and pro-sumtion).
- Should enable seamless information, applications, services, networks, provisioning and usage.
- Produce radically new infrastructures, architectures and platforms.
- Based on a combination of emerging common and standard infrastructures.
- IoS services at infrastructure level must be commonly shared, discoverable transparently and capable of living in an open and dynamic environment.
- Open infrastructure that allows to grow by means of participatory input and not tied to particular technologies nor owned by any entity.
The second point, has to do again with the "radical" changes on infrastructures, architectures and platforms. But these radical changes seem to imply the use of technology-agnostic infrastructures, open environments, transparent discovery techniques, etc. which already were promoted by SOAs. This means that the changes may be not so radical. In fact, many of these requirements can be found in system architectures already used by Amazon, Facebook, etc.
On the other side of the Atlantic ocean, the IoS concept seems to be driven by the Services Science initiative. I've not explored in depth the concept of Service Science and its relationship with the IoS yet, but it seems that conceptually arose earlier (as usual) in the U.S than in Europe, promoted by IBM.
To summarize all I've written above, to me, Internet of Services (and this can also be applied to the Internet of Things) are the emperor's new clothes/buzzwords that not so technical people in Europe (e.g. politicians, stakeholders, high level gurus...) are wearing/using regarding the new services that will be deployed in the so-called (and vague concept of) Future Internet whilst cloud computing are the new clothes/buzzwords that techies are spreading everywhere they go to refer to the architecture at the core of IoS that will support the execution of those services. Maybe the use of the future form can be changed to the present continuous form in order to refer to the services that many companies in the U.S. are providing right now.
Thursday, November 5, 2009
Cloudy Architectures
At this time, everyone has heard about cloud computing. In here, I pretend to offer my humble vision about them.
Cloud computing can be included in a new vision for software services delivered via networks called Internet of Services (IoS). In summary, IoS aims at providing a new set of ground-breaking services at Internet scale to the end users (both citizens and enterprises). In Europe, the IoS vision is embedded joint with the vision of the Internet of Things (IoT) in the actions of the so-called Future Internet Assembly (FIA).
So, cloud computing can be seen as the spine for building the IoS. I define a cloud computing platform as "a hybrid hardware/software elastic multi-tenant infrastructure that supports the implementation and deployment of applications and services at Internet scale". I call it infrastructure because it combines both hardware and software architectures. It is elastic because one of the main objectives of cloud computing is to offer/adapt the resources used by an application to its current needs at a given time. Finally, it is multi-tenant because a single cloud infrastructure can run multiple applications from different users, enterprises, departments etc. with heterogeneous requirements.
Another objective of cloud computing is to save money to the enterprises. Using a cloud infrastructure, the enterprises pay for the use of the IT resources. For example, in this way, enterprises do not have to worry about investing a lot of money in IT infrastructures in order to provide good QoS to potential users when dealing with unexpected increasing workloads that may last only short periods. They may simply rent additional resources from the cloud provider in order to satisfy such peaks. Therefore, this can avoid the economical effects of overprovisioning their infrastructures or underestimating the success of certain businesses. Payments can be done by means of the utility computing model, where the user pays for the resources consumed (e.g. like paying our electricity bills), or based on a suscription period (e.g. when we subscribe for a tech magazine for a one year period).
These kind of infrastructures have mainly three different layers of abstraction that encompass different architectures, both hardware and software. These layers include other buzzwords "à la mode" these days: Software as a Service (SaaS), Platform as a Service (PaaS) and Infrastructure as a Service (IaaS).
At the top level of the infrastructure is the SaaS layer, that contains the enterprise/user applications running on the cloud. This layer uses the facilities provided by the PaaS layer in order to provide an elastic behaviour to applications with regard to their current needs (basically the current workload and desired QoS attributes, e.g. performance, availability etc.). Finally the IaaS layer provides a virtualized abstraction (software) of the physical resources (hardware) that conform the substrate of the cloud infrastructure.
One of the main concerns of cloud computing is its adaptation to different environments. However, clouds are comming in different flavours: public (accessible for general users, e.g. Facebook), private (devoted to enterprise environments, e.g. Salesforce.com) or hybrid (e.g. cloud applications deployed on Google App Engine, Microsoft Azure or Amazon’s EC2).
In here we are interested in the architectures that can be integrated in the PaaS layer, so we'll write an entry about it in the near future.
Other references about cloud computing are:
A Short Introduction to Cloud Platforms by David Chappel
Berkeley University vision of Cloud Computing
The Future of Cloud Computing article at Sys-Con Media
IBM Cloud Computing
Current providers of cloud computing solutions are:
Amazon Elastic Computing Cloud service (EC2) (IaaS/PaaS)
Google App. Engine (PaaS)
Microsoft's Azure platform (PaaS)
Salesforce.com and Force.com (SaaS/PaaS)
Heroku (PaaS)
SAP Business ByDesign (SaaS)
IBM's Cloud Services and Computing on Demand (SaaS/IaaS)
Rackspace (IaaS)
Other visions:
IBM vision of the cloud
Oracle in the cloud
The 451 Group vision
Cloud computing can be included in a new vision for software services delivered via networks called Internet of Services (IoS). In summary, IoS aims at providing a new set of ground-breaking services at Internet scale to the end users (both citizens and enterprises). In Europe, the IoS vision is embedded joint with the vision of the Internet of Things (IoT) in the actions of the so-called Future Internet Assembly (FIA).
So, cloud computing can be seen as the spine for building the IoS. I define a cloud computing platform as "a hybrid hardware/software elastic multi-tenant infrastructure that supports the implementation and deployment of applications and services at Internet scale". I call it infrastructure because it combines both hardware and software architectures. It is elastic because one of the main objectives of cloud computing is to offer/adapt the resources used by an application to its current needs at a given time. Finally, it is multi-tenant because a single cloud infrastructure can run multiple applications from different users, enterprises, departments etc. with heterogeneous requirements.
Another objective of cloud computing is to save money to the enterprises. Using a cloud infrastructure, the enterprises pay for the use of the IT resources. For example, in this way, enterprises do not have to worry about investing a lot of money in IT infrastructures in order to provide good QoS to potential users when dealing with unexpected increasing workloads that may last only short periods. They may simply rent additional resources from the cloud provider in order to satisfy such peaks. Therefore, this can avoid the economical effects of overprovisioning their infrastructures or underestimating the success of certain businesses. Payments can be done by means of the utility computing model, where the user pays for the resources consumed (e.g. like paying our electricity bills), or based on a suscription period (e.g. when we subscribe for a tech magazine for a one year period).
These kind of infrastructures have mainly three different layers of abstraction that encompass different architectures, both hardware and software. These layers include other buzzwords "à la mode" these days: Software as a Service (SaaS), Platform as a Service (PaaS) and Infrastructure as a Service (IaaS).
At the top level of the infrastructure is the SaaS layer, that contains the enterprise/user applications running on the cloud. This layer uses the facilities provided by the PaaS layer in order to provide an elastic behaviour to applications with regard to their current needs (basically the current workload and desired QoS attributes, e.g. performance, availability etc.). Finally the IaaS layer provides a virtualized abstraction (software) of the physical resources (hardware) that conform the substrate of the cloud infrastructure.
One of the main concerns of cloud computing is its adaptation to different environments. However, clouds are comming in different flavours: public (accessible for general users, e.g. Facebook), private (devoted to enterprise environments, e.g. Salesforce.com) or hybrid (e.g. cloud applications deployed on Google App Engine, Microsoft Azure or Amazon’s EC2).
In here we are interested in the architectures that can be integrated in the PaaS layer, so we'll write an entry about it in the near future.
Other references about cloud computing are:
A Short Introduction to Cloud Platforms by David Chappel
Berkeley University vision of Cloud Computing
The Future of Cloud Computing article at Sys-Con Media
IBM Cloud Computing
Current providers of cloud computing solutions are:
Amazon Elastic Computing Cloud service (EC2) (IaaS/PaaS)
Google App. Engine (PaaS)
Microsoft's Azure platform (PaaS)
Salesforce.com and Force.com (SaaS/PaaS)
Heroku (PaaS)
SAP Business ByDesign (SaaS)
IBM's Cloud Services and Computing on Demand (SaaS/IaaS)
Rackspace (IaaS)
Other visions:
IBM vision of the cloud
Oracle in the cloud
The 451 Group vision
Subscribe to:
Posts (Atom)
