ESP – Product Engineering – Platform and Software Products

When I first heard about Server Virtualization, my first thought was, “Why did nobody think about this earlier”. Of course, I later learnt that the idea had been around for more than 4 decades. But then, why did it go into hibernation?

IBM introduced the concept of virtualization at a time when computers didn’t mean anything other than mainframes. It was much later that x86 and the PC revolutionized the world. So while the computer became smaller, cheaper, faster and more accessible, the very need for Virtualization disappeared. After all, why have Virtual machines when you can have ‘real’ ones?

But the exponential growth in computing capabilities (hardware processing power) has today resulted in heavily underutilized machines. Couple this with the concerns over rising carbon emissions (power consumption and cooling needs) and the increasing complexity in managing numerous servers in the data center; …and the stage was set for Virtualized machines to make a comeback.

There was but one small problem.

The x86 architecture which is predominant today, was not designed with virtualization in mind. Certain sensitive instructions expected the Operating System to be directly interacting with the hardware, making it difficult to realize ‘true’ Virtualization (as defined by the Popek and Goldberg criteria).

VMware was the first to find a way around this obstacle, by way of a method called the Binary Translation in which the privileged instructions were trapped by a software interpreter layer known as the Virtual Machine Monitor (VMM), and converted to ‘safe’ instructions that could be virtualized.

In the two approaches to Server Virtualization that I tried to describe in my previous post,  the guest OS required no changes and was quite unaware if it was sharing the hardware resources with other co-located virtual machines. The VMMs in this case provides the guest OS with a complete emulation of the underlying hardware. For this reason, this kind of virtualization is also referred to as ‘Full Virtualization’.

While the VMMs appears to solve the entire problem with regard to virtualized machines, it does bring in some level of performance degradation, caused by the extra processing (in terms of the ‘instruction translation’) that the hypervisor has to do.  This can be addressed if the original problems with the x86 architecture could be resolved…either in [1] OS (software) or in [2] hardware.

1) Paravirtualization:
The first ‘solution’ (in software) is what Paravirtualization tries to achieve.
In Paravirtualization, the Guest OS kernel is modified by replacing the non-virtualizable instructions with safe API ‘hooks’ into the hypervisor. In simple terms, this does away with the hypervisors botheration about translating these instructions; thus improving performance and efficiency.

The Xen Hypervisor open source project is one of the earliest proponents of Paravirtualization.

Paravirtualization is not ‘Full virtualization’ since it involves modifications to the Guest OS. So while it does bring about a significant improvement in terms of virtualization overhead, it has its limitations when it comes to compatibility and portability. For instance unmodified commercial Operating systems (like Windows Server 2003) cannot be supported by Paravirtualization techniques.

Since it involves modifications to the OS, Paravirtualization is also sometimes referred to as OS-assisted Virtualization.

2) Hardware- Assisted Virtualization:
Intel and AMD have been working on trying to address the virtualization limitations of the x86 architecture; and have come up with the Intel-VT and AMD-V respectively; extensions that provide virtualization support on some of their processors. As opposed to Paravirtualization (OS-assisted) or Binary Translation (Hypervisor-assisted), in Hardware-assisted virtualization the privileged instructions are now trapped and emulated in the hardware directly.

The advantage to this technique is that, OS kernels no longer need to be tweaked (as in Paravirtualization) and can run as-is. At the same time, the hypervisor also does not need to be involved in the inefficient Binary Translation of the sensitive instructions. Thus not only does Hardware-assisted Virtualization satisfy the Popek and Goldberg criteria (of Full Virtualization) it also provides improved performance.

However, Hardware-Assisted Virtualization is still in its early days and not all of its promises have yet been realized to an extent to render Binary Translation or Paravirtualization techniques obsolete.

 

Posted by Randhir I B at 07:43 AM | Permalink | Comments (0) | TrackBacks (0)

The formal interpretation of Moore's law - as regards transistor divisions on a silicon chip - has been reaching its stretched limits. However, the advent of multicore based processor design has opened up a horizon of opportunities for those applications which are designed to exploit available hardware parallelism. However, existing applications are also expected to run faster as they get near dedicated attention on individual cores as against the case on a single processor single core machine where threads had to context-switch based on slicing. Today, operating systems are aware of the topology of the processors on which they run and hence are designed to schedule application execution in a way to maximize benefits. Hence, applications designed to exploit available hardware parallelism fare even better.
Before the advent of multicore, performance betterments were introduced by increasing the density of transistors on silicon chips - but this came at the cost of increase in heat. In a data center, the problem was magnified because of the presence of very densely packed blade and rack servers. Add to that was the cost of energy required in cooling units to cool these servers.
Multicore based processors are designed to provide favorable cost advantages by providing better performance per watt and reduced energy requirements. Couple this with all the advantages that virtualization technologies provide by way of server consolidation - and now you know where CIOs and CTOs of companies with large data centers are looking.
In virtualized systems, hypervisors are designed to abstract out the hardware and provide virtual processors to each of the virtual machines running on top of them. However, it is known that there is some form of scheduling implemented in the hypervisor to determine which virtual machine would currently execute its instruction on the actual processor. With multicore bringing in the advantages of performance, energy savings and favorable price equations, virtual machines will be able to run on as dedicated a CPU as possible thus in turn passing on the performance/execution benefits to the applications running within them. The point to note here is that the power expended is lower with higher performance benefits.
Executing applications in silos on advanced multicore based multi-processor hardware will not be advantageous because there are very few applications that can leverage 4-8 processors (besides multiple cores within them). Applications running within virtual machines on multicore based hardware will help achieve expansion of workload with no performance decrements.
VMware’s ESX Server 3.0 is known to present multiple virtual CPUs to applications running within a virtual machine to exploit available resources in an SMP machine. Hence as an extension, I would assume that today's hypervisors would provide extend the benefits of multiple cores to an application running within a virtual machine so that parallelism implemented within the application is truly exploited even in a virtual machine environment. If this were so, then that would provide a heady mix of high power and cost savings with increased performance benefits in data centers.

Posted by Suraj Nair at 10:53 AM | Permalink | Comments (0) | TrackBacks (0)

Have you ever got involved in an Internationalization related project and faced a very basic question, “Which encoding should I use in my application?” I have come across this question several times and have realized that I still don’t have a simple and straightforward answer to this question. There are a lot of factors which need to be considered while going with a particular encoding and that choice might be the crucial factor which determines the success or failure of the project. Choosing the wrong encoding for the product, might result in severe performance issues which might ultimately cost a lot in terms of rework, delayed product launches or even loss in market share.

The evolution of character encodings started with the introduction of ASCII way back in 1963. This was a 7 bit encoding scheme and was quite convenient for representing letters, numerals, symbols etc. Around the same time IBM came up with its own EBCDIC system which was an 8-bit encoding scheme. Things worked for a while, but then the shortcomings of these encodings lead to the introduction of a lot of ad-hoc encodings which were sometimes language or location specific. Japan came up with the ‘JIS’ encoding schemes, Korea and Chinese came up with their own encodings, ISO-8859 was adopted by America, Europe and the Middle east etc. Big corporations like Microsoft and Apple introduced their own proprietary encodings. Exchanging data between different locales and geographies became a nightmare because of encoding issues. Unicode in a way tried to simplify things by becoming a superset of all the available encodings, but has also created some amount of confusion in terms UTF-8, UTF-16 and UTF-32. I’m sure everyone who has done sufficient reading on the various encodings available, will agree that Unicode is the way to go in order to ensure that we are following the same standards across geographies. But making the choice between UTF-8, UTF-16 and UTF-32 poses a challenge in itself.

While UTF-32 is perhaps the simplest way to store Unicode data, it is also the most expensive in terms of memory usage. For network intensive or storage applications, the impact on network data transfer and disk usage will be high. There is also the issue of byte-ordering while transmitting data between two machines. So even though it makes life simple for the programmer, this is not a preferred choice as far as Unicode encodings go, however memory is is not a concern for you, go with UTF-32.

Almost always, the choice has to be made between UTF-16 and UTF-8. UTF-16 also faces the byte-ordering issue, but it is still a very popular encoding because it is fixed width (for all practical purposes, but can be variable width by using surrogates). The native character type in Java and C# is 16-bit, which makes UTF-16 a good candidate for applications written in those languages. C/C++ users face an issue because UTF-16 is not the same as wchar_t on every platform. UNIX defines wchar_t as a 32-bit data type while Windows systems define it as 16-bits. This causes encoding issues in applications which involves communication between Windows and UNIX machines. UTF-16 is efficient when it comes to storing Asian characters since most of them fit into 2 bytes; however there is a lot of space wastage when UTF-16 is used to store ASCII characters which require just one byte. Hence the choice of encoding is also dictated by the locales which need to be supported.

UTF-8 is a variable width encoding ranging from 1 to 4 bytes and is backward compatible with ASCII. It also has wide support from the XML community. The big advantage of using UTF-8 over UTF-16 and UTF-32 is that it does not have any byte-ordering issues. Another advantage is that most string processing functions which work on bytes rather than characters work fine with UTF-8 encoding. So the amount of code change is minimized. However for some languages UTF-8 encoded data might take more space than the corresponding multi-byte encoding. Character indexing will be an issue with UTF-8 and special handling is required.

All 3 encodings have their pros and cons and none is really better than the other. The choice of encoding will depend on the programming language used, the locales and languages to be supported, the amount and kind of text processing involved in the application, the deployment scenarios i.e. whether there is going to be data exchange between 2 machines, performance baselines, disk space usage, the version of Unicode required etc. All these factors have to be considered in the analysis phase and the choice of encoding must be one of the inputs for the design phase. Needless to say, the penalty may be huge if a wrong encoding is selected in the early phases of the project life cycle.

If you are an I18N architect, I invite you to share your views on the choice of encoding and the factors which influence you to make that choice.

Posted by Aviraj Singh at 12:13 PM | Permalink | Comments (0) | TrackBacks (0)

Is it just me, or is there a direct correlation between a technology’s position on the hype curve at any given point of time and the prevalent confusion around the terms that abound describing it? Virtualization, as IBM defined it in the 1960’s, seemed rather simple, but today it can mean a lot of things.

To someone who’s relatively new to the world of virtualization; some of the terms that have come to describe the various approaches to server virtualization can be pretty daunting if not outright confusing.

How is ‘Hosted Virtualization’ different from ‘Bare-metal Virtualization’? What then is ‘Paravirtualization’? What is a hypervisor? Is ‘Hardware Emulation’ some kind of virtualization technique? And which of these (if at all) can be considered as ‘Full Virtualization’?
Wait! Did I hear you say ‘hardware-assisted virtualization’ and ‘OS-assisted virtualization’?

So, are there really so many different approaches to server virtualization? 

I’ll attempt to define some of the above terminologies in a simple manner while also discussing the different approaches to server virtualization. However, it needs to be kept in mind that some of these terms are popularized by certain vendors and do not have a common industry-wide acceptance. Plus, many of the terms are used rather loosely and interchangeably (which is why they are confusing).

Virtualization is nothing but an abstraction that gives the illusion of resources that are not physically present. From the above simplified illustration of a typical computer, there are two potential regions where we can ‘insert’ this abstraction layer to create a virtualized environment… (1) Between the applications and the OS, or (2) between the OS and hardware.
Let’s take a look at both of these.

1) Hosted Virtualization.
When the abstraction layer is provided by the Operating system itself, such that it allows other OS’s  to reside within; thus creating virtual machines (VM’s) within an existing OS, we call the underlying OS as the Host OS and the OS instances within each of the VM’s as the guest OS. This architecture then can be called as Hosted Virtualization (since an OS is ‘hosting’ it), as depicted below.

The Virtualization Layer in the above figure contains the software needed for hosting and managing the virtual machines. This Layer is usually called the Hypervisor (in this particular case of Hosted Virtualization it’s called Type 2 Hypervisor). The exact functionality of the hypervisor can vary based on the different architectures from different vendors.

As would be obvious by now, this approach requires the Guest Operating System to rely on the underlying Host OS for access to the h/w resources. In many instances the Host OS provides the drivers for accessing the h/w; and thus the type of Guest OS that can be supported is thus determined by the Host OS’s capabilities.
Some of the popular examples of Hosted Virtualization (with Type 2 Hypervisor) are Microsoft Virtual PC and Virtual Server and VMware Workstation and VMware Server.

Containers:
In a particular case of Hosted Virtualization, the Hypervisor can have very limited functionality, relying on the Host OS for CPU scheduling and Memory Management. In such cases the Hypervisor is called a container and the Virtualization type is called Container virtualization. It can be argued though that this is not Virtualization in the strict sense of the term; rather a technique that just allows for machine consolidation. In containers, usually the Guest OS has to be exactly the same as the Host OS (even down to the version and patches). This type of virtualization is also sometimes referred to as OS-level Virtualization.
Examples of Containers (or OS-level Virtualization) are Parallels Virtuozzo, OpenVZ and Solaris Containers.

Hosted Virtualization provides the flexibility and ease of use. One can install the virtualization product (which adds the Virtualization layer on top of the Host OS) and add the VM’s. When used solely for server consolidation purposes this is an ideal approach (esp., as containers). Typical scenarios can be running multiple web servers in their own VM’s (all are usually on the same kind of OS).
On the negative side; with an added Hypervisor on top of the Guest OS, the overheads can be significant. Also since the Guest OS relies on the Host for driver support, not all OS’s might be supported by a given Virtualization product.

2) Bare-metal Virtualization
Now consider the case, where the abstraction layer sits directly on the H/W and all the other blocks reside on top of it.

 

From the above figure, it’s obvious that we now no longer need a Host OS. Again, the Virtualization Layer is called a Hypervisor…in this case Type 1 Hypervisor; and contains functionalities like CPU scheduling and Memory Management.  Even though there is no Host OS, usually one of the Virtual Machine has certain privileged status (Control/Admin/Parent VM).

Also, depending on the architecture, the hypervisor may either contain the drivers for the h/w resources (referred to as a Monolithic hypervisor) or the drivers may be retained at the Guest OS level itself (in which case it can be called a Microkernelized Hypervisor).

This type of Virtualization Approach (without any Host OS) is called Bare-metal Virtualization.  This approach is also sometimes referred to as Hardware Emulation, because each of the VM is provided an emulated Hardware view by the hypervisor.
Examples of this architecture are Microsoft Hyper-V, VMware ESX Server and Xen Hypervisor.

The advantage of Bare-metal Virtualization is that since a thin Hypervisor resides directly on the hardware the overheads are substantially low. It also uses fewer resources (no separate CPU cycles or memory footprint as in the case of a full-fledged Host OS).
The disadvantage of this model is that there is still dependency on the Hypervisor for the drivers (at least in case of the Monolithic Hypervisor). Also (as I shall discuss in my next post) most implementations of the bare-metal approach require specific virtualization support at the Hardware level (“Hardware-assisted”).

-------------------------
In the next post I’ll try to discuss about some of the other terms like Paravirtualization and hardware assisted virtualization.

 

 

 

 

Posted by Randhir I B at 04:37 AM | Permalink | Comments (0) | TrackBacks (0)

Achieving Multi-tenancy aspects in a SaaS solution

In the earlier blog (Part-1), we had discussed the benefits of the SaaS model from both the customers (consumers) and from ISVs viewpoints. There are quite a few successful SaaS model based solutions from various vendors, of which Salesforce.com is a leading example.  But, it is also evident that a number of failures in ISVs/Hosted companies occur due to the inability to handle the necessary aspects of scalability, configurability, integration etc which result in huge implementation and operational cost. In this blog post, I will touch upon ‘Multi-Tenancy’ – a key architectural consideration in a SaaS solution – and cover various consideration points, technologies available and best practices .

Multi-tenancy at Data layer:

Multi-tenancy is a unique requirements imposed when considering a SaaS solution. Multi-tenancy is a mechanism to provide tenant specific views and features which enable customers to configure the SaaS solution as per tenant needs using a shared hosting data center and software. There will be need of multi-tenancy at different levels such as UI (User Interface), Business Layer and Data Layer.

Multi-tenancy at different layers of an application

Data layer –  Segregating data of different tenants and runtime collection of data based on the tenant context

Business layer – Rules/logic segregation at the business logic layer - Creation of a pool of tenant independent services which can be easily aggregated to serve any client

Presentation layer – Need to support teneant specific looks and feel and validation etc

In the Data layer the important consideration points include Economic considerations, Security considerations, Tenant considerations, Regulatory considerations and any localized/region specific considerations. Once analysis is performed with respect to these considerations, one can check on the different options available at the database level for implementing SaaS solutions such as separate DB for each tenant, separate schema for each tenant or Shared schema (which is the top maturity level). The right approach to choose the DB options has to be selected after thorough analysis of various consideration points such as time to market, marketing requirements such as number of customers expected and level of variations required from each tenant etc. 

Achieving Multi-tenancy with configurability at Application Layer

Configurability is very critical for a SaaS solution as there is no opportunity for customization at the UI layer (because of varying data fields or workflow) or at business services layer to support varying tenant-specific business rules.  To handle this, it is important to segregate areas of the solution which involve static components from  those that involve dynamic components (what I meant by components here is nothing but implementation of business services/needs in an applications).

Important points of consideration in the UI layer include tenant specific looks and feel, navigation changes as required, data validation changes, globalization support etc. Similarly consideration points for the Business layer include varying business rules, varying processing of data/flow as per region or locations etc.

It is important to leverage the right design patterns and techniques available to achieve configurability and flexibility at UI and Business logic layer. Some of the options available to achieve configurability at UI level include adaptive UI rendering, metadata driven UI rendering etc. These concepts  can be extended to the business logic layer implementation also through these configurable metadata properties from external metadata XML (metadata xml is the one which contains all properties related to application properties to handle customer specific customizations).

Reporting and other supporting features:

In addition to the areas mentioned above (considerations in the UI, App layer and Data layer), another important area is with regard to providing the flexibility to get tenant specific reports (one option to achieve this could be by using various templates on considering different consideration points). It is also important to consider how to provide flexibility in other NFR related features such as archival, data retention period etc.. There are chances that non consideration of the points discussed here might lead into customization which is not expected in a SaaS solution.

We shall discuss on other architectural areas such as integration challenges, scalability and security in the next blog.

Posted by Krishna Markande at 09:16 AM | Permalink | Comments (0) | TrackBacks (0)