Embrace Parallelism with Virtual Machines
Software architects and designers have been spoilt by the scale up in processor speeds over the years thanks to Moore's Law. But, that's all changing at a very fast pace today. Today's hardware designs (multicore based processors) require software to be designed with parallelism built into the system for performance scaling over the next generation hardware (with the promise of multiplying cores) . Herb Sutter has very beautifully explained the reasons for the need for change in software design strategy in his famous article "The Free Lunch is Over" - published sometime in 2005.
Let us consider software system that is running on a given server on modern multicore hardware. The system is considered to be well scaled if it is able to spawn enough threads to keep all the available cores busy and hence speeds up execution (for now, let's ignore specifics like overhead of thread management as against the benefit of parallelized execution etc.). Is there a limit to the ability to scale here ? Yes and it is the number of cores that are available on the server. Ideally, you would want to be able to dynamically make more and more cores available to be able to consume the tasks that are parallelizable but waiting for a core to be freed - but then you are limited by the number of cores available.
Moving the above problem to a wider canvas on a larger scale (the cloud computing environment for example), let's understand that software that can scale across multiple servers exhibits parallelism in it's own way. In this context, the processing element is a 'server' as against a 'core'. Ofcourse, making more and more server machines available to feed the software's hunger for parallelism is easier said that done (considering the high costs of server procurement, server management overhead, power management etc.) - unless you have considered virtualization.
Virtualization solutions to such problems would involve having the ability to to distribute incoming requests across a number of virtual machines - instead of physical servers. Such solutions have the ability to generate additional processing elements (virtual machines) based on increasing workload so that all available parallelizable tasks are being catered to. Besides, with reduced workloads, unused virtual machines can be made dormant thus saving on power costs etc . Notice, the ability to truly scale efficiently is not hindered by processor configurations (as seen in the standalone software system illustration above). Considerations along these lines are probably important considerations in architecting cloud computing solutions.
The concept of a 'processing element' in parallel programming patterns- which was usually a processing core - can now be extended to include virtual machines too . With that, it becomes easier to relate common parallel design patterns like Task Decomposition, Data Decomposition etc. to solving large problems using virtual machines. Embracing parallelism with virtual machines is a reality today.
