The Infosys Labs research blog tracks trends in technology with a focus on applied research in Information and Communication Technology (ICT)

« Microsoft C++ AMP is now 'Open' | Main | Is Parallel Computing a Rocket Science or Esoteric? Part 1 »

Hello Parallel Computing!

Computing applications are popularly serial in nature, meaning the program is designed to run on a single computer. Since they are run on a single machine, at any given instance there is just one programming instruction that is executed. Thus, clearly the instructions in a serial program are executed one at a time and one after the other, in the sequence in which they appear in the program. Given today's parallel infrastructure, it is a shame to run such serial programs on this powerful hardware. Clearly the hardware is quite underutilized. And this is exactly what parallel computing solves. To simply define, parallel computing allows more than one instruction to be concurrently executed and hence is able to use the dedicated computing resources effectively. So, for example, if a parallel program is run on a quad-core processor, it is possible to execute 4 simultaneous instructions at any given instance in time. A serial program, on the other hand, will only be able to utilize one of the 4 cores at any instant.
Now that I have (hopefully) your attention, let me pop a question. So is parallel computing as complex as it sounds to be? And, unfortunately, the answer is yes. It is a hard job to design and develop an error-free parallel program. Hard because thinking in parallel and designing parallel programs is not something most are trained for and also because this process takes a good amount of time.
In the world of HPC, parallel computing is a must in order to use computing resources like multi-core CPUs, GPUs and other accelerators. And, the program design varies from one computing resource to another.  It is because the computing resources can be classified to work best for a particular class of parallelism. On a high level, parallelism can be classified as Task Parallel and Data Parallel. To be able to design a well optimized parallel program, it will be essential to identify whether the problem is task or data parallel or is a mix of both. The next step will be to identify the appropriate computing resource. A data parallel program is well suited for a GPU. On the other hand, the CPU is best for task parallel programs. With the decision on the hardware made, comes the next step of actually designing the parallel program, developing this using the appropriate programming model and running this to check for correctness. This is a mammoth step and an important achievement. And then it comes to the star step, to quench the thirst for speed, it is essential to fine tune or optimize the program to achieve the much sought after speed up.
Things get really exciting in my side of the world, the HPC world. Keep watch to read about ways to solve the mysteries of parallel computing.

Post a comment

(If you haven't left a comment here before, you may need to be approved by the site owner before your comment will appear. Until then, it won't appear on the entry. Thanks for waiting.)

Please key in the two words you see in the box to validate your identity as an authentic user and reduce spam.

Subscribe to this blog's feed

Follow us on