Monthly Archives: June 2014

Installing High Performance Computing Cluster

A high performance computing (HPC) cluster is usually needed to analyze data from scientific instruments. For instance, I recently setup an HPC cluster using Red Hat Enterprise Linux 6.5 consisting of several nodes which will be used to analyze data generated from a gene sequencer machine.

Basically, to build the cluster, you need several machines with high speed processors and multiple cores, lots of memory, a high speed network to connect the nodes, and a huge and fast data storage. You also need to install an operating system – such as the Red Hat or CentOS Linux, and configure tools and utilities such as kickstart, ssh, NFS, and NIS. Finally, a cluster software or queueing system is needed to manage jobs to fully utilize the compute resources. One of the commonly used open source cluster software is Son of Grid Engine (SGE)  – an offshoot of the popular Sun Grid Engine.

An excellent write up for setting up an HPC cluster can be found at this Admin article.

The latest Son of Grid Engine version (as of this writing) is 8.1.7 and can be downloaded from the Son of Grid Engine Project Site.

Since the environment I setup is running Red Hat Enterprise Linux 6.5, I downloaded and installed the following rpms:

gridengine-8.1.7-1.el6.x86_64.rpm
gridengine-execd-8.1.7-1.el6.x86_64.rpm
gridengine-qmaster-8.1.7-1.el6.x86_64.rpm
gridengine-qmon-8.1.7-1.el6.x86_64.rpm

After the installation of the rpms, I installed and configured the qmaster, then installed sge (execd) on all the nodes. I also ran a simple test to verify that the cluster is working by issuing the following commands:

$ qsub /opt/sge/examples/jobs/simple.sh
$ qstat

MIT Sloan CIO Symposium

I recently attended the MIT Sloan CIO Symposium held in Cambridge, MA on May 21st, 2014. The event was well attended by Chief Information Officers (CIOs), VPs, business executives, academics, entrepreneurs, and professionals from companies all over the world. The speaker lineup was superb, the symposium was content rich, and the #mitcio twitter hashtag was trending during the event.

I enjoyed the symposium because of the combination of academic and business perspective coming from a diverse set of speakers. Hot topics such as security, big data, robotics and self-driving cars and its implications to society, and the evolving role of CIOs were big topics of conversation.

The key takeaways for me are the following:

1. The future role of CIOs and IT professionals in general will be service brokers. They will increasingly serve as in-house IT service providers, and as brokers for business managers and external cloud service providers.

2. On the issue of “build vs buy, and when does it make sense to build your own system”, the answer is — when it is a source for your competitive advantage, or when what you build will differentiate your business from anyone else.

3. CIOs increasingly have to work closely with the business to deliver on technology promises rather than focusing on the technology alone. They should have a seat at the executive table. CIOs need to stay in front of their organizations and should talk to boards regularly. They should be communicating the risks of IT investments and demonstrate its benefit to the business.

4. To maximize and communicate the business value of IT, use the following sentence when explaining the benefits of IT to business: “We are going to do ___, to make ___ better, as measured by ___, and it is worth ____.” Also, consider “you” and “your” as the most important words when translating the business value of IT.

5. In terms of the future of technology – everything is becoming data-fied. Brynjolfsson, the author of the book, “The Second Machine Age: Work, Progress, and Prosperity in a Time of Brilliant Technologies” said that “We are at the cusp of a 10 year period where we go from machines not really understanding us to being able to.” Thus we are seeing tremendous technological advancements in robotics and self-driving cars. With all these technological progress, we also have to think about how our culture, laws, ethics, and economics will be affected. For instance, how will employment be affected by robots that can generally do repetitive tasks? The advice from the panel is that “creative lifelong learners will always be in demand.”