Monthly Archives: March 2014

Avamar Backup Solution Review

I recently completed a hands-on training on Avamar management and was impressed by its deduplication technology. Deduplication happens at the client side which means less bandwidth consumed on the network and less space used for storage. Backup jobs run very fast, especially the subsequent backups after the initial baseline backup.

The Avamar disk-based backup appliance is based on Linux operating system, thus its Command Line Interface (CLI) commands are excellent. Its Redundant Array of Independent Nodes or RAIN architecture provides failover and fault tolerance across its storage nodes. It can also integrate with Data Domain as its backend storage nodes. It has an intensive support for VMware and NAS appliances via the NDMP accelerator device. Avamar servers can be replicated to other Avamar servers located at a disaster recovery site. The management GUI is intuitive for the most part and it’s very easy to do backup and restore.

However, I also found several shortcomings that I think could improve the product. First, the management GUI does not have an integrated tool to push the agent software to the clients. If you have hundreds of clients, you need to rely on third party tools such as Microsoft SMS to push the agent software. Second, there is no integrated management GUI. You have to run several tools to perform management tasks – the Avamar Administrator Console, Avamar Client Manager, Enterprise Manager, and Backup Recovery Manager. Third, there is no extensive support for Bare Metal Restore (BMR). Only Windows 2008 and later are supported for BMR. Finally, the system requires a daily maintenance window to perform its HFS checks and other tasks, during which very few backup jobs are allowed to run. This should not be a big deal though since a short backup window is usually enough to finish backup jobs because as I mentioned earlier, backups run very fast.

Overall, I consider Avamar coupled with the Data Domain appliance as the leading backup solution out there.

IT Infrastructure Qualification and Compliance

One of the requirements of building and operating an IT infrastructure in a highly regulated industry (such as the pharmaceutical industry, which is regulated by the FDA) is to qualify, or validate the servers, network, and storage when they are being built. Once built, any changes to the infrastructure should undergo a change control procedure.

Building the infrastructure and making changes to it should undergo verification. They should also be documented so that they can be easily managed and traced. These activities are really not that different from the best practices guide in operating an IT infrastructure, or even from the ITIL processes.

FDA does not dictate how to perform IT infrastructure qualification or validation, as long as you have documented reasonable procedures.

The problem is that some companies overdo validation and change control processes. The common problems I’ve seen are: (1) too many signatures required to make a change, (2) no automated procedure to perform the documentation – many still use papers to route documents (3) and finally, the people who perform the checks and balances sometimes do not understand the technology.

The result is that IT personnel get overwhelmed with paperwork and bureaucracy. This discourages them to make critical changes to the infrastructure such as performing security patches on time. This also leads to the relunctance of IT personnel to implement newer or leading edge technologies into their infrastructure.

Fortunately, the International Society for Pharmaceutical Engineering (ISPE) has published a Good Automated Manufacturing Practice (GAMP) guidance on IT Infrastructure Control and Compliance. Companies can create their own IT infrastructure qualification program and procedures based on the GAMP guidance document. They should be simple but comprehensive enough to cover all the bases. It is also important that these procedures be periodically reviewed and streamlined to achieve an optimized procedure.