Virtualization and Validation

A collision between technology and regulation is fast approaching.

As pharmaceutical, biotech and device companies – and their suppliers – look for ways to cut costs, technology is leaping to the forefront. Leading the pack is the idea of outsourcing data centers to vendors using computer virtualization. Stumbling along in the opposite direction is last century’s 21 CFR Part 11 and all of its costly misinterpretations.

Computer Virtualization

Computer virtualization has many different meanings. At its narrowest sense, one physical computer runs different operations under different software systems (such as your production line monitoring software, your email software, your word processing software, and so on all on the same computer). Each piece of software thinks it has the computer to itself. Virtualization can also be much broader, spreading your software over many different computers connected across different regions and time zones all around the world (i.e., “cloud computing”). From a cost savings perspective, virtualization is loved by chief financial officers as it reduces costs by 30% under its narrowest use and up to much more dramatic cost reductions near 80% if you rely upon the broader sense of virtualization.

Today, only 15-20% of companies embrace virtualization. Technology analysts expect data center virtualization to be adopted by more than 60% of companies world-wide by 2013, driven in large part by economic pressure. While virtualization may save money for pharma and device companies, especially those outsourcing their computer departments, the business risks from a compliance standpoint are very real. Virtualization is complex. Because so much can be spread in little bits and pieces across so many computers and networks (or all combined onto one computer), any single tiny, little change may have significant, unanticipated downstream impact. One of the closest analogies may be the way the internet works – and can break down.

Dealing with One Critical Virtualization Risk

Think about the way you access the internet today. When you start up your internet browser and go to a website like Google or Cerulean Associates, the pathway taken by your computer to show you that site goes through your company’s network or, when you are home, through your telephone or cable company’s network. If the wrong tiny, little switch is turned off somewhere in the vast telephone or cable network, you won’t be able to access the internet or maybe just half of the websites based on the East Coast of the US. Google and the millions of internet websites are still there, you just cannot get to them. The same vulnerability holds true when you virtualize your data center. Your company’s software, its production data, and so on is all still there – spread across a vast number of computers and networks (or all crammed onto one) – but any little glitch may cut off your access to it or, in the worst case, destroy some of that information. Information loss may be a minor irritant when trading emails with your friends, but the FDA does not smile kindly on companies that cannot produce production data.

Just as you manage your risk of accessing the internet at home by signing a contract with the professionals (e.g., your telephone or cable company) to handle the hook-ups, access rights and connection availability, so you should let those technology vendors that specialize in virtualization deal with all the network infrastructure and computer systems involved. You then focus on managing the risk of non-compliance with regulations. And therein lies the catch.

Virtualization is an advanced technology use that needs advanced regulatory interpretations. The slow pace of legislative and regulatory change provides a significant mismatch between the complexities of the fast-growing virtualization trend and the costly “validate everything” of 1997’s Part 11.

Part 11 Revised

In talking with officials at the FDA in preparation for my compliance seminar last year on revisions to Part 11 and the EU’s Annex 11, it became clear that the work of the FDA Part 11 revision group is largely complete. As I mentioned to seminar attendees, and in my May SmarterCompliance newsletter, the FDA’s revisions to 21 CFR Part 11 have been awaiting final center approval before they can be published.

For pharmaceutical companies looking at virtualization, a revised Part 11 will be just the change needed to avoid a headlong collision of technology and regulation. Details of the revised Part 11 and how to prepare your company are beyond the scope of this column; you can get the information, strategies and reference materials from the recorded version of my seminar, Understanding and Implementing the Revised FDA Part 11 and EU Annex 11.

Given the revised Part 11, its intent and its new focus, how then to tackle the compliance challenges inherent in virtualization and still save all that money?

Tackling Virtualization and Validation

As my clients recognize, the solution lies in moving away from a spotlight on the computer toward a focus on controlling outputs. Technology – whether a computer or a virtualized data center – is just a tool, a means to an end. That end is an electronic record that is “attributable, legible, contemporaneous, original, and accurate” (Dr. Stephen Wilson, Deputy Director, CDER, FDA, FDA Regulatory Perspective: Data Integrity, May 2006). This is the pathway to adopting good technology and good compliance.

To achieve success with virtualization and compliance, there are four key steps to take:

Homework. Do your homework on the type of technology outsourced provider you want. I’ve written a very popular article based on my own experiences years ago as a biotech and device executive trying to find good consultants and outsource providers. Follow the steps in that article (or in some of the others I’ve written that touch upon similar themes, “Cost-Effective IT Outsourcing” or “SMB Validation: Four Ways to do More for Less”) to pick the right vendor for your company.
Quality / Technical Agreement. Craft a quality or technical agreement with the virtualization vendor that identifies your minimum expectations in terms of monitoring, reporting, security, up-time (i.e., availability) and backups. To identify realistic expectations, have your computer department research out typical levels for each of those categories (for instance, average up-time expectation email service might be 98.4%). Then, conduct a risk analysis, assessing the risk to the product, the patient and your compliance for that level of service. Be prepared to pay more if you want rates of service higher than typical.
Independent Controls. Include in your agreement the ability to conduct independent verifications – either yourself or by hiring an independent mock FDA auditor – of the vendor’s controls around the virtualized data center, the security of your information and access to your stored or archived data. Electronic data is most vulnerable sitting in storage – whether on a computer disk or backed up onto tape. Work with an independent consultant who has experience in both Part 11 and FDA records management requirements to craft a set of control points and check-ins to conduct during the course of your contract with the virtualization vendor.
Polices and SOPs. You need to complete your management of the risks of virtualization with a strong policy and procedural framework. Here too you may find it advantageous to work with someone with both an IT and records management background – particularly if that individual has had to deal with records and litigation support; few things will give you a better sense of what to reasonably expect when it comes to controlling electronic information and data integrity than an experience or two justifying to a skeptical lawyer why some records were kept and some were destroyed. Practical policies and standard operating procedures need to be written, trained and enforced. The independent auditor you use to help you monitor or conduct due diligence on your virtualization vendor should make sure to incorporate of a review of the vendor’s records control policies and procedures as well.

With these four tactics, virtualization compliant with the revised Part 11 can cut costs and lower risk. This, in turn, can help you speed drugs to market and pass some of the cost savings onto consumers, creating a “win-win-win” for everyone from shareholders and investors, to the regulators and the public.

Final Thoughts

Having tackled compliance by shifting from physical computer and software validation to a risk-based validation of the overall virtualization environment and its outputs (e.g., electronic record integrity), you will face an interesting question: Since that approach works for the complex virtualized data center, why are we still taking the costly 1997 “validate everything” strategy with the rest of our 21^st century technology?

Are you ready?