The epiphany of data warehousing technologies in the pharmaceutical industry

The highly competitive pharmaceutical industry has seen many external changes to its landscape as companies consume each other increasing their pipelines while removing redundant functions and processes. Internally, companies have sought to streamline the discovery and development phases in an attempt to improve candidate selection and reduce the time to regulatory filing. In conjunction with efforts to screen and develop more compounds faster and more efficiently, database management systems (DBMS) have been developed for numerous groups supporting various R&D efforts. An outgrowth of DBMS evolution has been the birth of data warehousing. Often confused with DBMS, data warehousing provides a conduit for data residing across platforms, networks, and in different data structures. Through the use of metadata, the warehouse establishes connectivity of varied data stores (operational detail data, ODD) and permits identification of data ownership, location and transaction history. This evolution has closely mirrored and in some ways been driven by the electronic submission (formerly CANDA). The integration of the electronic submissions and document management with R&D data warehousing initiatives should provide a platform by which companies can address compliance with 21 CFR Part 11. Now more than ever "corporate memory" is being extended to the data itself. The when, why and how of successes and failures are constantly being probed by R&D management teams. The volume of information being generated by today's pharmaceutical companies requires mining of historical data on a routine basis. Data warehousing represents a core technology to assist in this endeavor. New initiatives in this field address the necessity of data portals through which warehouse data can be web-enabled and exploited by diverse data customers both internal and external to the company. The epiphany of data warehousing technologies within the pharmaceutical industry has begun and promises to change the way in which companies process and provide data to regulatory agencies. The improvements in drug discovery and reduction in development timelines remain to be seen but would seem to be rational if such technology is fully exploited.