Oracle was the first commercial database that supported clustering at the database level, in the early 1980s. Oracle 6.2 gave birth to Oracle Parallel Server (OPS), which used Oracle’s own DLM and worked very well with Digital’s VAX clusters. Oracle was the first database to run the parallel server.
In the early 1990s, when open systems dominated the computer industry, many UNIX vendors started clustering technology, mostly based on Oracle’s DLM implementation. Oracle 7 Parallel Server (OPS) used vendor-supplied clusterware.
In the Oracle Database 8 release, Oracle introduced a generic lock manager. Oracle’s lock manager is integrated with Oracle code with an additional layer called OSD (Operating System Dependent). Oracle’s lock manager soon integrated with the kernel and became known as the IDLM (Integrated Distributed Lock Manager) in later versions of Oracle.
Oracle Real Application Clusters version 9i used the same IDLM and relied on external clusterware. Oracle provided its own clusterware for all operating systems starting with 10g and enhanced it quite a bit in Oracle Database 11g with the introduction of server pools and additional APIs to manage third-party applications. Oracle Clusterware is the de facto clusterware and a requirement for running Oracle RAC.
Oracle Parallel Storage Evaluation
Along with the cluster software infrastructure, another key component, which grew along with Oracle RAC, is the development of parallel storage management systems. Earlier versions of Oracle clusters (known as Oracle Parallel Servers until Oracle 8i) depended on third-party software for parallel storage management. Cluster file systems such as Veritas Cluster File Systems (VxFS) and RAW disk partitions were used as foundations for parallel storage in Oracle RAC.
The unavailability of general-purpose clusterware and a cluster-wide file system was hampering the adoption of Oracle RAC. Oracle provided generic clusterware with Oracle 9i RAC called Oracle Cluster Manager (OraCM) for Linux and Windows. Along with the clusterware, Oracle also developed a cluster file system called Oracle Cluster File System (OCFS), which was used to build the shared storage for Linux and Windows environments. Since Oracle provided OraCM with OCFS, the adaption of Oracle RAC significantly advanced. However, OCFS had inherent limitations and was not very scalable.
Oracle introduced a similar product in Oracle Database 9i Release 2 called Oracle Disk Manager (ODM) for Solaris operating systems. ODM was capable of handling file management operations (which were handled outside Oracle prior to this, including creating and destroying files), and file identifiers in ODM replaced file descriptors.
With the success and wisdom gained from Oracle Disk Manager, Oracle was ready for a fully functioning cluster-aware database file system. Enter Automatic Storage Management (ASM). Starting with Oracle Database 10g, Oracle introduced ASM for mainline storage and promoted for the use of ASM extensively. Oracle ASM quickly gained traction in the market and performed flawlessly in Oracle benchmarks and large customer environments.
While Oracle ASM was being used for mainline storage of data, additional shared storage (outside ASM) was still required for some of the Oracle RAC components used to house cluster-configuration details, as well as special-purpose storage used during cluster configurations commonly known as “voting disks.” The initial versions of ASM lacked the capacity to store cluster components such as OCR and voting disks inside ASM, because clusterware should be started to access the data inside ASM-managed storage. However, current versions of ASM implement an intelligent trick to access the ASM data (the details of which are discussed in the later chapters).
Custom applications built using the Oracle database and even some commercial applications (such as Oracle E-Business Suite) used a wide variety of external files during data extraction and loading. A classic example is processing Call Data Records (CDR) in a telecom billing application or output files in Oracle Applications’ concurrent manager processing. Database applications similar to these required a scalable cluster-wide file system so that all the data could be accessed concurrently among the nodes.
To address the requirement for a cluster-wide file system, Oracle ASM was enhanced further to serve as a general-purpose file system, called ASM Cluster File System (ACFS). ACFS offered a seamless way to provide access to non-database files across the nodes and was used widely to store the database binaries, log files, and non-data files used in the Oracle database. The application programming interfaces (APIs) built on Oracle ASM allowed for the use of ACFS volumes such as traditional Network File System (NFS) mount points across the servers.
Oracle ASM with ACFS addresses the total storage solution for parallel storage requirements for Oracle Clusterware and eliminates the need for expensive third-party file systems or maintenance-heavy raw devices. Oracle ASM also provides volume manager and file system functionalities for an Oracle database and is an integrated part of the Oracle Clusterware architecture.
Oracle Parallel Server Architecture
An Oracle parallel or cluster database consists of two or more physical servers (nodes) that host their own Oracle instances and share a disk array. Each node’s instance of Oracle has its own System Global Area (SGA) and its own redo log files, but the data files and control files are common to all instances. Data files and control files are concurrently read and written by all instances; however, redo logs can be read by any instance but written to only by the owning instance. Some parameters, such as db_block_buffers and log_buffer, can be configured differently in each instance, but other parameters must be consistent across all instances. Each cluster node has its own set of background processes, just as a single instance would. Additionally, OPS-specific processes are also started on each instance to handle cross-instance communication, lock management, and block transfers.
Figure 1 shows the architecture of OPS. It is important to understand the inner workings of OPS in order to understand Oracle RAC because it provides a solid foundation for visualizing the under-the-hood operations of Oracle RAC. Although this is not mandatory knowledge for managing Oracle RAC, a deeper understanding of the OPS components makes you an expert in Oracle RAC because Oracle RAC evolved from an OPS foundation.
FIGURE 1. OPS architecture
