Skip to main content
NetApp Solutions

Step-by-Step Oracle Deployment Procedures on AWS EC2 and FSx

Contributors kevin-hoke

This section describes the deployment procedures of deploying Oracle RDS custom database with FSx storage.

Deploy an EC2 Linux instance for Oracle via EC2 console

If you are new to AWS, you first need to set up an AWS environment. The documentation tab at the AWS website landing page provides EC2 instruction links on how to deploy a Linux EC2 instance that can be used to host your Oracle database via the AWS EC2 console. The following section is a summary of these steps. For details, see the linked AWS EC2-specific documentation.

Setting up your AWS EC2 environment

You must create an AWS account to provision the necessary resources to run your Oracle environment on the EC2 and FSx service. The following AWS documentation provides the necessary details:

Key topics:

  • Sign up for AWS.

  • Create a key pair.

  • Create a security group.

Enabling multiple availability zones in AWS account attributes

For an Oracle high availability configuration as demonstrated in the architecture diagram, you must enable at least four availability zones in a region. The multiple availability zones can also be situated in different regions to meet the required distances for disaster recovery.

aws ora fsx ec2 inst 01

Creating and connecting to an EC2 instance for hosting Oracle database

See the tutorial Get started with Amazon EC2 Linux instances for step-by-step deployment procedures and best practices.

Key topics:

  • Overview.

  • Prerequisites.

  • Step 1: Launch an instance.

  • Step 2: Connect to your instance.

  • Step 3: Clean up your instance.

The following screen shots demonstrate the deployment of an m5-type Linux instance with the EC2 console for running Oracle.

  1. From the EC2 dashboard, click the yellow Launch Instance button to start the EC2 instance deployment workflow.

    aws ora fsx ec2 inst 02
  2. In Step 1, select "Red Hat Enterprise Linux 8 (HVM), SSD Volume Type - ami-0b0af3577fe5e3532 (64-bit x86) / ami-01fc429821bf1f4b4 (64-bit Arm)."

    aws ora fsx ec2 inst 03
  3. In Step 2, select an m5 instance type with the appropriate CPU and memory allocation based on your Oracle database workload. Click "Next: Configure Instance Details."

    aws ora fsx ec2 inst 04
  4. In Step 3, choose the VPC and subnet where the instance should be placed and enable public IP assignment. Click "Next: Add Storage."

    aws ora fsx ec2 inst 05
  5. In Step 4, allocate enough space for the root disk. You may need the space to add a swap. By default, EC2 instance assign zero swap space, which is not optimal for running Oracle.

    aws ora fsx ec2 inst 06
  6. In Step 5, add a tag for instance identification if needed.

    aws ora fsx ec2 inst 07
  7. In Step 6, select an existing security group or create a new one with the desired inbound and outbound policy for the instance.

    aws ora fsx ec2 inst 08
  8. In Step 7, review the instance configuration summary, and click Launch to start instance deployment. You are prompted to create a key pair or select a key pair for access to the instance.

    aws ora fsx ec2 inst 09
    aws ora fsx ec2 inst 09 1
  9. Log into EC2 instance using an SSH key pair. Make changes to your key name and instance IP address as appropriate.

    ssh -i ora-db1v2.pem ec2-user@54.80.114.77

You need to create two EC2 instances as primary and standby Oracle servers in their designated availability zone as demonstrated in the architecture diagram.

Provision FSx for ONTAP file systems for Oracle database storage

EC2 instance deployment allocates an EBS root volume for the OS. FSx for ONTAP file systems provides Oracle database storage volumes, including the Oracle binary, data, and log volumes. The FSx storage NFS volumes can be either provisioned from the AWS FSx console or from Oracle installation, and configuration automation that allocates the volumes as the user configures in a automation parameter file.

Creating FSx for ONTAP file systems

Referred to this documentation Managing FSx for ONTAP file systems for creating FSx for ONTAP file systems.

Key considerations:

  • SSD storage capacity. Minimum 1024 GiB, maximum 192 TiB.

  • Provisioned SSD IOPS. Based on workload requirements, a maximum of 80,000 SSD IOPS per file system.

  • Throughput capacity.

  • Set administrator fsxadmin/vsadmin password. Required for FSx configuration automation.

  • Backup and maintenance. Disable automatic daily backups; database storage backup is executed through SnapCenter scheduling.

  • Retrieve the SVM management IP address as well as protocol-specific access addresses from SVM details page. Required for FSx configuration automation.

    aws rds custom deploy fsx 01

See the following step-by-step procedures for setting up either a primary or standby HA FSx cluster.

  1. From the FSx console, click Create File System to start the FSx provision workflow.

    aws ora fsx ec2 stor 01
  2. Select Amazon FSx for NetApp ONTAP. Then click Next.

    aws ora fsx ec2 stor 02
  3. Select Standard Create and, in File System Details, name your file system, Multi-AZ HA. Based on your database workload, choose either Automatic or User-Provisioned IOPS up to 80,000 SSD IOPS. FSx storage comes with up to 2TiB NVMe caching at the backend that can deliver even higher measured IOPS.

    aws ora fsx ec2 stor 03
  4. In the Network & Security section, select the VPC, security group, and subnets. These should be created before FSx deployment. Based on the role of the FSx cluster (primary or standby), place the FSx storage nodes in the appropriate zones.

    aws ora fsx ec2 stor 04
  5. In the Security & Encryption section, accept the default, and enter the fsxadmin password.

    aws ora fsx ec2 stor 05
  6. Enter the SVM name and the vsadmin password.

    aws ora fsx ec2 stor 06
  7. Leave the volume configuration blank; you do not need to create a volume at this point.

    aws ora fsx ec2 stor 07
  8. Review the Summary page, and click Create File System to complete FSx file system provision.

    aws ora fsx ec2 stor 08

Provisioning of database volumes for Oracle database

Key considerations:

  • Sizing the database volumes appropriately.

  • Disabling capacity pool tiering policy for performance configuration.

  • Enabling Oracle dNFS for NFS storage volumes.

  • Setting up multipath for iSCSI storage volumes.

Create database volume from FSx console

From the AWS FSx console, you can create three volumes for Oracle database file storage: one for the Oracle binary, one for the Oracle data, and one for the Oracle log. Make sure that volume naming matches the Oracle host name (defined in the hosts file in the automation toolkit) for proper identification. In this example, we use db1 as the EC2 Oracle host name instead of a typical IP-address-based host name for an EC2 instance.

aws ora fsx ec2 stor 09
aws ora fsx ec2 stor 10
aws ora fsx ec2 stor 11
Note Creating iSCSI LUNs is not currently supported by the FSx console. For iSCSI LUNs deployment for Oracle, the volumes and LUNs can be created by using automation for ONTAP with the NetApp Automation Toolkit.

Install and configure Oracle on an EC2 instance with FSx database volumes

The NetApp automation team provide an automation kit to run Oracle installation and configuration on EC2 instances according to best practices. The current version of the automation kit supports Oracle 19c on NFS with the default RU patch 19.8. The automation kit can be easily adapted for other RU patches if needed.

Prepare a Ansible controller to run automation

Follow the instruction in the section "Creating and connecting to an EC2 instance for hosting Oracle database" to provision a small EC2 Linux instance to run the Ansible controller. Rather than using RedHat, Amazon Linux t2.large with 2vCPU and 8G RAM should be sufficient.

Retrieve NetApp Oracle deployment automation toolkit

Log into the EC2 Ansible controller instance provisioned from step 1 as ec2-user and from the ec2-user home directory, execute the git clone command to clone a copy of the automation code.

git clone https://github.com/NetApp-Automation/na_oracle19c_deploy.git
git clone https://github.com/NetApp-Automation/na_rds_fsx_oranfs_config.git

Execute automated Oracle 19c deployment using automation toolkit

See these detailed instruction CLI deployment Oracle 19c Database to deploy Oracle 19c with CLI automation. There is a small change in command syntax for playbook execution because you are using an SSH key pair instead of a password for host access authentication. The following list is a high level summary:

  1. By default, an EC2 instance uses an SSH key pair for access authentication. From Ansible controller automation root directories /home/ec2-user/na_oracle19c_deploy, and /home/ec2-user/na_rds_fsx_oranfs_config, make a copy of the SSH key accesststkey.pem for the Oracle host deployed in the step "Creating and connecting to an EC2 instance for hosting Oracle database."

  2. Log into the EC2 instance DB host as ec2-user, and install the python3 library.

    sudo yum install python3
  3. Create a 16G swap space from the root disk drive. By default, an EC2 instance creates zero swap space. Follow this AWS documentation: How do I allocate memory to work as swap space in an Amazon EC2 instance by using a swap file?.

  4. Return to the Ansible controller (cd /home/ec2-user/na_rds_fsx_oranfs_config), and execute the preclone playbook with the appropriate requirements and linux_config tags.

    ansible-playbook -i hosts rds_preclone_config.yml -u ec2-user --private-key accesststkey.pem -e @vars/fsx_vars.yml -t requirements_config
    ansible-playbook -i hosts rds_preclone_config.yml -u ec2-user --private-key accesststkey.pem -e @vars/fsx_vars.yml -t linux_config
  5. Switch to the /home/ec2-user/na_oracle19c_deploy-master directory, read the README file, and populate the global vars.yml file with the relevant global parameters.

  6. Populate the host_name.yml file with the relevant parameters in the host_vars directory.

  7. Execute the playbook for Linux, and press Enter when prompted for the vsadmin password.

    ansible-playbook -i hosts all_playbook.yml -u ec2-user --private-key accesststkey.pem -t linux_config -e @vars/vars.yml
  8. Execute the playbook for Oracle, and press enter when prompted for the vsadmin password.

    ansible-playbook -i hosts all_playbook.yml -u ec2-user --private-key accesststkey.pem -t oracle_config -e @vars/vars.yml

Change the permission bit on the SSH key file to 400 if needed. Change the Oracle host (ansible_host in the host_vars file) IP address to your EC2 instance public address.

Setting up SnapMirror between primary and standby FSx HA cluster

For high availability and disaster recovery, you can set up SnapMirror replication between the primary and standby FSx storage cluster. Unlike other cloud storage services, FSx enables a user to control and manage storage replication at a desired frequency and replication throughput. It also enables users to test HA/DR without any effect on availability.

The following steps show how to set up replication between a primary and standby FSx storage cluster.

  1. Setup primary and standby cluster peering. Log into the primary cluster as the fsxadmin user and execute the following command. This reciprocal create process executes the create command on both the primary cluster and the standby cluster. Replace standby_cluster_name with the appropriate name for your environment.

    cluster peer create -peer-addrs standby_cluster_name,inter_cluster_ip_address -username fsxadmin -initial-allowed-vserver-peers *
  2. Set up vServer peering between the primary and standby cluster. Log into the primary cluster as the vsadmin user and execute the following command. Replace primary_vserver_name, standby_vserver_name, standby_cluster_name with the appropriate names for your environment.

    vserver peer create -vserver primary_vserver_name -peer-vserver standby_vserver_name -peer-cluster standby_cluster_name -applications snapmirror
  3. Verify that the cluster and vserver peerings are set up correctly.

    aws ora fsx ec2 stor 14
  4. Create target NFS volumes at the standby FSx cluster for each source volume at the primary FSx cluster. Replace the volume name as appropriate for your environment.

    vol create -volume dr_db1_bin -aggregate aggr1 -size 50G -state online -policy default -type DP
    vol create -volume dr_db1_data -aggregate aggr1 -size 500G -state online -policy default -type DP
    vol create -volume dr_db1_log -aggregate aggr1 -size 250G -state online -policy default -type DP
  5. You can also create iSCSI volumes and LUNs for the Oracle binary, Oracle data, and the Oracle log if the iSCSI protocol is employed for data access. Leave approximately 10% free space in the volumes for snapshots.

    vol create -volume dr_db1_bin -aggregate aggr1 -size 50G -state online -policy default -unix-permissions ---rwxr-xr-x -type RW
    lun create -path /vol/dr_db1_bin/dr_db1_bin_01 -size 45G -ostype linux
    vol create -volume dr_db1_data -aggregate aggr1 -size 500G -state online -policy default -unix-permissions ---rwxr-xr-x -type RW
    lun create -path /vol/dr_db1_data/dr_db1_data_01 -size 100G -ostype linux
    lun create -path /vol/dr_db1_data/dr_db1_data_02 -size 100G -ostype linux
    lun create -path /vol/dr_db1_data/dr_db1_data_03 -size 100G -ostype linux
    lun create -path /vol/dr_db1_data/dr_db1_data_04 -size 100G -ostype linux

    vol create -volume dr_db1_log -aggregate aggr1 -size 250G -state online -policy default -unix-permissions ---rwxr-xr-x -type RW

    lun create -path /vol/dr_db1_log/dr_db1_log_01 -size 45G -ostype linux
    lun create -path /vol/dr_db1_log/dr_db1_log_02 -size 45G -ostype linux
    lun create -path /vol/dr_db1_log/dr_db1_log_03 -size 45G -ostype linux
    lun create -path /vol/dr_db1_log/dr_db1_log_04 -size 45G -ostype linux
  6. For iSCSI LUNs, create mapping for the Oracle host initiator for each LUN, using the binary LUN as an example. Replace the igroup with an appropriate name for your environment, and increment the lun-id for each additional LUN.

    lun mapping create -path /vol/dr_db1_bin/dr_db1_bin_01 -igroup ip-10-0-1-136 -lun-id 0
    lun mapping create -path /vol/dr_db1_data/dr_db1_data_01 -igroup ip-10-0-1-136 -lun-id 1
  7. Create a SnapMirror relationship between the primary and standby database volumes. Replace the appropriate SVM name for your environment.s

    snapmirror create -source-path svm_FSxOraSource:db1_bin -destination-path svm_FSxOraTarget:dr_db1_bin -vserver svm_FSxOraTarget -throttle unlimited -identity-preserve false -policy MirrorAllSnapshots -type DP
    snapmirror create -source-path svm_FSxOraSource:db1_data -destination-path svm_FSxOraTarget:dr_db1_data -vserver svm_FSxOraTarget -throttle unlimited -identity-preserve false -policy MirrorAllSnapshots -type DP
    snapmirror create -source-path svm_FSxOraSource:db1_log -destination-path svm_FSxOraTarget:dr_db1_log -vserver svm_FSxOraTarget -throttle unlimited -identity-preserve false -policy MirrorAllSnapshots -type DP

This SnapMirror setup can be automated with a NetApp Automation Toolkit for NFS database volumes. The toolkit is available for download from the NetApp public GitHub site.

git clone https://github.com/NetApp-Automation/na_ora_hadr_failover_resync.git

Read the README instructions carefully before attempting setup and failover testing.

Note Replicating the Oracle binary from the primary to a standby cluster might have Oracle license implications. Contact your Oracle license representative for clarification. The alternative is to have Oracle installed and configured at the time of recovery and failover.

SnapCenter Deployment

SnapCenter installation

Follow Installing the SnapCenter Server to install SnapCenter server. This documentation covers how to install a standalone SnapCenter server. A SaaS version of SnapCenter is in beta review and could be available shortly. Check with your NetApp representative for availability if needed.

Configure SnapCenter plugin for EC2 Oracle host

  1. After automated SnapCenter installation, log into SnapCenter as an administrative user for the Window host on which the SnapCenter server is installed.

    aws rds custom deploy snp 01
  2. From the left-side menu, click Settings, and then Credential and New to add ec2-user credentials for SnapCenter plugin installation.

    aws rds custom deploy snp 02
  3. Reset the ec2-user password and enable password SSH authentication by editing the /etc/ssh/sshd_config file on the EC2 instance host.

  4. Verify that the "Use sudo privileges" checkbox is selected. You just reset the ec2-user password in the previous step.

    aws rds custom deploy snp 03
  5. Add the SnapCenter server name and the IP address to the EC2 instance host file for name resolution.

    [ec2-user@ip-10-0-0-151 ~]$ sudo vi /etc/hosts
    [ec2-user@ip-10-0-0-151 ~]$ cat /etc/hosts
    127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
    ::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
    10.0.1.233  rdscustomvalsc.rdscustomval.com rdscustomvalsc
  6. On the SnapCenter server Windows host, add the EC2 instance host IP address to the Windows host file C:\Windows\System32\drivers\etc\hosts.

    10.0.0.151		ip-10-0-0-151.ec2.internal
  7. In the left-side menu, select Hosts > Managed Hosts, and then click Add to add the EC2 instance host to SnapCenter.

    aws rds custom deploy snp 04

    Check Oracle Database, and, before you submit, click More Options.

    aws rds custom deploy snp 05

    Check Skip Preinstall Checks. Confirm Skipping Preinstall Checks, and then click Submit After Save.

    aws rds custom deploy snp 06

    You are prompted with Confirm Fingerprint, and then click Confirm and Submit.

    aws rds custom deploy snp 07

    After successful plugin configuration, the managed host's overall status show as Running.

    aws rds custom deploy snp 08

Configure backup policy for Oracle database

Refer to this section Setup database backup policy in SnapCenter for details on configuring the Oracle database backup policy.

Generally you need create a policy for the full snapshot Oracle database backup and a policy for the Oracle archive-log-only snapshot backup.

Note You can enable Oracle archive log pruning in the backup policy to control log-archive space. Check "Update SnapMirror after creating a local Snapshot copy" in "Select secondary replication option" as you need to replicate to a standby location for HA or DR.

Configure Oracle database backup and scheduling

Database backup in SnapCenter is user configurable and can be set up either individually or as a group in a resource group. The backup interval depends on the RTO and RPO objectives. NetApp recommends that you run a full database backup every few hours and archive the log backup at a higher frequency such as 10-15 mins for quick recovery.

Refer to the Oracle section of Implement backup policy to protect database for a detailed step-by-step processes for implementing the backup policy created in the section Configure backup policy for Oracle database and for backup job scheduling.

The following image provides an example of the resources groups that are set up to back up an Oracle database.

aws rds custom deploy snp 09