Erase and Rewind – Redeploying Oracle Database Appliance (2.3.0 image)

After using an Oracle Database Appliance for a bit of testing, it was time to get it ready for production. I decided to start with a fresh deploy and re-initializing shared storage.

Note that this is the 2.3.0 base image, and that the procedure may differ somewhat for other versions.

The included cleanupDeploy.pl script will pretty much just shut everything down, erase everything on the shared storage and reboot both nodes. It will leave the network configuration for the iLOM and most files stored on the system drives, including deployment configuration files you may have stored in /root/

It will remove network configuration and set the root password back to the default of welcome1.

[root@oma1 ~]# cd /opt/oracle/oak/onecmd
[root@oma1 onecmd]# ./cleanupDeploy.pl

Please enter the root password for performing cleanup:
Re-enter root password:

About to clear up OAK deployment,public network connectivity will be lost,root password will be set to default and both nodes will be rebooted

Do you want to continue(yes/no):

The script starts by asking twice for the root password before requesting a final confirmation. Make no mistake, after this point all services will be shut down and all data will be erased!

After a few minutes, the script completes with

Exiting…
[root@oma1 onecmd]#
Connection to oma1 closed by remote host.
Connection to oma1 closed.

and we are ready to start a new installation.

The first thing we need to do is connect to the iLOM either by serial or ethernet, or connect a keyboard and monitor so we can set up the initial network configuration. The cleanup script will not mess with the current configuration of the iLOM ethernet interface, so web and ssh console should be available.

-> start /SP/console
Are you sure you want to start /SP/console (y/n)? y

Serial console started. To stop, type ESC (

Oracle Linux Server release 5.8 Kernel 2.6.32-300.32.3.el5uek on an x86_64

oak1 login: root
Password:
Last login: Mon Oct 22 15:53:20 from 192.168.60.46

[root@oak1 ~]# cd /opt/oracle/oak/bin/
[root@oak1 bin]# ./oakcli configure firstnet

To work around a bug with the cleanupDeploy.pl we need to add a localhost entry to /etc/hosts on both nodes.

[root@oak1 ~]# echo 127.0.0.1 localhost.localdomain localhost >> /etc/hosts

I’m connected to the iLOM serial console through ssh, so I have to reconnect to the newly configured interface to enable X support for the deployment GUI.  I have a configuration file stored in the /root folder, so I call the deploy using that file as parameter file.

Connection to oma1-ilom closed.
roy@kayna:~$ ssh oma1 -l root -X
root@oma1’s password:
Last login: Tue Oct 23 13:01:39 2012 from 192.168.60.46
[root@oak1 ~]# cd /opt/oracle/oak/bin/
[root@oak1 bin]# ./oakcli deploy -conf /root/oak_deploy.params
Log messages in /tmp/oak_1350990145126.log
Loaded configuration file /root/oak_deploy.params successfully
Running Oracle Appliance Manager

When deployment starts, depending on your network configuration, the deploy will sometimes make a mess of the default route.  If you see this line when running route

default * 0.0.0.0 U 0 0 0 bond0

you will need to immediately delete the entry and set the correct default gateway (on both nodes), less the deploy may very well fail the ValidateEnv step, because it can’t access the DNS servers to check for the SCAN listener DNS entry.

[root@oak2 ~]# route delete default
[root@oak2 ~]# route add default gw 192.168.0.1

Hopefully the deploy will now complete successfully. If not, you can always jump back to the top, clean up the deploy and start fresh for another attempt.