Virtualization Sysadmin: Netbackup

Showing posts with label Netbackup. Show all posts

Sunday, June 30, 2019

Veritas NetBackup OpsCenter upgrade

Pre-Checks:

1. OpsCenter should run on same version as Netbackup is running or should run on higher version. OpsCenter/OpsCenter Analytics must always be running the same or higher version.

2. Make sure that OpsCenter is compatible with Operating System and Netbackup.

3. OpsCenter should always run on a dedicated server and not on a NBU master

4. Ensure that all required ports are open on OpsCenter Server.

OpsCenter Port requirement:

Technote: https://www.veritas.com/support/en_US/article.100002391

Steps to Upgrade OpsCenter Analytics

1. Backup OpsCenter Database

OpsCenter is shipped with a database backup script that performs backup of the database , the corresponding database password file, and the encryption key file without interrupting its operations, which is referred to as hot backup. On UNIX as well as Windows platforms, the script overwrites the existing database files before backing up or restoring them.

Note:

Regular file system backups are not sufficient for backing up the OpsCenter database. You must schedule periodic hot backups for the OpsCenter database to avoid losing any important data.

To back up the OpsCenter database

Log on to the OpsCenter database server host in one of the following ways:

Windows	As an administrator or user in the Administrator group
UNIX	root

Open the Windows command prompt or the UNIX console.
Run the backup script that is appropriate for your platform. Specify one of the following backup directories depending on your platform:


LINUX	/opt/SYMCOpsCenterServer/bin/dbbackup.sh /u01/backup/OpsCenter/

The backup script backs up the database files, database password files, and encryption key file in the backup directory that you specified.
Ensure the compatibility of the OpsCenter with the OS Version ad Bit.
Download OpsCenter installation file to the OS.
Run the installation and follow the process.

Note: OpsCenter should always run on a dedicated server and not on a NBU master

Documentation:

Backing up the OpsCenter database

Technote: https://www.veritas.com/support/en_US/doc/27537447-127427700-0/v28842080-127427700

Download OpsCenter:

Link: https://www.veritas.com/content/support/en_US/article.100019594.html

How to Backup OpsCenter Database:

Link: https://www.veritas.com/support/en_US/doc/27537447-127427700-0/v28842080-127427700

Upgrading OpsCenter on UNIX

Use the following procedure to upgrade to OpsCenter software on UNIX hosts.
Note:

Veritas recommends that you do not cancel or interrupt the installation process once it is started.

To upgrade OpsCenter Server on UNIX

If you plan to upgrade your backup product and OpsCenter, ensure that you upgrade OpsCenter first. When upgrading OpsCenter, always upgrade the OpsCenter Server first followed by the Agent and then the View Builder.
Open a UNIX console and log on as root on the target host.
Mount the OpsCenter product DVD on the OpsCenter Server that you want to upgrade.
Untar OpsCenter_7.7_Unix.tar.gz
Cd /u01/share/soft/OpsCenter_7.7_Unix
Type the following command: ./install. Press Enter.
[root@em01 OpsCenter_7.7_Unix]# ./install

The Welcome message is displayed. Press Enter to continue.

The installer then checks if OpsCenter Server is installed on the system or not. It prompts you in case OpsCenter Server is already installed. The installer also examines the system for existing packages.

The following prompt is displayed:

Where should the existing OpsCenter database and configuration files be backed up?

An 'OpsCenterServer_backup' directory will be created within the directory that you specify to store these

files. (/var/symantec/)

Type the directory name in which the existing OpsCenter database and configuration files can be saved and then press Enter.

To accept the default directory path (/var/symantec), press Enter.

The installer displays a list of components that will be installed or upgraded like PBX, OpsCenter Server, OpsCenter user interface and so on. Review this list and press Enter to continue.

The installer prompts you with the following question:

installOpsCenterServer is now ready to upgrade OPSCENTERSERVER.

Are you sure you want to upgrade OPSCENTERSERVER? [y,n,q] (y)

Press Enter to continue OpsCenter upgrade.

The installer prompts you with the following question:

Participate in the NetBackup Product Improvement program? [y,n,q] (y)

Press Enter if you want to participate in the NetBackup Product Improvement program or press n if you do not want to participate.

The installer prompts you with the following question:

Do you want to import users from <remote AT host name> to OpsCenter AT?

Press Enterto import users from the earlier OpsCenter versions.

For more details, refer to the About Importing Authentication Settings section.

If the import fails, you need to reset password of all OpsCenter(vx) users.

By default, in OpsCenter, the database is upgraded in the beginning. However, you can choose to first upgrade the OpsCenter software.

The database upgrade process starts. In this process, the OpsCenter data is migrated to the new OpsCenter database version. The database process may take time depending on your database size.

The following message is displayed:

The database upgrade is in progress. This may take some time based on the database size. You can see the progress and current status in /var/tmp/(directory)/db/log/dbManager_<timestamp>_.log file.

After complete installation, the logs are copied to the following location:/var/VRTS/install/logs/LogDirectory/db/log

LogDirectory is generated during the upgrade progress.

The installer displays the OpsCenter packages that are installed. Depending on the installed packages, the following message may be displayed:

The following packages were found on the system. However the package

versions are older than the ones required by OpsCenter.

VRTSOpsCenterServer existing version was found on the system,

but OpsCenter requires VRTSOpsCenterServer new version

VRTSOpsCenterGUI existing version was found on the system,

but OpsCenter requires VRTSOpsCenterGUI new version

VRTSOpsCenterLegacyServer existing version was found on the system,

but OpsCenter requires VRTSOpsCenterLegacyServer new version

The installer then uninstalls the older packages and installs new version of OpsCenter Server.

OpsCenter Server is installed. Configuration changes are made to the system.
All the OpsCenter processes are started. The following information is also displayed:

o Web URL to access OpsCenter console

o Location of install log and summary files.

Note:

After successful upgrade, OpsCenter database password is set to an auto-generated string, if the earlier password is still the default one, that is 'SQL'. Veritas recommends that you change the database password after the installation.

If you had changed the default password in the earlier version, you do not need to change it again. The installer retains the old password that you can use in the current version.

Decommissioning a media server

Decommissioning Netbackup Media Server

Following information about is decommissioning a NetBackup media server.

Your path through the nbdecommission wizard depends on how you respond to the wizard prompts. Depending on your environment and how you respond to prompts, the wizard may advise you to perform an action and then exit. To continue in the wizard, you must run the wizard again after you perform the advised action. You may have to exit and rerun the wizard several times.

If active jobs exist on the media server, you must cancel them before the command can begin to decommission the media server. Alternatively, you can wait until they finish.

The OldServer does not have to be up and responsive.
Veritas recommends that you preview the media server references before you decommission a media server.

The nbdecommission command logs to the standard NetBackup administrator commands log directory.
Two procedures exist, as follows:
1. Replace an old media server with a new media server
2. Decommission a media server.

To replace an old media server with a new media server
1. Run the nbdecommission command on the master server or on a media server that is not the object of this operation. The following is the command syntax:

On Windows:
install_path\NetBackup\bin\admincmd\nbdecommission -oldserver OldServer [-newserver NewServer] [-file decom_ops.txt]

On UNIX:
/usr/openv/netbackup/bin/admincmd/nbdecommission -oldserver OldServer [-newserver NewServer] [-file decom_ops.txt]
Replace OldServer with the name of the host to be decommissioned.

-newserver is optional. If you specify a new server, the new server becomes the default media server for the replacement operations. If you do not specify a new server, the wizard prompts you for the new server for each storage type that contains valid backup images. This method is useful if you want to move backup images to different media servers. For example, you can move backup images from tape storage to one media server and backup images from disk storage to another media server.
-file is optional. It writes the command operations to the specified file. Replace decom_ops.txt with a name that denotes its purpose or contents. Veritas recommends that you use the -file option to maintain a record of the command operations.

2. Follow the prompts and perform the requested actions.
For example, the command may make changes on the master server and on multiple media servers. You may be required to restart the NetBackup services on those servers so that the changes take effect.

To decommission a media server
a) Run the following command on the master server or on a media server that is not the object of this operation. The OldServer does not have to be up and responsive.

On Windows:
install_path\NetBackup\bin\admincmd\nbdecommission -oldserver OldServer

On UNIX:
/usr/openv/netbackup/bin/admincmd/nbdecommission -oldserver OldServer
Replace OldServer with the name of the host to be decommissioned.

b) Follow the prompts and perform the requested actions.

bphdb exit status = 5449: The script is not approved for execution

Registering authorized locations used by a NetBackup database script-based policy

To add an authorized location

Open a command prompt on the client.
Use nbsetconfig to enter values for an authorized location. The client privileged user must run these commands.

On UNIX:
root@erpddb01:/usr/openv/netbackup/bin/nbsetconfig
nbsetconfig>DB_SCRIPT_PATH = /export/home/oraprod/scripts
nbsetconfig>DB_SCRIPT_PATH = /export/home/oraprod/scripts/rmanfull.daily
nbsetconfig>
<ctrl-D>

Reference: NetBackup Support Knowledge base

Tuesday, June 25, 2019

Issue with Robots Missing Path

è To Check Robots with missing path

o Java GUI

n Under Media and Device Mgmt àUnder Devices àSelect Robots

o Command

n /usr/openv/volmgr/bin/tpconfig –d

80 Drive161 hcart3 TLD(6) DRIVE=7 /dev/nst27 UP

81 Drive162 hcart3 TLD(6) DRIVE=8 /dev/nst28 UP

82 Drive108 hcart3 TLD(6) DRIVE=9 /dev/nst29 UP

Currently defined robotics are:

TLD(1) robotic path = /dev/sg291

TLD(3) robotic path = /dev/sg76

TLD(4) robotic path = /dev/sg52

TLD(5) robotic path = /dev/sg98

TLD(6) robotic path = /dev/sg87

TLD(7) robotic path = /dev/sg74

EMM Server = usvhvoilms001

/usr/openv/volmgr/bin/tpconfig -l

èDelete the robot with missing path either java GUI or Command

o Java GUI (Recommended)

n Under Media and Device Mgmt àUnder Devices àSelect Robots àSelect the robot with missing path àRight Click and Select Delete

o Command

n /usr/openv/volmgr/bin/tpconfig –delete -robot <robot number>

èRestart Media Manager processes.

èIf received any error while deleting robot, please follow this steps

o Please check any resource holding this robot using this command

n /usr/openv/netbackup/bin/admincmd/nbrbutil –dump

n If you see any resource holding this robot please use this command to release it nbrbutil -resetMediaServer <media server name>

nbrbutil -releaseMDS

nbrbutil -releaseMedia

èCheck the robot at Operating System Level

o /usr/openv/volmgr/bin/scan

o /usr/openv/volmgr/bin/tpautoconf –r

o cat /proc/scsi/scsi

If found robot at OS level, please run Device Config Wizard on Master server and rescan the devices on master server and follow the on screen steps to add the robot.

èRestart Media Manager processes.

èTo Confirm Robot is added

o Java GUI

n Under Media and Device Mgmt àUnder Devices àSelect Robots

o Command

n /usr/openv/volmgr/bin/tpconfig –d or

/usr/openv/volmgr/bin/tpconfig -l

èTo Test the robot

o Use /usr/openv/volmgr/bin/robtest.

Netbackup Important Commands and Explanations

Master Server

1) Check the license details
/usr/openv/netbackup/bin/admincmd/get_license_key
2) Stop and Start the netabackup services
i) /etc/init.d/netbackup stop (start)               —> graceful stop and start
ii) /usr/openv/netbackup/bin/bp.kill_all      —> Stop backup including GUI sessions, ungraceful
iii) /usr/openv/netbackup/bin/bp.start_all   —> Start the backup                                        iv) /usr/openv/netbackup/bin/initbprd         —> starts the master server
v) /usr/openv/netbackup/bin/vmd                —> starts the media server
vi) /usr/openv/netbackup/bin/jnbSA             —> Starts the GUI sessions

3) Scan the tape devices
#sgscan (in Solaris) #/usr/openv/volmgr/bin/scan (in AIX)

4) Display all the netbackup process
#bpps –x

5) Check the backup status
In GUI —> Activity monitor
In CLI —> #bpdbjobs -report

6) Lists the server errors
#bperror
#bperrror –U –problems –hoursago 1
#bperror –U –backstat -by_statcode -hoursago 1

7) Display information about the error code
#bperror –S <statuscode>

8) Reread bp.conf file without stop/start the netbackup
#bprdreq -rereadconfig

Media Server (starts with bpxxx )
1) List the storage units
#bpstulist –U
2) List media details
# /usr/openv/netbackup/bin/goodies/available_media
This cmd is retrieving all info about media that is available/full/expired/Frozen
3) List all the netbackup jobs
#bpdbjobs –report <hoursago>
4) Freeze or Unfreeze media
In GUI,
In CLI, #bpmedia –unfreeze [-freeze] –ev <media ID>
5) List media details
#bpmedialist -ev <media ID> 6) List the media contents
#bpmedialist –U –mcontents –m <mediaID>
7) List the information about NB images on media
#bpimmedia –mediaid <ID> -L
8) List backup image information
#bpimagelist -U (general)
# bpimagelist -media –U (for media)
9) Expire a tape
# bpexpdate –d 0 –ev <mediaID> -force
10) Expire client images
#bpimage –cleanup –allclients
11) Which tapes are used for taking backup
In GUI, Backup and Restore –> Find the Filesystem –> Preview Media Button
In CLI, #bpimagelist –media –U

Volume Commands (starts with vmxxx)
1) Tape Drive (vmoprcmd)
1) List the drive status, detail drive info and pending requests
In GUI, Device mgmt
In CLI, #vmoprcmd
#vmoprcmd –d ds (status)
#vmopcrmd –d pr (pending requests)
2) Control a tape device
In GUI, Device mgmt
In CLI, #vmoprcmd [-reset] [-up] [-down] <drive number>

2) Tape Media commands (vmpool,vmquery,vmchange,vmdelete)

1) List all the pools
In CLI, #vmpool –listall -bx
2) List the scratch pool available
#vmpool -list_scratch
3) List tapes in pool
In GUI,
In CLI, #vmquery –pn <pool name> -bx 4) List all tapes in the robot
In GUI,
In CLI, #vmquery –rn 0 –bx
5) List cleaning tapes
In CLI, #vmquery –mt dlt_clean –bx
6) List tape volume details
#vmquery –m <media ID>
7) Delete a volume from the catalog
#vmdelete –m <mediaID>
8) Changes a tapes expiry date
#vmchange -exp 12/31/2012 hr:mm:ss –m <media ID>
9) Changes a tape’s media pool
#vmchange -p <pool number> -m <media ID>

3) Tape/Robot commands (starts with tpxxx)
1) List the tape drives
#tpconfig –d
2) List the cleaning times on drives
#tpclean -L
3) Clean a drive
#tpclean –C <drive number>

Client Commands
i) List the clients
#bpplclients

Policy Commands
i) List the policies
#bppllist –U
ii) List the detailed information about the policies
#bpplist –U -allpolicies

Netbackup Cheat Sheet

Master Server Daemons/Processes

Request daemon	bprd
Scheduler	bpsched (started with bprd)
Netbackup database manager	bpdbm (started with bpsched)
Job Monitor	bpjobd (started with bpdbm)

Media Server Daemons/Processes

Communications daemon	bpcd
Backup and restore manager	bpbrm (started with bpcd)
Tape Manager	bptm (started with bpbrm)
Disk Manager	bpdm (started with bpbrm)
Media Manager	ltid
Bar code reader	avrd (started with ltid)
Remote device management/ controls volume database	vmd (started with ltid)
Roboticdaemon (one on each media server) talks to tldcd	tldd (started with ltid)
Robotic control daemon talks to the robot directl via scsi	tldcd (started with ltid)

Catalogs

Master Server
Information about backed-up files	image - /opt/openv/netbackup/db
Storage Unit, Global Configuration, Catalog backup configuration.	config - /opt/openv/netbackup/db
Backup Policy information	class - /opt/openv/netbackup/db
Job status information	jobs - /opt/openv/netbackup/db
Netbackup logs with error and status information	error - /opt/openv/netbackup/db
Information on volumes, volume pools, scratch pool and volume groups	volume - /opt/openv/volmgr/database
Media Server
Tracks assigned volumes (media that has data them)	media - /opt/openv/netbackup/db
Information about devices managed by the media server	device - /opt/openv/volmgr/database

Log and Information Files

Netbackup and Patch versions	/opt/openv/netbackup/bin/version
Media Version	/opt/openv/volmgr/version
Patch Level history	/opt/openv/netbackup/patch/patch.history
Buffer size	/opt/openv/netbackup/db/config/SIZE_DATA_BUFFERS
Number of buffers	/opt/openv/netbackup/db/config/NUMBER_DATA_BUFFERS
Network Buffer Size	/opt/openv/netbackup/NET_BUFFER_SZ (default = 32)
Java GUI authorisation	/opt/openv/java/auth.conf
Catalog type (binary or ASCII)	/opt/openv/netbackup/db/config/cat_format.cfg
Netbackup and media manager parameter files	/opt/openv/netbackup/bp.conf /opt/openv/volmgr/vm.conf
Corrupt Database image files (5.0 and above)	/opt/openv/netbackup/db.corrupt

Server Commands

Check license details	/opt/openv/netbackup/bin/admincmd/get_license_key
Start Netbackup	netbackup start /opt/openv/netbackup/bin/initbprd (master) /opt/openv/volmgr/bin/vmd (media)
Stop Netbackup (does not disconnect GUI sessions)	netbackup stop /opt/openv/netbackup/bin/admincmd/bprdreq -terminate (master) /opt/openv/netbackup/bin/bpdbm -terminate (master)
Stop Netbackup and kill all GUI sessions	/opt/openv/netbackup/bin/goodies/bp.kill_all
Start the GUI	/opt/openv/netbackup/bin/jnbSA
Scan for tape devices	sgscan (solaris) ioscan (HPUX)
Display all Netbackup processes	bpps -a
lists servers errors	bperror -U -problems -hoursago <number of hours> bperror -U -backstat -by_statcode -hoursago <number of hours>
display information on a error code	bperror -statuscode <statuscode> [-recommendation]
Reread bp.conf file without stopping Netbackup	bprdreq -rereadconfig
Check database consistency	bpdbm -consistency 1 bpdbm -consistency 2 Check for the below lines: Bad image header Does not exist
Netbackup Recovery
Device catalog is intact	bprecover -l -m <media ID> -d dlt (listing) bprecover -r -m <media ID> -d dlt (recovering)
Device catalog is gone or corrupted	bprecover -l -tpath <tape_path> (listing) bprecover -r -tpath <tape_path> (recovering)
Disk backups	bprecover -l -dpath <disk_path> (listing) bprecover -r -dpath <disk_path> (recovering)

Volume Commands

Tape Drive and Inventory Commands
List drive status, detail drive info and pending requests	vmoprcmd
List the tape drive status	vmoprcmd -d ds
List the pending requests	vmoprcmd -d pr
Control a tape device	vmoprcmd [-reset][-up][-down] <drive number>
List all changes in the robot(but do not update)	vmupdate -recommend -rt tld -rn 0 vmcheckxxx -rt tld -rn 0 -recommend
Empty the robot and re-inventory (using barcodes)	vmupdate -rt tld -rn <robot number> -rh <silo slave> -vh <host> -nostderr -use_barcode_rules -use_seed -empty_ie
Tape Media Commands
List all pools	vmpool -listall -bx
List tapes in pool	vmquery -pn <pool name> -bx
List all tapes in the robot	vmquery -rn 0 -bx \|grep 'TLD' \| sort +4
List cleaning tapes	vmquery -mt dlt_clean -bx
List tape volume details	vmquery -m <media ID>
Delete a volume from the catalog	vmdelete -m <media ID>
Change a tapes expiry date	vmchange -exp 12/31/06 23:59:58 -m <media ID>
Change a tape's media pool	vmchange -p <pool number> -m <media ID>

Media commands

List the storage units	bpstulist -U
Freeze or unfreeze media	bpmedia [-freeze][-unfreeze] -ev <media ID>
List media details	bpmedialist -ev <media ID>
List media contents	bpmedialist -U mcontents -m <media ID>
List backup Image Information	bpimagelist -backupid <image ID>
Expire client images	bpimage -cleanup -allclients
Expire a tape	bpexpdate -d 0 -ev <media ID> -force
List all netbackups jobs	bpdbjobs -report [-hoursago]
Move media from one media server to another	bpmedia -movedb -newserver <media server> -oldserver <media server>

Tape/Robot commands

List tape drives	tpconfig -d
List cleaning times on drives	tpclean -L
clean a drive	tpclean -C <drive number>
change a drives cleaning frequency	tpclean -F <drive> <frequency>
set a drives cleaning time to zero	tpclean -M <drive>
Move tapes within robot using robtest	robtest commands that can be used are as follows: s s (show slots) s d (show drives) s i (show load port) m s250 d5 (move tape from slot 250 into drive 5) uload d5 (unload tape from drive 5) m d5 s250 (move tape from drive 5 to slot 250) m s250 i1 (mov tape from slot 250 to load port 1)
List load port tapes	echo "s i q" \| tldtest -r /dev/sg/c0t4l0
List all slot contents	echo "s s q" \| tldtest -r /dev/sg/c0t4l0
List tape drive contents	echo "s d q" \| tldtest -r /dev/sg/c0t4l0
Move a tape in s100 to drive 1	echo "m s100 d1" \| tldtest -r /dev/sg/c0t4l0
Move a tape to load port 1	echo "m s100 i1" \| tldtest -r /dev/sg/c0t4l0

Archiving Commands

list archive info	bpcatlist -client all -before Jul 01 2006 bpcatlist -client all -before Aug 01 2006
archive and remove images	bpcatlist -before Jul 01 2006 \| bpcatarc \| bpcatrm
restore archive files	bpcatlist -before Jul 01 2006 \| bpcatres

Client commands

test client connectivity

bpclntcmd [-ip <ip addres>]
bpclntcmd [-hn <hostname>]
bpclntcmd [-pn]
bpclntcmd [-sv]

Diagnostic commands

/usr/openv/netbackup/bin/bpclntcmd -clear_host_cache
/usr/openv/netbackup/bin/bpclntcmd -self
/usr/openv/netbackup/bin/bpclntcmd -pn