How to Prove Your IBM i is Recoverable without a Real DR Test

How good do you feel about your backup strategies? Did you know you can perform a simple operation two to four times a year to put much of your uneasiness to rest? It’s true. Simply audit your backups. 

What Is an Audit?

Auditing involves running independent checks of your backup strategies to look for holes in your plan. The effort can take as little as 30 minutes every three months or, at the very least, twice a year. Auditing your system requires a thorough understanding of what a good backup strategy entails. On every copy of IBM i (we say IBM i because you might be using a system with logical partitioning), you should execute these save commands:

• Save System (SAVSYS)
• Save Security Data (SAVSECDTA)
• Save Configuration (SAVCFG)
• Save Library (SAVLIB)
• Save Document Object (SAVDLO)
• Save Object (SAV)

SAVSECDTA and SAVCFG be executed daily to save your application data and system configuration changes. SAVSYS should be executed monthly or after major PTFs have been added, as should SAVLIB and SAV for IBM-supplied libraries and directories, respectively. SAVDLO is moving towards obsolescence as more people are using directories to store unstructured or non-library type objects. This command may only need to be run once a month. For every copy of IBM i, there are different strategies and, of course, variations to these commands. For more on this, refer to the “Planning a Backup Strategy,” available on IBM’s iSeries Information Center website.

In IBM i, you can examine object descriptions, history log entries, PRTSYSINF, and certain data areas to quickly determine if your system is being backed up. However, this doesn’t guarantee you’ll know how to find the tape that contains the data. That’s something you’ll get only with a good manual logbook or one of the tape-management solutions on the market such as Robot Save. For expert tips on media management and more, check out the Recovery Without Disaster planning guide for IBM i.

Save Data Areas

The QSYS library contains a series of data areas that are updated every time you run certain save commands. While the following list is not all-encompassing, you can check these data areas to see if at least some of the save commands are being executed:

• QSAVALLUSR (SAVLIB *ALLUSR to save all user data libraries)
• QSAVGFG (SAVCFG to save hardware configuration data)
• QSAVIBM (SAVLIB *IBM to save IBM libraries)
• QSAVLIBALL (SAVLIB *NONSYS to save all libraries)
• QSAVSYS (SAVSYS to save IBM i, licensed internal code, configuration data, and security information)
• QSAVUSRPRF (SAVSECDTA to save profiles and authority information)
• QSRSAV21 (Displays with Save option 21 is used; shows dates, times, commands, etc.)

You might think you could examine the contents of the Data Area (DSPDTAARA) on any of these data areas to see when the command was executed, but that’s not the case. You must use the Display Object Description (DSPOBJD) command, DSPOBJD OBJ(QSYS/QSAV*) OBJTYP(*DTAARA), for the data area and look at when the save date value on this object description was updated (see Figure 1). This displays the last time that one of these commands was executed. Within seconds you can tell if any system backup strategies have holes. Note that these data areas do not address the SAVDLO and SAV commands. Also, QSRSAV21 is in QUSRSYS unless you have never done the GO SAVE option 21 on your system.

Image

Figure 1: DSPOBJD QSAV* OBJTYPE(*DTAARA)

PRTSYSINF

The Print System Information (PRTSYSINF) command is handy for summarizing, by library and folder, when these data holders were last backed up. The downside of this command is that it generates about 20 other reports. Ideally, IBM would add a parameter to the command or allow placing this information in a database file. The spooled file, QPEZBCKUP, contains the information, and there should be two spooled files on your system with this name. This command does not address SAVSYS, SAVSECDTA, SAVCFG, or SAV saves, nor does it address whether individual objects were backed up.

Critical business data is stored in libraries, directories, and sometimes folders. Don’t ignore verifying good backups for directories and for the OS! These areas of the system can cause a lot of grief if you don’t have good backups.

History Logs

IBM i history logs can provide good auditing information about how well backups are being executed on your system. Unfortunately, most users set up their systems to delete history logs after a few days. The more history logs you have, the better you can audit your backup. You should have at least seven days of history logs to examine.

To view a history log, execute the Display Log (DSPLOG) command. By default, this command shows the history log in a chronological sequence. It’s a lot of information to look through when searching for a problem, but if you know the IBM i message ID you’re searching for, you can speed up this process. IBM i backup commands issue message IDs that are registered in the history logs. The following list shows message IDs for various save commands:

• CPC3701 (Sent for each library that’s completely saved to media)
• CPC3722 (Sent for each library that’s completely saved to a save file)
• CPC9410 (Completion message for SAVDLO saved completely to media)
• CPC9063 (Completion message for SAVDLO saved completely to save file)
• CPC370C (Completion message for SAV save of directory completely to media)
• CPC370D (Completion message for SAV save of directory completely to save file)
• CPF3771 (Number of objects saved/not saved during SAVLIB command, issued if not a complete save)
• CPF3837 (Number of objects saved/not saved during SAV command, issued if not a complete save)
• CPF9410 (Number of objects saved/not saved during SAVDLO command, issued if not a complete save)

The DSPLOG command allows you to specify a date and time range, and to limit the list to a specific message ID or IDs—DSPLOG PERIOD((*AVAIL022702)) MASGID(CPF3837). You can also direct the output to a printer or an interactive display. The default for the command is to display the information. If you want a hard copy, specify an output queue on the parameter for output. If you display the history log interactively, you can position your cursor in a message and press function key F10 to view details about the save activity.

Displaying the history log guarantees you’re executing these commands on a regular basis and shows the jobs that are executing the backups. It is the first option (thus far) that shows any history of executing the SAV command on a system. It’s also a good way to look for objects that aren’t being saved. The message IDs given may vary for different IBM i release levels, and there are more message IDs that might be worth looking for in the backup and recovery area.

If you know how to use SQL, you can access this information via the QSYS2.HISTORY_LOG_INFO table, but you must be on IBM i 7.3 or higher.

Object Descriptions

Every object on the system can be updated with the last save date information from one of the save commands. Every save command has an Update Save History parameter. Most commands default to *YES as shipped by IBM. The SAV command defaults to *NO.

The end result of updated save information is that you can dump an object description into an outfile (a database file that some commands can create as output) and then query the results to see if it was ever saved. For example, all of the library descriptions on the system reside in the QSYS library. Every time a SAVLIB command is executed, the save information in each library description is updated. Thus, you can execute the command, DSPOBJD OBJ(QSYS/*ALL) OBJTYPE(*LIB), send the results to an outfile, and then query those results. In the query, you might look for libraries that were never saved. Technically, you could use the same DSPOBJD command and put all output for every object in all libraries into an outfile and query/SQL these results to locate objects that never have been saved.

Outfile Support on Save Commands

Every save command supports the OUTPUT parameter, which allows you to generate a database file that summarizes the results of each save operation. If you execute the SAVLIB command against the QUSRSYS library, the outfile lists all of the IBM i objects that were and weren’t saved during this operation. Then you can query/SQL just this database file for objects that weren’t backed up.

Most of the tape-management packages that run on IBM i use the OUTPUT parameter to build an inventory of which objects are on which volume. It’s handy for keeping an up-to-date inventory of what was saved. Although the SAV command doesn’t support outfiles, you can direct the output to a report or stream file. The report is useful because you can limit it to only the errors that occurred during the SAV operation. These errors are generally the results of object locks. This can be beneficial for auditing purposes.

Outfile support on a save command only helps you if you have the feature turned on for every backup. If you have it turned on, make sure the output is being examined, otherwise this feature is of little use and causes overhead for your backups.

Outside the System

As part of this process, you should audit the facility where you store your offsite media. You also should verify that you have a proper procedure for recovering your data, as well as the proper materials to complete the restoration. Keep the contact information for staff members who can help you handy in case you have any difficulties. Finally, add IBM’s “Recovering Your System” manual (be sure to get the right version for your IBM i OS level) to your resource library. The manual is also available on the iSeries Information Center website.

While auditing can’t replace actual recovery testing, a thorough audit of your backup process can help you avoid problems when you do test your recovery procedure.