OpenVMS DEBUG: Often underutilized and unappreciated

We are a diverse group. Some IT professionals spend all of their time developing software. Others never even see the text of a source program, other than a scripting language such as DCL or PERL, outside of a textbook. Still others are strictly involved in operations and have no interest in programming matters of any stripe.

Going forward, there will be several ongoing threads in this column. One thread will emphasize issues of interest to developers. Another ongoing thread will be topics of interest to those who write DCL and other scripts on OpenVMS. Management and operations issues will be a third thread. In addition to these three ongoing threads, other topics will make appearances.

This column is the first in a series of columns about the capabilities and use of the OpenVMS debugger, DEBUG. The OpenVMS Documentation set includes an extensive treatment of DEBUG in the “VSI OpenVMS Debugger Manual”. The manual is available on the OpenVMS www site as a PDF.

DEBUG is one of the fundamental software development tools on OpenVMS. At its most elemental, DEBUG allows a programmer to interrupt program execution at pre-selected points in the source program, display and possibly alter variables, and continue execution.

Even this extremely limited use of DEBUG is a quantum leap from inserting and removing "print" statements for debugging. The ability to set breakpoints and examine storage is an elemental aspect of debugging that far predates the VAX.

It is simple use DEBUG on a program. The basic steps are the same for all of the OpenVMS processors:

compile with the /DEBUG and /NOOPTIMIZE qualifiers (DEBUG can be used with optimized code, but it is more, sometimes very much more, difficult to correlate source code to executable code if /NOOPTIMIZE is omitted)
link the executable image using the /DEBUG qualifier
execute the program using the RUN command

When the program image is activated, control is first given to DEBUG, which then invokes the actual user image.

In its simplest form, the DEBUG can be instructed to merely run the program, using the GO command. When the program finishes execution, control will be returned to DEBUG and the user will receive a DBG> prompt (This process can be automated, which will be the subject of a future column).

There are at least four different ways in which User-mode DEBUG can be used by a developer. The most basic use of DEBUG is the original line-by-line mode essentially unchanged from the original release of VAX/VMS in 1977. DEBUG has not been static since that time. The intervening three decades have seen the addition of - full-screen mode (Chapter 7), DECwindows support (Part III, Chapter 8, et seq.), and PC client support (Part IV, Chapter 11). While the mechanics differ slightly, the underlying strengths of OpenVMS DEBUG remains unchanged.

It is worthwhile to start with an extremely basic example of using DEBUG. To ensure that all can make use of the example with a vanilla version of base OpenVMS, this column will use MACRO-32 for examples. MACRO-32 is symbolic version of the native VAX instruction set. On VAX processors, MACRO-32 is a classic macro assembler. On Alpha and Itanium processors, MACRO-32 is a compiled by architecture-specific compilers (AMACRO for Alpha; IMACRO for Itanium) into equivalent programs. While today's usage of MACRO-32 is limited, it does have the advantage that the needed assembler/compiler is part of the base OpenVMS distribution, so every OpenVMS system can be used to work with these examples. (versions of analogous sample programs in other popular languages are available):

        .ENTRY  TEST,^M>R2>
        MOVZBL  #157,STATUS     ;Set the value of STATUS to 157
        MOVZBL  #SS$_NORMAL,R0  ;Set "Success" return code.
        RET                     ;Return to caller.
        .PSECT  DATA,WRT,NOSHR
STATUS: .BLKL   1               ;One Longword (32 bit) storage location.
        .END    TEST

This source program needs to be assembled (compiled on Alpha or Itanium). The DCL command is the same in all cases: $ MACRO/DEBUG [/NOOPTIMIZE] TEST (on Alpha/Itanium, add the /NOOPTIMIZE). Next, the object file must be converted into an executable image using the OpenVMS LINKER. The DCL command to do this is: $ LINK/DEBUG TEST.

Using the DCL command: $ RUN TEST will now automatically start the program with DEBUG activated.

At the DBG> prompt, enter SET BREAK %LINE 2, then ENTER, then SET BREAK %LINE 3" and ENTER. DEBUG will gain control before the first line of executable code (on VAX, this will be line 10, the first MOVZBL instruction; on ALPHA/ITANIUM, the first actual executable instruction is the generated preamble, so the first executable instruction will appear to be associated with the .ENTRY statement). To actually start the program, enter STEP.

After entering STEP, DEBUG should receive control before the first executable line of the program (line 10; MOVZBL #157,STATUS). Note that since the object file is MACRO-32, the case of the symbol name is not significant, as it would be in C/C++ and related languages. Enter the command EXAMINE STATUS to display the current value stored in STATUS. It should be 0 (on OpenVMS, non-initialized storage is initially set to zero for security reasons). Then enter the STEP command to execute one source statement. DEBUG will come back with break at %line 11. Now enter the EXAMINE/DECIMAL STATUS command again. The value will now be 157 (in decimal, since we specified /DECIMAL on the EXAMINE command.

Now type GO, and the program will complete, returning to the DBG> prompt. CTRL-Z will now return control to DCL. A log file of this entire dialogue is included in the example archives, as is a DCL command file that performs the entire example. In future columns of this thread, we will further explore more involved examples of how DEBUG can be used to advantage.

Edited to relocate documentation hyperlinks to VMS Software Inc. www site (February 21, 2020)

URLs for referencing this entry

Long:	http://www.rlgsc.com/blog/openvms-consultant/openvms-debug.html
Short:	http://rlgsc.com/r/20090203.html