Single Command Batch Jobs: An Alternative to SPAWN or Command Files

Small command files clutter far too many directories. It is a tendency I see both within my own project directories and in those belonging to colleagues and clients. This plethora of command files implements a variety of tasks. Some tasks require logging; other tasks are too long in duration to run interactively. This begs the question: How can normal OpenVMS facilities eliminate the need for these files?

Examining these files reveals that many consist of but two lines: a single DCL command preceded by a SET DEFAULT to a directory, often the directory containing the command file itself. On occasion, the files have a third line, EXIT. The commands are typically simple DCL commands with potentially extended execution times (e.g., ANALYZE/DISK_STRUCTURE/READ_CHECK, DELETE/ERASE, PURGE, ZIP). Is there a better way to manage this collection of tasks? Is there an approach or technique that can eliminate the need for these ensembles of small files? This is the genesis of the single command batch concept.

OpenVMS is known for its architectural power. OpenVMS world relies on context and facilities provide a solid foundation to for new techniques, without altering that foundation. The fundamental facilities are well-designed and robust. This allows one to do far more with less. This is not the case on many other operating system platforms. Running single commands in the batch queues is one case: two simple pieces of DCL can eliminate the need for hundreds of separate files. As is often the case, a solid understanding of the fundamentals allows just such an extension.

OpenVMS has many ways of running programs beyond simply typing a command at the DCL prompt. Programs can also be executed from:

a command file
a command file submitted to batch
a detached process (privileges required), or
a sub-process using the DCL SPAWN command

Each of these techniques has strengths and weaknesses. SPAWN can be used to run single commands, but an interruption of a parent process (e.g., a network glitch; Cntrl-Y) will abort the sub-process. Some of these problems can be solved by changing settings. Communications interruptions can be reduced by using virtual terminals, setting /INPUT=NL: can prevent Cntrl-Y from stopping sub-processes. However, not every system is set up for virtual terminals, and some events cannot be evaded. Virtual terminals are also not a solution if the connection is not re-established within the reconnect interval.

Detached processes require elevated privileges (e.g., DETACH). Ordinary users are not granted elevated privileges. If the system crashes, detached processes are gone. There are also no inherent mechanisms for restarting detached processes.

Batch jobs are completely independent of the submitting process. If system operation is interrupted, uncompleted batch jobs can be re-started automatically; either when the system returns to operation or on another member of the OpenVMS cluster. However, the batch facility processes command files, not individual commands. It is thus common for a user or system manager to accumulate a large collection of extremely short command files for various tasks. This creates clutter, along the lines of an electronic version of a kitchen tool drawer. Locating any particular tool is a challenge.

The ability to submit single commands as batch jobs without the need to create a temporary command file would eliminate this clutter. OpenVMS provides us the components to create precisely this capability.

A new paradigm is unnecessary. The existing batch queue model is well-suited to this use. The existing batch queue facilities can serve as the foundation for a single command execution mode. In effect, the result is a SUBMIT command with a literal operand. Such a command is reminiscent of the literal operands present in many machine instruction sets, including the PDP-11, VAX, and Alpha. With this facility, a long running operation (e.g., a DELETE/ERASE of thousands of large files) can be sent to a batch queue with no need for a small command file.

This new capability can be completely implemented by supplementing the existing batch facility with two small DCL procedures. SUBMIT is already tested and fully supported. Normally, batch jobs are started by SUBMITting a command file. However, it is also possible to provide a DCL command or parameter list to a batch process as well. In this way, the current default device and directory could be transmitted to the batch job using the parameter list. A generic command file accepts these parameters and actually submits to the batch queue; it is a library DCL procedure that acts as a generic execution wrapper. This wrapper retrieves the needed information from the parameters and executes the actual supplied command. This approach enables a command be executed in batch without creating a custom command file.

Two procedures are needed: one that submits the job, and one that actually runs the command. Some of the basic SUBMIT command qualifiers (e.g., /AFTER, /LOG_FILE, /NAME, /QUEUE, and /USER) are useful, and can be supported without difficulty.

This approach is more than theoretical, it has been implemented in a pair of demonstration command files: SUBMIT_SINGLECOMMANDJOB.COM and RUN_SINGLECOMMANDJOB.COM (both files are available from my firm's www site in the demonstration area at http://www.rlgsc.com/demonstrations/single-command-batch-jobs.html).

SUBMIT allows the user to supply eight user-specified parameters. These parameters are supplied to the batch job by the queue manager when the batch job begins execution. SUBMIT_SINGLECOMMANDJOB.COM and RUN_SINGLECOMMANDJOB.COM use the first two of these parameters to transfer the user’s current default string (including both device and directory information) and the actual command to be processed to the batch job. The remaining parameters are available for other uses. In other words, P1 contains the user environment and P2 contains the batch command.

This approach is completely safe. No elevated privileges are needed to use these command files. These files may be used by any user with sufficient privileges to run a batch job. Therefore there are no additional security implications. These procedures are fully safe for students to use in an educational setting. If desired, system managers or team leaders may install these files in project or system-wide directories for common use. Ordinary OpenVMS users with access to DCL and batch queues may use these files from within their own directories. There is no risk.

Single command batch is another example where OpenVMS provides more capabilities to both system managers and regular users than is apparent at first glance. This is another example of the inherent power and extensibility of the OpenVMS architecture.

URLs for referencing this entry

Long:	http://www.rlgsc.com/blog/openvms-consultant/single-command-batch-job.html
Short:	http://rlgsc.com/r/20100311.html