In this piece, we will take a closer look at an integral part of automation in SimCorp Dimension, the batch functionality. Although this part is well known to most technical users of SimCorp Dimension, there are nevertheless some slightly lesser-known parts that we hope to highlight here.
When you open the Batch Job Group, the default grid view shows the batch job ID and name, sort, frequency, and active flag. But information about the batch job task and the ID of the underlying window it executes is not immediately available unless it is added. You can do this by right-clicking on the grid > View Properties and then choosing Select Fields. Add the fields 'Batch Task' and 'Key Value 1'. This makes it a lot easier to navigate a Batch Job Group without having to drill down to the individual Batch Job window. Key Value 2 is seldom used, an example of this is the Batch task ‘Transfer Reformatted Data to Message Queue’ which allows you to specify a filter in Key Value 1 and the message queue in Key Value 2.
Two more fields are worth mentioning in the Batch Job Group window, they can be found under ‘Select Fields’; the ‘On OK go to’ and ‘On Error go to’ fields. Having several Batch Jobs specified in a Batch Job Group, if one fails, you can direct it to go to another Batch job further down the list rather than the next one. If there are no errors, go to yet another Batch Job. The value to enter in these 2 fields are the ‘Sort’ number of the grid view.
New installations using Simcorp’s standard installation will notice that the log file in the Batch Job Group window is the same. This is made possible in the Reference File by specifying a Component type as ‘SimCorp Dimension config’ with a value batch, in the ‘Name Components’ section (the grid in the Reference File window). When running the Batch Job Group, this will insert the name of the Batch Job Group when it is being executed, thereby reducing the number of setups required. Note you can use the ‘SimCorp Dimension config’ to specify a folder and/or be part of the file name, thereby creating an easier structure for batch jobs. Remember to check the box ‘Create directory if missing’ in the Reference file, so when you set up a new Batch Job Group you don’t have to worry about it failing due to a missing folder being set up on the network.
The log file of Batch Job Group the log file contains a summary of the execution. When troubleshooting an issue, it might be useful for the investigation to generate a more detailed log of the execution. This is possible if you activate ‘Keep temp lines in log’ on the Batch Job Group found in the Functions menu under the Extra Information sub-window. Note the details generated can be very long, so this should be activated with care.
Under the Extra Information window for Batch Job Groups, you will also find a check box to ‘Allow overruling of parameters’. Setting this field allows you to input parameters in the ‘command line’ when starting a Batch Job Group such as -ba_parm=xyz. This is especially useful for Data Extractors using variables, where parameters from the batch command line can be used to feed into the variables of the Data Extraction.
The list of batch job tasks in the single batch job window can be overwhelming and there is no way to find part of a task by typing in the field. This is what the field to the far right does. Try for example to type ‘delete’ in the field, and then press the ‘arrow down’ in the batch task next to it, and you will notice a lot shorter list of all the tasks that contain the specified word.
Executing batches via Batch Queues can streamline the execution process. The overhead of starting a batch is reduced, it is easier to scale and automate the process and exception handling becomes easier. Add a few more fields in the Batch Monitor which are hidden by default to show ‘Last execution time’ and ‘workspace size’, both helpful when looking for potential issues. For each job in the window, you can right-click to start or stop a job and view historical runs and the latest log. The Batch Queue Monitor can be used for researching errors, the function menu can remove a job from the queue, change priority, or change queue (if you have multiple queues setup). In recent releases, it is now also possible to execute a batch job group on a queue as a different batch user, provided multiple batch users have been set up.
Lastly, we would like to mention that it is possible to start a Batch Job Group on a Batch Queue from the Communication Server. Use the SCIn port with the SCIn_BatchQueue label to insert a Batch Job Group into a batch queue.
We hope this has offered some insight into the most effective use of Batch Jobs and Queues. Let us know if you have any queries or comments, we look forward to hearing from you.