In today's post we are delving into the primary tool for importing data into SimCorp Dimension; the Filter Tool Box, also called Data Format Setup. We are going to help you explore and take a closer look at some of the lesser-known parts of the tool. We start by looking at the option of how to save error records.
The default is to save your error records in a so-called SC format. What this means is that they will be saved in a data file and in an order defined by the field SC column in the Data Format Setup. This column is hidden in the Field List tab and can be shown by adding the field. When you save a new setup it will set values in this field by the order they were entered. If you add a new field, it will be added as the last value to the SC column, regardless of whether this field has a low input order.
If you want a more recognizable format for error records, the better option is to use the Original Data Format option. This format creates error records in the original character set with headlines and comments from the original file.
Another option in the same view is the option to ignore data for closed fields. This is not set by default. This can be a convenient one to set from the beginning and many people do this without thinking twice. Generally speaking, this can be ok, but you need to ensure you know the data and how it is handled in SimCorp Dimension. Just because a field is closed, it does not mean that no value should be filled in for the type of data set you are trying to import. Maybe another field is set wrong, causing the field to be closed for input which you did not anticipate. To avoid mistakes, we suggest leaving this as per the default option, and only check the ignore data for the closed fields option if you have thoroughly tested your scenarios and are certain it is ok to do so.
The pre-run and reformat functions are often avoided, but you don’t have to. The help text on each option is quite good and can be very powerful instead of using a translation or formula and in some cases, it can even be the only available option. One pre-run function we want to point out here is the option to use other delimiters besides what is listed in the separation type drop-down on the main window, and even simple XML. In other words, before changing the way data is sent or start looking at a communication server solution, check if a pre-run function can solve the problem for you first.
A great feature SimCorp Dimension has to offer is the option to create a data file. For any Data Format Setup you have built, you can open the corresponding window (for example the bond static data window) and select a number of records from that window. If you select Create Data File from the functions menu, the DFS will pull the data from the window you loaded data into, and put it into the file format specified on the Data Format Setup. This becomes a quick and easy way to transport test data from one environment to another. Note that there is a volume limitation, so you should not attempt to do this for large data volumes.
Lastly a few tips on the Example Records sub-window. The Reuse Windows function is useful when testing the same window on several records to avoid that SimCorp Dimension opens the same window multiple times. With this option set, it will clear the existing window and load the next data record into it. The file menu to View File Columns gives a nice overview of which columns your external data records are in and compares multiple records from the file easier. And the file menu ‘Reload file’ is useful if you make changes to the file to fast reload the file and update the example record viewer.
With all this in mind, whether you are new to it or not, you should be able to utilize the Filter Tool Box with a little more ease. If there are any further questions or comments you may have in regards to this topic don’t hesitate to let us know.