Beyond File-Based Interfaces: Connecting to External REST APIs in SimCorp One

REST APIs – What are They?

The IT system landscape for buy-side firms typically consists of a core system, such as SimCorp One, alongside a wide range of other external applications. Integrating and connecting these applications to the core system is therefore always a key focus of implementation projects.

A basic connection is limited to the exchange of data between the applications involved via interfaces. The real goal, however, is rather to achieve a closer integration of the applications, going beyond mere data exchange.

For example, direct feedback may be required to confirm whether data has been successfully received and validated; or processing workflows within an application may need to be triggered and controlled by external actions.

An application can make such functions available to other external applications via an API (Application Programming Interface).

RESTful APIs, which follow the client-server model, have established themselves as the architectural style for APIs. The application running on the server enables other software systems (clients) to communicate with it and trigger actions via the implemented API.

REST stands for Representational State Transfer and is based on the principle that the client and server function completely independently of each other. On the server side, no sessions are open for individual clients. Each request from a client to the server is self-contained and contains all the information necessary for the request and its processing (statelessness).

Communication via REST APIs (typically) takes place over the Internet using the HTTP protocol.

This approach is ideal in today’s IT landscape, where applications are often cloud-based, making communication over the Internet using standardized protocols the obvious choice.

REST APIs and SimCorp One

SimCorp One also offers a REST-based API (called Web API) that enables other applications to access the data in the system and trigger various processes, such as those in the Front Office or Compliance areas. SimCorp was continuously expanding its functionality and scope during the last releases.

However, in this article I will focus on the reverse use case which is equally possible: controlling external applications that offer a REST API from within SimCorp One. This scenario is quite plausible given the system’s role at the core of the customer’s IT landscape. Despite its many advantages, I haven’t seen many integrations of external systems with SimCorp One that have been implemented using a REST API. Traditional file-based interfaces, for example using the SFTP protocol or record-based communication via proprietary channels such as WebSphere MQ, are still the norm.

Why is that? The concept was developed in the year 2000 and is therefore not new. Additionally, web-based applications and services based on REST API are becoming increasingly widespread. SimCorp has historically focused on other integration methods and developing the internal Web API.

Nevertheless, the implementation of a fully automated interface to external REST APIs in SimCorp One is possible with today’s functionalities!

The WebRequest Port of the Communication Server

The tool of choice for implementation is the SimCorp One Communication Server, which provides access to all internal system data and offers a wide range of options for importing external data. Data export and import can be seamlessly integrated into the SimCorp One workflow.

To communicate with the outside world, so-called “ports” can be defined in the Communication Server that connects to the outside world by various communication channels, such as file, (S)FTP, or MQ services.

Anyone looking for a REST API-specific solution among the available ports will not find one immediately but will come across the <WebRequest> port upon further searching.

The <WebRequest> port can be used to access a web service endpoint. This is a specific network address (URL) through which the web service can be addressed to exchange data or trigger operations. This utilizes internet-based communication via HTTP, analogous to common REST API services.

A service provided over the Internet via a REST API can certainly also be referred to as a web service. However, the <WebRequest> port only supports web services that use the SOAP (Simple Object Access Protocol) protocol, a standard developed by Microsoft for message exchange based on XML.

Unlike web services using the SOAP standard, REST API calls initially operate purely on a URL basis using HTTP and do not (necessarily) require XML data.

Unlike SOAP, the message format of REST-based web services is not limited to XML but can also take other formats, such as CSV (Comma-Separated Values) or the frequently used JSON (JavaScript Object Notation).

Connecting to REST APIs via <WebRequest> ports is therefore not a practical solution.

Calling a REST API using the webrequestcustom command

When searching for the keyword “REST” in the Communication Server Mediator help, you will find the <webrequestcustom> cartridge command.

According to the Mediator help, this command allows you to send out requests using various methods (verbs) in accordance with the HTTP 1.1 protocol (such as GET, HEAD, POST, PUT, DELETE). The counterpart can be a SOAP web service but also based on a REST API.

The different HTTP 1.1 protocol verbs are (also) required when connecting to REST APIs:

GET requests are used to retrieve information from the target system, while POST requests are used to send information to the target system. In addition, PUT and DELETE requests are used to update or delete existing data objects in the target system.

REST requests have a very simple structure and are therefore easy to implement by calling the webrequestcustom function. For example, a service call based on a REST API to determine GPS coordinates from a postal address could be stated as follows in the Communication Server cartridge:

<webrequestcustom method=“GET“ url=“https://example.geocoder.com/ service/csv?address=1600+Pennsylvania+Ave,+Washington+DC“ contenttype=“application/json” output=”myAddressGPS”> 

<header name=”Authorization” value=“Bearer @var(myToken)” /> 

</webrequestcustom> 

In addition to the HTTP method GET and the URL (which contains the endpoint, service path, and query parameters), the statement also includes header information to transmit authentication data.

The result of the query is stored in the variable `myAddressGPS`, and the `contenttype` parameter specifies the format (in this case, JSON).

The command line illustrates a key advantage of the REST API connection compared to other interface technologies, such as SFTP: the request and return of the result are a single process step that is executed with minimal delay.

In the example above, a so-called bearer token was used for authentication. This is a cryptographic string that serves as credentials for accessing protected API resources and is a central element of modern security concepts such as OAuth 2.0.

Bearer tokens must be requested from and generated by the target system in advance. A call to do this using the <webrequestcustom> could look like the following:

<webrequestcustom input="MyCredentials" method="POST" url="https://example.webservice.com/auth/openidonnect/requesttoken" contenttype="application/x-www-form-urlencoded" output="MyToken"/> 

The content (body) to be transmitted in POST requests is specified via the “input” parameter and contains authentication data such as the username and password. This sensitive data therefore only needs to be transmitted once; all subsequent requests can identify themselves using only the obtained Bearer token.

For security reasons, the Bearer token has a limited validity period, typically between 15 and 60 minutes. After that, a new token must be obtained. To be able to reuse the Bearer token, I recommend storing it in the Communication Server for example, via a persistence port - along with its expiration time.

A fundamental prerequisite for the <webrequestcustom> call is, of course, that the machine on which the Communication Service is running can communicate with the external address.

For security reasons, communication from the corporate infrastructure to the outside world often takes place via a proxy service. This is also supported by the <webrequestcustom> function and can be specified using the “proxy” parameters.

Processing in the Communication Server is based on XML. For many REST API interfaces, communication often takes place using the JSON format. Like XML, JSON contains descriptive information but is more compact than XML, which reduces the size of the data packets to be transmitted.

Conveniently, the Communication Server includes built-in functions to convert data from JSON to XML and vice versa.

<json2xml> 

Or

<xml2json> 

These two commands make it easy to implement a workflow in the SimCorp One Communication Server that handles the data.

Implementing complex REST API requests using XSLT Stylesheet Scripts

The <webrequestcustom> function reaches its limits if the integration of the external system requires the exchange of large amounts of data using complex data structures. These are often implemented via REST API calls in the form of multipart/formdata requests, whereby different data packages can be uploaded either directly or from files.

The Communication Server documentation for the <webrequestcustom> function does not specify how such requests should be implemented. The same applies if one wishes to download data from the REST API counterpart into files. The function currently appears to support only simple structured requests.

There is however a way to specifically extend the functionality of the Communication Server: the integration of scripts into XSLT stylesheets, which are executed within the Communication Server cartridges workflow.

The use of scripts in XSLT is a general functionality provided by the Microsoft .NET Framework, with C#, JavaScript, or Visual Basic supported as programming languages.

When the stylesheet is loaded, the script functions are compiled into MSIL (Microsoft Intermediate Language) and executed as part of the XSLT transformation.

The underlying stylesheet must be defined in XSLT v1.0. In XSLT stylesheets version 2.0 and higher, which are also supported by the Communication Server, it is not possible to embed scripts.

When using a C# script, the entire range of Microsoft .NET functions is available, including the generation of HTTP requests to call REST API interfaces. The only requirement is that the corresponding .NET library file (.dll – Dynamic Link Library) is available in the Communication Server.

To generate REST API calls, objects from Microsoft’s System.Net.Http class library are required.

The .dll file is generally part of the SimCorp One installation (located in the \bin folder of the respective instance). If necessary, it can also be obtained and installed separately via the Microsoft .NET Framework.

The .NET class libraries are described in detail in the Microsoft .NET online documentation, often with examples in C#.

SimCorp One integration

We now want to integrate the REST API interface into the existing processing workflows. There are several ways to do this in SimCorp One.

If requests are to be triggered manually by users, I prefer to use extraction setups that are initiated via an Online Job. This is convenient, and when executing the extraction setup, the user can specify custom parameters.

Integration is also possible via Batch-Jobs Groups or even Auxiliary Status Jobs for transaction-based interfaces. For processes that are part of daily operations, the interface could be integrated into the Fund/Portfolio STP workflow.

Feedback from the API system, in turn, can be imported into SimCorp One via the Communication Server using the standard methods (e.g. creating Alerts or using classic import via Data Format Setups). Alternatively, notifications can be sent outside the system, for example via email.

The possibilities are nearly endless!

Summary

Integrating applications with SimCorp One via a REST API offers a number of advantages:

• It is a cross-platform, standardized interface

• Thanks to web-based communication, it can be used seamlessly in cloud environments. Only minimal adjustments are required when migrating or porting your systems

• The integration is independent of third-party systems for message transfer (such as file transfer tools or MQ infrastructure)

• Secure data exchange is ensured using HTTPS for encrypted transmission and authentication concepts (OAuth 2.0)

• Direct data delivery with immediate feedback from the recipient system

• In addition to pure data exchange, processes can also be triggered directly in the connected target system

• Implementation in the Communication Server makes virtually any use case feasible

• Integration of feedback into SimCorp One, for example via alerts. Integration into the fund/portfolio or transaction STP workflow is also conceivable.

If large amounts of data need to be transferred, this is not a fundamental obstacle. The architecture of the recipient system must be designed accordingly. The security concept for the connection should also be carefully reviewed (authentication concept, encrypted transmission).

The biggest obstacle in implementing a REST API connection is the requirement that the partner system is offering such an interface at all. If this is the case, a web-based connection should definitely be considered.

The SimCorp One Communication Server contains the required toolset for connections to external APIs. The easiest and most flexible approach is to use the built-in webrequestcustom function for simple REST API requests and (if needed) customized Xslt script functions for more sophisticated tasks.

---

Interested in implementing REST API integrations in your SimCorp One environment? We'd love to talk.

Or alternatively, if you'd prefer to go the other way around and make SimCorp One functionality and data available to other applications via its Web API module, get in touch with our team and let's explore what's possible together.

About the Author

Thomas Hoelle has been working on SimCorp Dimension / One implementation projects around the world for 24 years, always close with the clients and always hands-on. He has held numerous roles in this capacity, both as a Senior Project Manager and Chief Business Consultant at SimCorp and, since 2014, as an independent consultant.

He has overseen numerous new implementations of the software, often as a front-to-back system, and possesses extensive business expertise in both asset management processes and the legal framework. He leverages this knowledge, combined with a deep understanding of the SimCorp One system and the technology behind it, to ensure the best possible implementation of each client’s specific requirements. In doing so, he works closely with Dimensional Community.