Web Services for Processing Data

Web Services Calls for Data Integration & ETL

  • Call SOAP web services
  • Call RESTful web services
  • Import OpenAPI definitions
  • Support for generic HTTP web services
  • Flexible settings and input parameters
  • Pass results to mapping component or file
  • Shopify/GraphQL support
  • Auto-generation of JSON/XML schemas
  • Implement calls to AI APIs for data processing
  • Pass results to mapping component or file
  • Support for WS-security
  • Support for OAuth 2.0
  • Building SOAP web services visually

Web Services Calls in Data Mapping

In addition to calls to Shopify/GraphQL APIs, MapForce supports calling SOAP and REST web services directly from within a mapping. You can insert a Web service call into a mapping and supply input parameters and username/password or other authentication. Connect output components to the Web service to pass the result to another component or write it to a file.

MapForce also supports generic HTTP Web services that typically carry custom request or response structures in the message body. MapForce supports both JSON or XML data in the request or response body, allowing you to call virtually any HTTP Web service that requires or returns XML or JSON structures.

MapForce Web Service

Manual definition of settings in the Web Service Call definition dialog allows developers to define settings based on a template URL. This is a convenient step when developers test and refine Web service calls in a Web browser window, since they can copy the URL from the browser to become the template.

Looking for a real world example? Here is a detailed tutorial on Web Services Data Integration.

Support for Web Services Security

MapForce supports authentication based on the WS-Security (Web Services Security) standard via client certificates and calling Web services via HTTPS. The parameters needed to establish secure communication are defined as part of the data mapping so execution is automated.

Support for OAuth

OAuth 2.0 is an open standard for authorization that provides a way to provide client applications secure, delegated access to web resources using access tokens rather than passwords. MapForce includes support for calling Web services that require OAuth 2.0 authorization.

This is enabled in the Credentials Manager (opened by right clicking in an active mapping project). OAuth authorization details can be transferred to other execution environments such as MapForce Server and FlowForce Server with the help of credential objects.

Calling APIs for AI-powered Data Integration

Since many AI systems, such as OpenAI’s GPT-4, are available via API, it is immediately possible to integrate their functionality in data transformation projects in MapForce.

Using built-in, no-code tools to define web services requests in MapForce, it’s easy to set up calls to an API, including the OpenAI API, Azure OpenAI API, AWS AI Services, and so on, to enable AI-powered data processing in any data mapping project.

Using web services calls for AI-powered data transformation

The broad steps for configuring AI functionality in MapForce include:

  • Insert a web services component in your data mapping project to process source data being written to a target data structure
  • Configure the web services call settings with:
    • The API URL
    • Your organization’s login credentials
    • The JSON request and response schemas that correspond to the API call (these can be autogenerated by MapForce or XMLSpy)
  • Continue configuring the mapping project as required

Alternatively, you can import the OpenAPI definition (below) of a RESTful service and MapForce will automatically process the structure of the call, removing the need for manual configuration.

Import OpenAPI Definitions

Setting up a REST web services call requires supplying details including the web service URL, a request method (e.g., GET, POST, PUT), request and response structures (e.g., XML, JSON, custom MIME types), and parameters. You can add this manually, by specifying a WADL file or URL that includes the details, or by importing an OpenAPI definition.

The OpenAPI Specification (OAS) is a standardized format for defining and describing RESTful APIs. It provides a language-agnostic way to outline the structure, endpoints, request and response formats, parameters, authentication methods, and other aspects of an API.

When you specify the OpenAPI file that describes the web service you require, MapForce automatically processes the structure of the call and displays it as a graphical data mapping component. Removing the need for manual configuration significantly streamlines the process of integrating with web services.

Import OpenAPI definition for data integration