About NoSQL Databases
This subsection discusses the main points of NoSQL databases. Note that NoSQL databases do not support relationships and custom SQL statements.
Data structures
JSON Schema
MapForce uses JSON Schema to describe the structure of NoSQL database components. Note that in MapForce a data structure is an essential prerequisite for data mapping. Instead of fields in a database table, the user must embed a JSON schema to get a mappable structure. The schema can be user-created or auto-generated from the database contents. See details in Assign JSON Schema. MongoDB supports storing a schema in the database to validate documents.
Data in JSON and BSON
MongoDB stores documents in BSON (Binary JSON). CouchDB and CosmosDB store data in JSON. See JSON and BSON to find out more about their differences.
BSON function library
The bson function library is available in the Libraries window, which allows creating and manipulating BSON types. To find the bson libarary, type bson in the text box located at the bottom of the Libraries window.
Component Settings
The Component Setting dialog box enables you to configure various database-related settings, such as connection and login settings, timeout, transaction handling, and tracing settings. Depending on the database type, different settings may be available. To find out more about the component settings, see DB Component Settings.
Querying, filtering, joining DB data
DB Query
Currently, there is limited support in the DB Query pane. The DB Query pane allows you to see the list of available collections in your database, but query execution is not supported.
SQL/NoSQL Where/Order Component
You can filter and sort database data using the SQL/NoSQL-WHERE/ORDER component.
Filtering data in MongoDB and CouchDB
In MongoDB and CouchDB, JSON-based syntax is used to filter and sort database data; the "@" sign is used instead of ":" before a parameter in the query filter. To find out more about syntax for querying documents in a collection, see the MongoDB documentation.
Filtering data in CosmosDB
CosmosDB uses SQL as a query language.
If the name of a container in your CosmosDB has special characters (e.g., +, -, @, etc.) or is a keyword, this name will be unusable in queries, and you will get a syntax error from the database. To avoid potential issues, MapForce uses the following syntax for CosmosBD queries:
SELECT * FROM ROOT AS c WHERE c.<field> ORDER BY c.<field>
The ROOT keyword in the FROM clause references the current container you are querying. Since the ROOT keyword cannot be used to address fields in WHERE and ORDER BY clauses, the container is given the alias name c that is assigned to the container by means of the AS keyword. Assigning the alias name to the container makes it possible to filter and sort the container's contents.
Note that in the WHERE/ORDER dialog, all field references must be prefixed with the container alias name c (see example below).
SELECT * FROM ROOT AS c WHERE c.age > 20 ORDER BY c.name
Join component
The Join component works in non-SQL mode only.
Target DB actions
NoSQL databases support four actions for target DB collections. When a NoSQL database is used as a target component, the button (the Action button) appears next to the collection name. The Action button changes depending on the chosen action. When you click on this button, the dialog box Database Collection Target Action opens and suggests choosing one of the four target actions: Insert, Replace, Replace or Insert, and Delete (see screenshot below).
The description of the target actions is summarized in the list below:
•The Insert action: The mapped data will be inserted into the target database in addition to the data (if any) that is already in this target database.
•The Replace action replaces the whole document in the target database if its _id matches the one in the mapped data.
•The Replace or Insert action replaces the whole document in the target database if its _id matches the one in the mapped data. Otherwise, the Insert action is performed.
•The Delete action deletes the whole document in the target database if its _id matches the one in the mapped data.
Besides selecting a target action, you can also enable bulk transfer and specify the batch size.
Primary key
Note that for MongoDB, the primary key is always _id. In CouchDB and CosmosDB, the primary key is the id field. The primary key is relevant for replace, merge, and delete operations.