DataMosaicDocs · Connecting to data
REST APIs
Calling an HTTP service — auth, the response shape, and reading more than the first page.
An API source makes one HTTP request and turns the response into rows.
The bar above the tabs holds the method — GET, POST, PUT, PATCH, DELETE, HEAD, OPTIONS — the URL, and the Send button. Pressing Enter in the URL runs it.
Mosaic only ever reads, so PUT, PATCH and DELETE are refused. POST is refused too until you switch on This POST only reads, under the URL — for a search or query endpoint, such as GraphQL, that answers with data and changes nothing.
A GraphQL mutation is refused whatever the method and whatever the switch says — in the body, in a form field or in a query parameter: "This source sends a GraphQL mutation, which changes data on the other end, and Mosaic only ever reads." Queries run as normal.
There is no Connection tab and no Test connection button. An API source is verified by running it, and Mosaic says so if you try.
#Params
Query-string parameters, each with a key, a value, a description and an on switch.
Where a Check already defines parameters that this source does not use, they are offered above the table as chips — "Add storeId as a query parameter of the same name", with Add all beside them.
A parameter holding several values needs to know how to spell them:
| Option | Produces |
|---|---|
| a,b — one comma-joined value | k=a,b |
| k=a&k=b — the key repeated | k=a&k=b |
| k[]=a&k[]=b — bracketed | k[]=a&k[]=b |
| ["a","b"] — a JSON array | k=["a","b"] |
#Auth
| Type | Fields |
|---|---|
| No auth | — |
| Basic | Username and password |
| Bearer token | Token |
| API key | Key, value, and whether it goes in a header or the query string |
| OAuth 2.0 client credentials | See below |
Switching type resets to that type's own defaults rather than merging, so you cannot leave a stale secret behind in a field you can no longer see.
Basic has a paste box worth knowing about. Paste an Authorization: Basic … header from Postman and press Decode, and the two fields fill in: "Nothing is sent anywhere — it is decoded here."
#OAuth 2.0 client credentials
Three collapsed sections:
Token request — token URL, client ID, client secret, scope, whether credentials go in the header or the body, and the grant type (client_credentials by default). The same Basic-paste box is here too.
Extra token body fields — additional form fields beyond grant_type and scope.
Extra token headers — "Oracle IDCS expects Accept-Charset: UTF-8."
Tokens are cached in memory only, keyed by the shape of the token request. The result pane labels each run token or token cached so you can see which happened.
#Headers and Body
Headers is a plain key/value list.
Body offers None, JSON, Raw (with a content type, text/plain by default), Form data, or x-www-form-urlencoded.
#Extract
This is the tab that decides what a row is.
| Field | Default | Meaning |
|---|---|---|
| Rows path | Empty | "Dot path to the array in the response, e.g. data.rows. Leave empty if the response is itself an array." |
| Flatten depth | 3 | Nested objects become dot-notation columns up to this depth |
| Keep arrays as JSON text | On | Off drops arrays entirely |
| One row per item of | Empty | A list inside each row, e.g. lineItems.nodes. Each item becomes a row of its own, with the row's other fields repeated beside it |
The path is dot-separated and supports indexing — items[0].rows works.
What happens when the path does not find an array:
| It resolves to | Result |
|---|---|
| Nothing | 0 rows, with a warning that the path matched nothing |
| An array of objects | One row each, flattened |
| An array of scalars | One row each, as a value column |
| A single object | One row, with a warning saying so |
| Any other scalar | One row, as a value column |
#Paging
Under Extract, and the help explains why it matters:
Some services answer with the first page and a way to ask for the rest. Reading only the first is how rows that exist are reported missing.
That is worth taking seriously. A paged API read one page deep produces a Check that reports thousands of rows as missing, and every one of them is a lie.
One request is the default.
Ask for a window at a time — you name the how-many parameter (limit), the how-far-in parameter (offset), rows per page (500), and optionally a field saying there is more. Left blank, a full page is taken as there may be more and a short one ends it.
Follow the link to the next page — you name where the next address is. A string there is the address; a list of links is searched for the one marked next.
Carry a cursor to the next page (GraphQL) — you name where the cursor is (data.orders.pageInfo.endCursor), optionally the flag saying there is another page (…pageInfo.hasNextPage), and what it is sent back as (after). With a GraphQL body the cursor is set as that variable; otherwise it goes in the query string under that name. When a service answers throttled — a 429, or GraphQL's THROTTLED — Mosaic waits as long as the service says and asks again, rather than failing.
All three modes have a Stop after guard, 500 pages by default, "a guard against a service that always says there is more". Hitting it warns rather than failing.
#Settings
| Field | Default |
|---|---|
| Timeout (ms) | 30000 |
| Max rows | 50000 |
| Proxy URL | — |
| Follow redirects | On |
| Verify TLS certificate | On |
A response that is not JSON fails with "Response body is not valid JSON, so it cannot be projected into columns." An HTTP status of 400 or above fails with the code and text.