Choosing Records
A custom API source reads records from one list endpoint, page by page, and keeps the ones you choose. The list settings are on the Connection step; the conditions and the document limit are on the Records step.
The list response
Vectord needs to know where the records are in each list response. Enter that in Response Array Path as a dot path:
{
"data": [
{
"id": "T-1001",
"subject": "Printer offline",
"status": "open",
"priority": 2,
"updated_at": "2026-10-30T14:05:00Z"
},
{
"id": "T-1002",
"subject": "Reset my password",
"status": "closed",
"priority": 3,
"updated_at": "2026-10-29T09:12:00Z"
}
],
"next_cursor": "c2FtcGxlLTI",
"has_more": true
}
Here Response Array Path is data. Leave it empty when the response is the array itself ([ {...}, {...} ]).
Dot paths name a field by its keys, separated by dots: data, info.next_page_token, customer.plan. They can't contain array indexes (data[0]), wildcards or spaces, and have at most 10 parts. The same paths are used for pagination fields, conditions and mapping.
Pagination
Choose the style your API documents. Every style has the same safety rules:
- A page that doesn't contain the records array at the configured path stops the sync with a pagination error.
- A next position the API returned recently (the same cursor, link, page or offset as an earlier page) stops it too, so a broken API can't loop forever.
- A has-more or end flag must be
trueorfalse. A flag that contradicts the rest of the response (it says more pages exist but there's no next cursor, say) is an error, never the end of the list.
Cursor
The response carries a token for the next page; the next request sends it back in a query parameter.
| Field | Example value | Meaning |
|---|---|---|
| Cursor Parameter | cursor | The query parameter that carries the token |
| Next Cursor Path | next_cursor | Where the next token is in the response (text or a number) |
| Has-More Flag Path (optional) | has_more | A true/false field that says whether more pages exist |
With the response above, the second request is GET /v1/tickets?cursor=c2FtcGxlLTI. The list ends when the next token is missing, null or empty. With a has-more flag, true without a token, or false with one, is an error.
{ "data": [ ... ], "info": { "more_records": true, "next_page_token": "eyJwYWdlIjoyfQ" } }
Here Cursor Parameter is page_token, Next Cursor Path is info.next_page_token and Has-More Flag Path is info.more_records.
Next-page link
The response carries the address of the next page.
| Field | Example value | Meaning |
|---|---|---|
| Next Link Path | links.next | Where the next page's address is in the response |
| End Flag Path (optional) | done | A true/false field about the end of the list |
| When the end flag is true | More pages exist or The list is done | What true means at the end flag path |
{
"items": [ ... ],
"links": { "next": "/v1/articles?after=A-2040" }
}
The link may be absolute (https://api.example.com/v1/articles?after=A-2040) or relative to the API address (/v1/articles?after=A-2040); either way it must stay on the API address. The link is followed exactly as given: fixed parameters and the page size are sent only on the first request. The list ends when the link is missing, null or empty.
{ "records": [ ... ], "done": false, "nextRecordsUrl": "/services/data/v62.0/query/01g-2000" }
Here Next Link Path is nextRecordsUrl, End Flag Path is done, and When the end flag is true is The list is done.
Page number
The next request asks for the next page number.
| Field | Example value | Meaning |
|---|---|---|
| Page Parameter | page | The query parameter that carries the page number |
| First Page | 1 or 0 | The number of the first page |
| Page Size Parameter | per_page | Required for this style |
| Has-More Flag Path (optional) | meta.has_next | A true/false field that says whether more pages exist |
Requests are GET /v1/tickets?per_page=100&page=1, then page=2, and so on. Without a flag, the list ends at the first empty page. With a flag, it ends when the flag is false; true with an empty page is an error.
Offset
The next request skips the records already read.
| Field | Example value | Meaning |
|---|---|---|
| Offset Parameter | offset | The query parameter that carries the offset |
| Page Size Parameter | limit | Required for this style |
| Has-More Flag Path (optional) | has_more | A true/false field that says whether more pages exist |
Requests are GET /v1/tickets?limit=100&offset=0, then offset=100 (the offset grows by the number of records the page returned). The end rules are the same as for page numbers.
Page size and fixed parameters
- Page Size is the number of records per request, from 1 to 200 (default 100). It is sent in Page Size Parameter when you name one; page number and offset styles need one.
- Advanced: fixed query parameters are added to every list request the app builds (up to 20); a next-page link is followed as given, without them. Use them for things your API requires, like
fields=Last_Name,Company,Descriptionorstate=all. Each name may appear once, and none may repeat the pagination or page-size parameter.
The page-size and pagination parameters must differ.
Which records
On the Records step, choose:
- All records to keep every record the endpoint returns; or
- Matching records to keep only records that meet every condition you add (up to five).
There's no default: you must choose one. Conditions run on our server after the API returns each page. They don't change the request, so a filter your API supports (like status=open as a fixed parameter) saves requests on large lists.
Each condition has a Field (a dot path, suggested from the sample), a type, an Operator and, for most operators, a value:
| Type | Operators | Notes |
|---|---|---|
| text | Equals, Does not equal, Contains, Starts with | Case matters; values up to 500 characters |
| number | Equals, Greater than, Less than | |
| date | On or after, Before | ISO 8601 only: 2026-01-31 (midnight UTC) or 2026-01-31T09:00:00Z. Offsets such as +02:00 or +0000 are understood |
| boolean | Is true, Is false | |
| any type | Is missing, Is null |
A value of the wrong type (text where a number was expected, or a date that isn't ISO 8601) never matches. A missing or null value matches only Is missing or Is null.
Example: to keep open tickets with priority 1 or 2, add status text Equals open and priority number Less than 3.
Preview Matches
Preview Matches reads the first page again and counts how many of up to 50 records match. It's a sample, not a total: Total matching shows Unknown because only a full pass through the list can count it. You need a preview before Continue to Mapping.
The document limit and the per-step batch
Maximum Documents in This Project is the most documents this source may keep, from 1 to 100,000. It's a lifetime total, not a monthly or per-sync allowance:
- it isn't reset over time;
- your plan's document allowance applies as well, whichever is reached first;
- it can't be set below the documents the source already tracks.
It's different from the batch: each sync step reads up to 10 pages (within about 40 seconds) and adds at most 10,000 new documents. A large API is imported over several steps; see After the first Start.
When the source reaches its limit, matching records beyond it are held back and counted on each pass; the source shows Limit reached, and its Monitor dialog shows how many were deferred. Documents it already tracks keep refreshing. Raise the limit (or upgrade) and approve the review, and the next pass adds them. See Limit reached.
Next: Mapping fields.