Feeds on a schedule
Let OrbSearch fetch your shop's export from its URL on a schedule, so the catalog stays current on its own.
Most shops already serve their export at a URL, such as the feed they give Google Merchant Center. Set that URL as the catalog source, and the control plane fetches it on a schedule. A changed feed goes live as an import would; an unchanged one changes nothing.
Set the feed
The source is one URL, how often to fetch it, and what a changed feed does.
On Catalog → Imports, the Catalog source sits next to the file drop. Add a feed URL opens its settings:
- Feed URL, the address your shop serves its export at;
- Sign-in: None, User and password or Token in a header;
- Fetch, from Every 5 minutes to Once a day;
- When the feed changed: Publish at once, or Keep as a draft to review.
Save sets the source.
Set the URL and how often to fetch it, in minutes:
curl -s -X PUT -H "Authorization: Bearer $key" -H 'Content-Type: application/json' $api/catalog-source \
-d '{"url": "https://shop.example/feeds/export.xml", "interval_minutes": 60}'interval_minutes takes 5 to 1440. "on_change": "draft" keeps a changed feed as a draft instead of publishing it. A feed behind a sign-in adds authentication, a token in a header or a user and password ("kind": "basic"):
{ "authentication": { "kind": "header", "name": "X-Api-Key", "value": "…" } }A new URL or a new sign-in is fetched at once, then once per interval. The feed can be any format the import reads. A password or token is kept encrypted, sent only to the feed's own host, and never shown again.
Reference: PUT /api/v1/catalog-source · authentication · on_change
What a fetch does
Each fetch asks the server whether the feed changed since the last one, with If-None-Match and If-Modified-Since, and ends one of five ways:
- Unchanged (
unchanged): the server says so, or the bytes are the ones fetched before. No index run is queued. - Published (
published): the feed became an import and goes live. Its index run has thetriggerfetch, shown as Scheduled fetch on Releases. - Kept as draft (
drafted): the feed waits as a draft to review, since the source keeps changes as drafts. - Held back (
held_back): changes publish at once, but the feed would remove half the live products or more, so it waits as a draft until you confirm it. - Failed (
failed): see below.
A newer feed replaces the draft an earlier fetch left, unless you changed that draft's files. The run's report says what went live.
Reference: CatalogFetch · result
When a fetch fails
What was live stays live. The fetch says why in failure, with a sentence for the merchant: a host that cannot be reached, a refused sign-in, an error status, or a file that is empty, larger than 2 GB or unreadable, among others.
A held-back feed, or three failed fetches in a row, put the source on Overview under Needs your attention.
Reference: failure · needs_attention
Private networks
A feed on a private, loopback or link-local address, such as one on the shop's own network, is refused as address_refused, after every redirect too. A self-hosted stack whose feed lives on its own network allows it in .env:
ORBSEARCH_FEEDS_FROM_PRIVATE_NETWORKS=trueReference: ORBSEARCH_FEEDS_FROM_PRIVATE_NETWORKS
Fetch now, look first, stop
Between scheduled fetches, you can fetch at once, see a feed before it goes live, or stop fetching.
Fetch now on the source's card fetches within seconds, and the card follows it until it is through. Below it, the last five fetches show their result, size and entries, and a feed that waits links to Review the draft.
To see what a feed becomes before you schedule it, Import from a URL in the file drop fetches it once and opens it as a draft. The pencil beside Fetch now opens the settings again, where Remove stops fetching.
Ask for a fetch, read how the latest one went, or stop fetching:
curl -s -X POST -H "Authorization: Bearer $key" $api/catalog-source/fetch >/dev/null
curl -s -H "Authorization: Bearer $key" $api/catalog-source | jq -c '.fetches[0] | {result, failure, message}'
curl -s -X DELETE -H "Authorization: Bearer $key" $api/catalog-source >/dev/nullGET answers the source with next_fetch_at and its last ten fetches, and 404 while none is set.
Stopping keeps every import the fetches made.
Reference: POST .../fetch · GET /api/v1/catalog-source · DELETE /api/v1/catalog-source
Next
- Importing and mapping: correct how the feed's columns are read before it goes live.
- The import report: what each index run took in and what it left out.
- Live prices and stock: change prices and stock between two feeds.