Push data to Ronja
Pushing means your own code sends data to Ronja on its own schedule: a nightly export, a script that calls a supplier’s API, an app that records events. This page is about which path to pick and how to keep each run fast, safe to repeat and light on rebuilds. Each request is in Send data via the API.
Pick the shortest path first
Section titled “Pick the shortest path first”| Where the data is | What to use | Why |
|---|---|---|
| In your own database: PostgreSQL, MySQL, SQL Server or BigQuery | A connection (Admin), not push code: store the credentials as a secret and create a connection that references it. See Database connections; the full recipe is at /docs/api/guides/import-a-database.md on your Ronja domain |
Ronja syncs the tables you choose on your cadence into a Feature, as Integration tables, and keeps track of how far she got. There is no script for you to keep running |
| In a file you have once | Upload files | No code at all |
| Rows your own code produces: API results, readings, a summary you compute | A Dynamic table, row push | The write is synchronous: when it returns, the rows are queryable. No build step |
| Parquet files you already have: a warehouse export, a nightly dump | An Integration table, file push, then Build | Files accumulate by filename and are built together |
| Already in Ronja, and you want to reshape it | A Derived table, not a push. See Table types | Ronja runs the SQL and rebuilds it when its inputs change |
Send few, large writes
Section titled “Send few, large writes”Every write that changes the table rebuilds the Derived tables built from it and fires every Automation watching it, and an append adds a data file. An append with no rows changes nothing and triggers nothing. So the cheapest run is the one with the fewest writes:
- One write per table per run. Collect the run’s rows, then send them in one request, not one request per record. A script that writes every few seconds rebuilds everything downstream every few seconds and fires its Automations just as often.
- Send Parquet, not JSON, once the data is large. A JSON write takes at most 50,000 rows or 8 MiB. A Parquet upload to the same table takes up to your organization’s upload limit (1 GB unless Ronja has changed it) with no row limit, and the same rows are a small fraction of the bytes. With pandas it is one line:
df.to_parquet(buf). - On JSON, send values with their real types. Numbers as numbers (
"qty": 3, not"qty": "3") and trimmed strings. A quoted number on a column of numbers is a type change (on a table with the Legacy typing marker, see Switch a table to typed rows). See When a column changes type.
Make every run safe to repeat
Section titled “Make every run safe to repeat”A run that fails halfway gets run again, by you or by a scheduler, so make a repeat harmless:
- Refresh with
replace. If each run has the whole current dataset, send it with"writeMode": "replace". Sending it twice leaves the same table. Keepappendfor records that are genuinely new, and know that anappendthat timed out may still have landed, so retrying it can write the batch twice. - Name pushed files after what they hold. On an Integration table, re-uploading a filename replaces that file. A file named by its date (
orders_2026-10-03.parquet) is overwritten by a retry rather than added a second time. - Retry the answers that say to. A 409 on a write means the table was busy for more than 30 seconds, with other writes or with Ronja tidying its data files, or that the server was busy with other conflicting writes: nothing was written, so wait a moment and send it again. A 409 with
"code": "version_mismatch"is different: your guarded write found the table changed, so read it again before you resend. A 503 on the release is safe to retry too. A server error (500, 502, 504) or a dropped connection is safe to resend only for a call that is safe to repeat: areplacewrite or the release, not anappend. - Change a few rows with SQL, and give it a key. To update or delete some rows, send the statements to
/modifywith anIdempotency-Keyheader: a retry with the same key within 24 hours changes nothing a second time. - Store the table ID. Keep the
table-…ID from when you created the table rather than looking the table up by name on every run.
Write several tables as one change
Section titled “Write several tables as one change”When one run writes several tables that a Derived table reads together, hold back each table’s rebuild with ?cascade=defer and release them in one call, so the report rebuilds once over all the new data; the steps and rules are in Send several tables at once.
A complete nightly push
Section titled “A complete nightly push”This script refreshes two Dynamic tables as one change. It sends each as Parquet with replace, so every call in it is safe to repeat, retries the busy and server-error answers, and releases both tables after the last write. Give the client timeout room for your largest write, and more than the 30 seconds a write waits for a busy table before its 409: a write the client gives up on before it finishes lands nothing, and one that finished stays landed, so the safe retries below are what make a timeout harmless.
import io, os, timeimport pandas as pdimport requests
BASE = os.environ["RONJA_BASE_URL"]S = requests.Session()S.headers["Authorization"] = f"Bearer {os.environ['RONJA_API_TOKEN']}"
TABLES = {"orders": "table-a1b2c3d4e5", "customers": "table-f6g7h8i9j0"}RETRY = {409, 500, 502, 503, 504}
def call(method, path, **kw): # Every call below is safe to repeat, so a failure that may be passing is retried. for attempt in range(6): if attempt: time.sleep(2 ** attempt) # 2, 4, 8, 16, 32 seconds try: r = S.request(method, BASE + path, timeout=900, **kw) except (requests.ConnectionError, requests.Timeout): continue if r.status_code not in RETRY: if not r.ok: # any other error is a real refusal: stop, with Ronja's reason raise RuntimeError(f"{method} {path}: {r.status_code} {r.text}") return r.json() raise RuntimeError(f"{method} {path} kept failing")
def push(name, df): buf = io.BytesIO() df.to_parquet(buf, index=False) out = call("POST", f"/api/v2/feature/model/{TABLES[name]}/rows/parquet", params={"writeMode": "replace", "cascade": "defer"}, headers={"Content-Type": "application/octet-stream"}, data=buf.getvalue()) print(name, out.get("cascade")) # "deferred": held for the release below
push("orders", load_orders()) # your own extractspush("customers", load_customers())
print(call("POST", "/api/v2/feature/model/cascade", json={"tableIds": list(TABLES.values())}))# -> {"released": ["table-a1b2c3d4e5", "table-f6g7h8i9j0"], "notPending": []}If push("customers", …) keeps failing, the script stops before the release. The orders write stays held until the automatic release, so rerun the script once the failure is fixed.
For Integration tables the loop is the same with files: upload each table’s files, start every Build with ?cascade=defer, wait for every build to finish, then release.