diff options
Diffstat (limited to '')
| -rw-r--r-- | memos/WM-097.txt | 54 |
1 files changed, 40 insertions, 14 deletions
diff --git a/memos/WM-097.txt b/memos/WM-097.txt index 53c1df6..9e35268 100644 --- a/memos/WM-097.txt +++ b/memos/WM-097.txt @@ -13,7 +13,8 @@ Category: Tutorial 2026-06-22 Abstract The hardest part is getting your data out of Gel in a useable format, - the rest is easy. + the rest is easy (but mind your links on the way in, warning near + the end). Body @@ -21,6 +22,8 @@ Body requires arcane knowledge but don't fret, I've uncovered the incantations to free your data! + Perform this full backup, for peace of mind. + ```sh gel dump --all --format=dir gel-export ``` @@ -47,7 +50,10 @@ Body gel:///DATABASE?port=PORT&password=PASSWORD&tls_ca_file=%3C...%3E&tls_security=no_host_verification ``` - Now we can set some variables to make the last bits pseudo-legible. + Now we can set some variables to make the last bits pseudo-legible. + `sslmode=require` because Gel refuses plaintext connections, and + without it `psql` fails with a misleading auth error before the real + "TLS required" one. ```sh DSN="postgresql://edgedb@localhost:PORT/DATABASE?sslmode=require" @@ -57,16 +63,9 @@ Body export PGPASSWORD="PASSWORD" ``` - Perform a raw SQL dump to `gel-export/data.sql`: - - ```sh - pg_dump \ - "$DSN" --data-only --no-owner --no-privileges \ - > gel-export/data.sql - ``` - - We create a `.tsv` file first, without any escaping that could mangle - the data. + Gel's binary dump is useless to anything but Gel, so to get tabular + data out we go through its SQL adapter. List the tables into a `.tsv` + first, without any escaping that could mangle the identifiers: ```sh psql "$DSN" -At -F $'\t' -c " @@ -91,6 +90,22 @@ Body done < gel-export/tables.tsv ``` + --- + + A quick note on what's in those CSVs, because it bites at import + time (and it bit me)! + + Gel maps each object type to a table with `id` and `__type__` + columns, single properties as plain columns, and (what got me) single + LINKS as `<link>_id` UUID columns (your `owner` link becomes an + `owner_id` column holding the target's id). + + Multi links and multi properties aren't in these tables at all; they + live in their own `source`/`target` tables and import as + separate files. + + --- + I'll assume you installed Disc[1] already. Great! You'll need to get it setup with your schema. Run that migration! @@ -116,8 +131,19 @@ Body disc db import ./gel-export/data --on-conflict skip ``` - You'll get an import manifest of what was processed and skipped. For - me, it was abstract types and (empty) computed links. + --- + + You'll get an import manifest of what was processed and skipped. + Remember those `<link>_id` columns? Disc resolves them into your + single links, but anything it can't map surfaces here as a warning. + Confirm your links got set and that no non-empty column was + quietly discarded. + + A previous version of Disc silently dropped every `<link>_id` column + and because I'm dogfooding this, I found the issue and fixed. This is + bleeding edge folks! + + --- Run `disc serve` and navigate to your database's `/ui` and you should see populated objects in the Data tab. |