aboutsummaryrefslogtreecommitdiff
path: root/memos/WM-101.txt
blob: 62a89f0a3e388c1b5df45ce531d692933fb6b85d (plain) (blame)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
Document: WM-101                                                 P. Webb
Category: Tutorial                                            2026-09-02

             How to migrate from Gel to Disc, in Production

Abstract

   Unforeseen gotchas galore, but I figured it out.

Body

   The last post/guide about this[1] only handled local development and
   as we know, what "works on my machine" often doesn’t translate well
   to other machines.

   In order to perform the backup of my Gel database on my server, I
   first needed to create the `gel-export` folder in the directory of
   the app Gel was running in before giving ownership and permissions to
   the `gel` user.

   ```
   # run these commands as root user
   chown gel: gel-export
   chmod u+w gel-export
   su gel
   ```

   `su gel` switches you from the `root` user to the `gel` user.

   Replace `INSTANCE` with the name of your Gel instance to get your
   user and password (if you don’t already know them).

   ```
   # run this command as gel user
   gel instance credentials --insecure-dsn -I INSTANCE

   # you should see something like
   gel:///DATABASE?port=PORT&user=USER&password=PASSWORD&tls_ca_file=%3C...%3E&tls_security=no_host_verification
   ```

   ```
   # run this command as gel user
   # you will be prompted for that password you saw earlier
   gel dump --all --format=dir gel-export --admin --user USER --instance INSTANCE --tls-security insecure --password
   ```

   I ran into errors complaining about missing packages so I exited back
   to my `root` user and ran these:

   ```
   apt install -y postgresql-common
   /usr/share/postgresql-common/pgdg/apt.postgresql.org.sh
   apt install -y postgresql-client-18
   ```

   I was able to proceed with the rest of the export process after that.

   When deploying my new Disc-aware program though, I found I needed two
   `systemd` files. The server install script[2] for Disc is designed to
   "just work" and will create a service file for you and have Disc
   running automatically but there’s another running process that’s
   required, and that’s after creating a new user (as `root`) and giving
   said user a password. I named my new user `disco` because `disc` was
   already taken by the install script and you cannot switch to said
   user, for security reasons (no login allowed).

   Here is my service file for the Nickel API:

   ```
   [Unit]
   Description=NICKEL API
   Documentation=https://nickel.video
   After=network.target

   [Service]
   ExecStartPre=/opt/disc/bin/disc start
   ExecStartPre=/opt/disc/bin/disc migrate
   ExecStartPre=/opt/disc/bin/disc codegen
   ExecStart=/var/www/api/nickel
   Group=disco
   Restart=on-failure
   User=disco
   WorkingDirectory=/var/www/api

   [Install]
   WantedBy=multi-user.target
   ```

   The Nickel API is a compiled executable. Before that starts running,
   I have Disc boot up, run migrations if necessary, and codegen for the
   compiled SDK. It’s quite possible this can be optimized.

   Oh, I had to ensure the `disco` user had ownership over the codegen
   directory (`dbschema/`) so migrations wouldn’t fail:

   ```
   chown -R disco:disco /var/www/api/dbschema
   ```

   Here is the other service file I had to make to serve the Disc UI and
   database for the API to reach. I should probably rename it to
   `api-serve` so it’s not confusing to future me.

   ```
   [Unit]
   Description=Disc Start Server
   After=network.target postgresql.service api.nickel.service
   Wants=disc.service api.nickel.service

   [Service]
   ExecStart=/opt/disc/bin/disc serve
   Group=disco
   Restart=on-failure
   RestartSec=5
   SyslogIdentifier=disc-start
   User=disco
   WorkingDirectory=/var/www/api

   [Install]
   WantedBy=multi-user.target
   ```

   Once confiming the service files worked, I had to finish the data
   import process.

   ```
   su disco

   disc migrate
   disc db import ./gel-export/data
   disc db import ./gel-export/data --on-conflict skip
   ```

   Computed fields in my Gel export gave me issues. I made backup copies
   of the affected files, deleted the columns causing problems, and the
   import went through! Fully prepared to manually enter the data of the
   deleted columns, I accessed the Disc UI and was surprised to see my
   computed columns present! Which, makes sense in hindsight…the other
   data in the backup would obviously compute these fields on import.

   I’ll add a fox for that in Disc.

   ---

   This was quite a painful process TBQH. I was obviously motiviated to
   push through it but good grief! The "Happy Path" and DX needs tuning.

   The Disc UI doesn’t come with authentication so that you can design
   your own (or just disable it in production). Here’s how I handle that
   with Caddy:

   ```
   # generate a password in your favorite password manager and
   # have Caddy hash it
   caddy hash-password
   ```

   Make note of that generated hash! Then, update `basic_auth` in your
   Caddyfile (and make sure `PORT` matches whatever your Disc UI wants:

   ```
   database.nickel.video {
     basic_auth {
       SOME_USERNAME <SOME_HASH>
     }

     encode gzip
     reverse_proxy localhost:PORT
     root * /path/to/your/project
   }
   ```

   Last step is to configure your firewall!

   ```
   ufw allow https
   ufw allow ssh
   ufw deny PORT
   ufw enable
   ```

   This makes your server reachable via HTTPS (no plain HTTP), allows
   you to continue connecting to your server via SSH, rejects any
   outside connections to your Disc UI’s port, and enables the firewall.

   I feel like this "tutorial" was all over the place and I certainly
   felt that way while I was trying to keep track of all the pieces,
   haha! At some point I had to uninstall Gel because I think it was
   interfering with Disc.

   Please excuse the mess, there’s more work to be done to make Disc the
   best, smoothest database to ever exist. 🕸️

References

   [1] </WM-097>
   [2] <https://disc.sh/install-server.sh>