sqldb/v2 as separate package
#10175
sqldb/v2 as separate package
#10175
Conversation
ViktorTigerstrom
changed the title
sqldb/v2 as separate package
ViktorTigerstrom
force-pushed
the
2025-08-sqldb-v2-separate-package
branch
3 times, most recently
from
6b5e428 to
6899a39
Compare
| replace github.com/golang-migrate/migrate/v4 => github.com/lightninglabs/migrate/v4 v4.18.2-9023d66a-fork-pr-2 | ||
|
|
There was a problem hiding this comment.
ok afaict, the big thing we added in the fork was the post migration call-backs logic.
Something id just like some clarity on before we push forward here is:
in our own rolled migration strategy in LND today that takes care of code migrations, we know that the migration version table will only be updated once the code migration is complete. Is the same true for the post-migration callback?
There was a problem hiding this comment.
I discussed this with guggero previously.
Currently, no that is not true unfortunately, as the version table will be updated if the SQL migration succeeds, while the code migration fails. But I'm planning to address that asap 🔥!
The current plan is to submit a new PR to our migrate library fork. This PR will add a separate tracking table for code migrations. With this change, the logic will ensure that if the db is initialized and the persisted code migration version is lower than the SQL migration version, the code migration will be re-run prior to any SQL migration.
I haven't gotten to updating the migrate fork yet though. Perhaps we should hold of with merging this one until that change has been merged to the migrate fork, so that we don't need to push a new PR that updates the dependency in lnd though.
Does that sound like a good plan to you @ellemouton?
ViktorTigerstrom
force-pushed
the
2025-08-sqldb-v2-separate-package
branch
from
6899a39 to
7ca3000
Compare
In the upcoming commits, we will introduce a new sqldb module, sqldb version 2. The intention of the new sqldb module, is to make it generalizable so that it contains no `lnd` specific code, to ensure that it can be reused in other projects. This commit adds the base of the new module, but does not include any implementation yet, as that will be done in the upcoming commits.
This commit moves all non lnd-specific code of sqldb/v1 to the new sqldb/v2 module. Note however, that without additional changes, this package still needs to reference lnd, as references to the lnd `sqlc` package is required without further changes. Those changes will be introduced in the upcoming commits, to fully decouple the new sqldb/v2 module from lnd.
This commit introduces a new struct named `MigrationStream`, which defines a structure for migrations SQL migrations. The `MigrationStream` struct contains the SQL migrations which will be applied, as well as corresponding post-migration code migrations which will be executed afterwards. The struct also contains fields which define how the execution of the migrations are tracked. Importantly, it is also possible to define multiple different `MigrationStream`s which are executed, to for example define one `prod` and one `dev` migration stream.
This commit updates the `sqldb/v2` package to utilize the new `MigrationStream` type for executing migrations, instead of passing `[]MigrationConfig`'s directly.
This commit updates the definition of the `BaseDB` struct to decouple it from lnd`s `sqlc` package. We also introduce new fields to the struct to make it possible to track the database type used at runtime.
Drop the GetSchemaVersion and SetSchemaVersion methods for the PostgresStore and SqliteStore structs in the sqldb/v2 package.
In order to make it possible to replace `tapd`'s internal `sqldb` package with the new generic `sqldb/v2` package, we need to make sure that all features and functionality that currently exist in the `tapd` package are also present in the new `sqldb/v2` package. This commit adds such additional missing features to the `sqldb/v2` package.
Previously, the test db helper files were suffixed with "_test", which would indicate that the files specifically contained tests. However, these files actually contain helper functions to be used in tests, and are not tests themselves. To better reflect their purpose, the files have been renamed to instead be prefixed with "test_".
ViktorTigerstrom
force-pushed
the
2025-08-sqldb-v2-separate-package
branch
from
7ca3000 to
233f1fb
Compare
|
Thanks a lot for the review @ellemouton 🔥🎉! Did one separate push to address the latest feedback (and add the contents of #10193) + one push to rebase this PR on master. I'm not re-requesting a review of you just yet until we have decided on #10175 (comment) @ellemouton. |
|
Will address failing CI checks on the next push after the next review round. |
|
Based the PR on the now updated |
|
@ziggie1984: review reminder |
This PR addresses Phase 1 of the release plan of comment #9762 (comment) in #9762.
This PR extracts the non-specific
lndspecific logic of thesqldbpackage into a newsqldb/v2package, and introduces the extra functionality thatsqldb/v2adds.