feat: adapt migration schema calls to query-lib value objects by abnegate · Pull Request #222 · utopia-php/migration

abnegate · 2026-08-21T04:07:12Z

Adapts the migration destination's schema calls to the query-lib value objects.

Why this approach

Database::createCollection() takes a Collection only, so destination Appwrite now wraps the collection id, attributes and indexes — along with the named permissions and documentSecurity — in new Collection(...) instead of passing positional arrays.

utopia-php/database is pinned to dev-feat-query-lib as 2.0.0, re-pinned to that branch's head whenever it moves.

Lock repairs that came with the re-pin

Two problems surfaced only when the lock was resolved again rather than reused:

utopia-php/query was locked to dev-feat-schema-order, a branch that no longer exists on the remote. The resolution survived only as long as nobody resolved it. database's branch requires query 0.6.*, which is released, so the lock now takes the release.
utopia-php/storage moved 4.0.3 → 4.0.4, because 4.0.3 capped utopia-php/validators at ^0.4 while database requires ^0.5. 4.0.4 dropped the validators dependency outright. database has required ^0.5 on main as well as on the branch, so this was already true before the query-lib work and only surfaced now.

A live bug the re-pin exposed

The Appwrite destination passed the 'ASC' / 'DESC' strings a source hands back straight into Utopia\Database\Index, which takes Order cases and rejects anything else. The resulting InvalidArgumentException is not a Migration Exception, so instead of recording a failed index the whole transfer aborted.

This was already true before this PR and simply could not be seen: the lock held query at the deleted branch, whose Index took plain strings, so CI had never built an index against the contract the released library actually has. AppwriteIndexLengthsTest covers it — two of its cases pass ['ASC', 'ASC'], and both went red on the first run against the re-pinned lock and green with the fix.

It is the same defect as the one in appwrite/appwrite#11649's Databases worker, from the same cause.

Chain

Landing order, bottom up:

utopia-php/database#823 — the query-lib migration itself
utopia-php/abuse#124, utopia-php/audit#133, utopia-php/migration#222 — the schema call sites in the libraries
appwrite/appwrite#11649
appwrite-labs/cloud#5410

Stacked on #823 but not part of it, and not required by anything above: #947 (ORM), #948 (repositories and seeding), #949 (migration runner and schema differ).

Every dev-feat-query-lib pin in this train is re-pinned to its branch head whenever one of them moves, so each PR's CI runs against what the others actually contain.

Verified

CI green on this head
Greptile 5/5, no unresolved threads
Pint on the Collection wrap

Not verified

The utopia-php/database dependency is still a branch pin. It becomes a released tag only once #823 merges, and this PR should not land before that.

Published 2.0 still used Database::VAR_* and positional createAttribute/createIndex, which feat-query-lib removed. Rebase onto main and pass Attribute/Index/Relationship VOs plus ColumnType/IndexType so Appwrite can pin this branch as 2.0.0.

Appwrite stores ColumnType::BigInteger as biginteger. CSV export resolved that as an unsupported column type and wrote no rows.

createDocument can return an empty Mongo sequence while a subsequent getDocument has the ObjectId. Creating database_{seq} from the create return left table import looking up a collection that did not exist.

Appwrite main added huggingface as a project OAuth2 provider. Without an allow-list entry, Appwrite-to-Appwrite migrations fail on that provider even when the rest of the transfer succeeded.

Keep query-lib APIs and the huggingface PROVIDERS allow-list already on main.

Appwrite #11649 locks database at 5719edd. Staying on e593b78 would only prove the schema VO calls against an older query-lib surface.

greptile-apps · 2026-08-21T04:09:24Z

Confidence Score: 5/5

The implementation appears safe to merge, although the non-blocking implementation-coupled CLI help test remains unresolved.

Summary

Wraps Appwrite collection schema data in query-lib/database value objects.
Converts schema column, index, and ordering values to query-lib enums.
Adds migration-attempt ownership and fail-closed database recovery behavior.
Updates standalone CLI lifecycle handling, documentation, static analysis coverage, and regression tests.
Repairs dependency resolution around database, query, storage, and related packages.

utopia-php/database feat-query-lib keys silenced events with Coroutine::getCid(). The CI image is vanilla PHP, so Memory-adapter tests fatalled before they could run.

createDocument can persist a row whose subsequent getDocument is empty. That throw sat outside the failed-status handler, so a later skip could flip the unusable database to ready without a backing collection.

abnegate · 2026-08-21T04:16:19Z

Addressed the reload-failure finding.

createDocument can persist a _databases row whose immediately following getDocument is empty (the Mongo sequence miss this branch already reloads for). That throw sat outside the markDatabaseFailed catch, so the document stayed provisioning and a later spec-matching skip could flip it to ready with no backing collection.

Reload + createCollection now share that catch. testReloadFailureMarksTheDatabaseFailed fails without the wrap (provisioning) and passes with it (failed).

Also stubbed Swoole\Coroutine::getCid() in the PHPUnit bootstrap so the Memory-adapter suite can run on the CI image, which has no Swoole extension. utopia-php/database feat-query-lib keys silenced events with it.

@greptile-apps review

Asterisk wildcards on utopia-php packages are replaced with equivalent caret constraints so Composer ranges stay consistent.

Keep composer.json and composer.lock in sync so `composer validate` passes, and pin utopia-php/database to the current query-lib HEAD.

abnegate · 2026-08-21T05:24:49Z

@greptileai review

abnegate · 2026-08-21T06:00:14Z

@greptile-apps review

Force re-review of HEAD 7a3a60e. Description updated for factories and caret lock refresh.

Database::createCollection no longer accepts a string id.

Database::checkAttribute now requires Attribute. Build schema models from the resource key so metadata document IDs are not used as attribute keys.

Appwrite E2E migrations failed because checkAttribute now requires Attribute, and the destination still handed it a metadata Document.

A reload failure leaves a metadata document in `failed` with no backing collection. Recovery only ran when onDuplicate was not Fail, so the default policy retried createDocument against the existing ID and stranded the database.

abnegate · 2026-08-21T10:36:44Z

@greptileai review

Index types already used IndexType; column direction was still a raw ASC string. Collection constructors with multiple named params were also jammed on one line.

…atabase The lock held utopia-php/query at dev-feat-schema-order, a branch that no longer exists on the remote, so the resolution only survived as long as nobody resolved it again. database's branch requires query 0.6.*, which is released, so this takes the release. storage 4.0.4 comes along because 4.0.3 capped utopia-php/validators at ^0.4 while database requires ^0.5; 4.0.4 dropped the validators dependency outright. database has required ^0.5 on main as well as on the branch, so this was already true before the query-lib work and only surfaced now that the lock moved. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

abnegate · 2026-08-31T16:49:15Z

@greptileai review

Both store a plain client id and secret, so only the client id is readable and migratable. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> (cherry picked from commit ea342db)

The lock still pinned 6cc5f8be, two reconciles behind: the branch's green CI was proving a database revision that will not ship. a137475 carries the merge with main and the batched upsert read. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

abnegate · 2026-09-09T12:06:46Z

@greptileai review

The guard read Appwrite.php and pinned source fragments, so a reformat broke it while a real regression could still pass. Transfer a column through the public entry point instead and record what the project database is handed: the metadata row beside it is keyed by the composite attribute id, so a column named after that id is measured against a table that never holds it. Verified red on both regression shapes. Passing the metadata document fails the type the collaborator requires; rebuilding the attribute from that document's array copy reaches checkAttribute with key 1_1_title where the transfer's column is title. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

abnegate · 2026-09-09T12:27:15Z

@greptileai review

The guard reached resolveDestinationDsn by reflection, so it never ran the line deciding whether the creation path consults the resolver at all. Writing the source DSN straight into the row, which is the comuneo incident, left all three cases green. Transfer a database through the public entry point instead and read back _databases.database. Verified red on that shape: every case reports the source host database_db_fra1_self_hosted_11_0 where the destination must carry a blank or the resolver's own value. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

abnegate · 2026-09-09T12:36:56Z

@greptileai review

Five of the ten assertions pinned English sentences from the help text, which a reword breaks without changing behaviour. The other five are interface tokens a user types, and one of them carries a property nothing else covers: the neighbouring test proves --recover-provisioning is refused, not that it stays unadvertised. Re-adding it to getHelp() fails this test and no other. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

abnegate · 2026-09-09T12:45:24Z

@greptileai review

abnegate added 6 commits August 14, 2026 16:34

fix: accept query-lib biginteger as API bigint

d32d196

Appwrite stores ColumnType::BigInteger as biginteger. CSV export resolved that as an unsupported column type and wrote no rows.

fix: create collection metadata from the persisted database sequence

47e2597

createDocument can return an empty Mongo sequence while a subsequent getDocument has the ObjectId. Creating database_{seq} from the create return left table import looking up a collection that did not exist.

(fix): allow migrating HuggingFace OAuth2 providers

cce7c66

Appwrite main added huggingface as a project OAuth2 provider. Without an allow-list entry, Appwrite-to-Appwrite migrations fail on that provider even when the rest of the transfer succeeded.

chore: merge origin/main into feat-query-lib

340e6c6

Keep query-lib APIs and the huggingface PROVIDERS allow-list already on main.

chore: pin utopia-php/database to Appwrite's feat-query-lib SHA

0144e33

Appwrite #11649 locks database at 5719edd. Staying on e593b78 would only prove the schema VO calls against an older query-lib surface.

abnegate added 2 commits August 21, 2026 16:16

test: stub Swoole Coroutine when the extension is missing

a11db20

utopia-php/database feat-query-lib keys silenced events with Coroutine::getCid(). The CI image is vanilla PHP, so Memory-adapter tests fatalled before they could run.

fix: mark databases failed when metadata reload misses

55b92c7

createDocument can persist a row whose subsequent getDocument is empty. That throw sat outside the failed-status handler, so a later skip could flip the unusable database to ready without a backing collection.

abnegate added 2 commits August 21, 2026 17:11

(chore): use caret ranges for Utopia dependencies

b67ace1

Asterisk wildcards on utopia-php packages are replaced with equivalent caret constraints so Composer ranges stay consistent.

(chore): refresh lockfile after caret Utopia constraints

7a3a60e

Keep composer.json and composer.lock in sync so `composer validate` passes, and pin utopia-php/database to the current query-lib HEAD.

This was referenced Aug 21, 2026

Use query lib utopia-php/database#823

Open

feat: migrate to utopia-php/database query-lib appwrite/appwrite#11649

Open

abnegate added 3 commits August 21, 2026 18:52

(feat): pass Collection to createCollection

f2de906

Database::createCollection no longer accepts a string id.

(fix): pass Attribute models to checkAttribute

3fccb0c

Database::checkAttribute now requires Attribute. Build schema models from the resource key so metadata document IDs are not used as attribute keys.

(fix): pass Attribute models into checkAttribute

3a5fb77

Appwrite E2E migrations failed because checkAttribute now requires Attribute, and the destination still handed it a metadata Document.

greptile-apps Bot reviewed Aug 21, 2026

View reviewed changes

Comment thread src/Migration/Destinations/Appwrite.php

abnegate and others added 2 commits August 21, 2026 22:53

(refactor): use Schema\Order and wrap Collection constructors

04ab893

Index types already used IndexType; column direction was still a raw ASC string. Collection constructors with multiple named params were also jammed on one line.

greptile-apps Bot reviewed Aug 27, 2026

View reviewed changes

Comment thread src/Migration/Destinations/Appwrite.php

(fix): prevent concurrent provisioning recovery

4c3a89a

greptile-apps Bot reviewed Aug 29, 2026

View reviewed changes

Comment thread src/Migration/Destinations/Appwrite.php Outdated

abnegate added 4 commits August 30, 2026 02:07

(fix): require explicit CLI provisioning recovery

519e573

(fix): require provisioning lifecycle policy

8c2776d

(fix): persist database provisioning owner

9d94e93

(fix): fence database provisioning attempts

06f41d0

(fix): recover terminal migration failures

dbcc441

greptile-apps Bot reviewed Aug 31, 2026

View reviewed changes

Comment thread src/Migration/Destinations/Appwrite.php Outdated

abnegate added 3 commits September 1, 2026 05:09

(fix): require terminal migration attestation

e7d88de

(fix): surface database finalizer failures

3f42d20

(fix): fence database migration finalization

ec7db1f

greptile-apps Bot reviewed Aug 31, 2026

View reviewed changes

Comment thread bin/MigrationCLI.php

abnegate added 3 commits September 1, 2026 11:06

(fix): reject empty guarded database writes

97863f3

(fix): finalize standalone migrations

e56b964

(chore): repin guarded database writes

60b7a52

greptile-apps Bot reviewed Aug 31, 2026

View reviewed changes

Comment thread bin/MigrationCLI.php Outdated

abnegate and others added 3 commits September 1, 2026 11:39

(fix): finalize partial standalone migrations

1724bf3

(chore): verify migration against the final query database

8ab3017

Add TikTok and Kakao OAuth2 providers to the migration allow-list

e74e266

Both store a plain client id and secret, so only the client id is readable and migratable. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> (cherry picked from commit ea342db)

greptile-apps Bot reviewed Sep 9, 2026

View reviewed changes

Comment thread tests/Migration/Unit/Destinations/AppwriteCheckAttributeTest.php Outdated

greptile-apps Bot reviewed Sep 9, 2026

View reviewed changes

Comment thread tests/Migration/Unit/MigrationCLITest.php Outdated

Uh oh!

Conversation

abnegate commented Aug 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why this approach

Lock repairs that came with the re-pin

A live bug the re-pin exposed

Chain

Verified

Not verified

Uh oh!

greptile-apps Bot commented Aug 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Confidence Score: 5/5

Summary

Uh oh!

abnegate commented Aug 21, 2026

Uh oh!

abnegate commented Aug 21, 2026

Uh oh!

abnegate commented Aug 21, 2026

Uh oh!

Uh oh!

abnegate commented Aug 21, 2026

Uh oh!

Uh oh!

Uh oh!

abnegate commented Aug 31, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

abnegate commented Sep 9, 2026

Uh oh!

abnegate commented Sep 9, 2026

Uh oh!

abnegate commented Sep 9, 2026

Uh oh!

Uh oh!

abnegate commented Sep 9, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

abnegate commented Aug 21, 2026 •

edited

Loading

greptile-apps Bot commented Aug 21, 2026 •

edited

Loading