chore: upgrade aws-sdk to 3 for ddb util mock #3337

svidgen · 2025-09-16T19:23:35Z

Description of changes

Instructions for Q to help with coding, testing, investigating test/e2e failures
Q assisted upgrade aws-sdk v2 -> v3, starting with ddb utils

CDK / CloudFormation Parameters Changed

Issue #, if available

Description of how you validated changes

Checklist

PR description included
yarn test passes
[E2E test run linked](https://us-east-1.console.aws.amazon.com/codesuite/codebuild/594813022831/projects/amplify-category-api-e2e-workflow
/batch/amplify-category-api-e2e-workflow:ff165713-ece3-4038-8546-3a08020433fe?region=us-east-1)
Tests are changed or added
Relevant documentation is changed or added (and PR referenced)
New AWS SDK calls or CloudFormation actions have been added to relevant test and service IAM policies
Any CDK or CloudFormation parameter changes are called out explicitly

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

- Migrate ssmClient.ts from aws-sdk v2 to @aws-sdk/client-ssm v3 - Update all method calls to use command pattern instead of .promise() - Update error handling to use exception names instead of error codes - Update package.json to add @aws-sdk/client-ssm dependency - Remove aws-sdk v2 dependency from amplify-category-api package - Update test file to use v3 SDK mocking patterns - All tests passing, build successful This completes the amplify-category-api migration to AWS SDK v3.

…unt ids

AWS AppSync service changed error message format from 'GraphQL API {id} not found.' to 'API not found'. This is not related to SSM migration but due to AWS service behavior change. Updated test expectation to match current service response.

This reverts commit 3de734d.

- Update amplify-util-mock DynamoDB utilities to use @aws-sdk/client-dynamodb - Replace aws-sdk-mock with aws-sdk-client-mock for testing - Fix type definitions to use proper SDK v3 enums (KeyType, ScalarAttributeType) - Update test patterns and mock assertions for SDK v3 compatibility - All DynamoDB tests passing with 86%+ coverage

- Fix TypeScript import syntax in DynamoDB test files - Add comprehensive e2e testing workflow to AmazonQ.md - Update dependency licenses and yarn.lock files

- Replace SDK v2 client from dynamodb-simulator with proper SDK v3 DynamoDBClient - Fix client config access for SDK v3 compatibility - Store emulator URL on client for datasource configuration - Resolves mock_e2e_tests failures in CI

- Add yarn e2e-list to enumerate recent build batches - Add yarn e2e-failed to show failed builds with log commands - Add yarn e2e-logs to fetch build logs for specific builds - Update documentation with new commands - Provides low-context mechanism for Q to query e2e test results

- Display source branch for each build batch - Helps identify relevant batches for current working context - Branch shown alongside timestamp for better context matching

- Change default from 10 to 20 most recent batches - Update help text and documentation - Provides better coverage of recent test activity

- Add optional filter parameter to distinguish e2e vs canary runs - E2E runs: contain 'e2e-workflow' in batch ID - Canary runs: contain 'canary-workflow' in batch ID - Usage: yarn e2e-list [limit] [e2e|canary] - Helps focus on relevant test types for troubleshooting

…wirej/aws-sdk-v3-amplify-category-api

…st run

ShadowCat567

In general this looks good, especially since most of this is just find and replace.

ShadowCat567 · 2025-09-24T14:17:24Z

packages/amplify-util-mock/package.json

    "amplify-function-plugin-interface": "^1.9.7",
    "amplify-nodejs-function-runtime-provider": "^2.5.23",
    "aws-appsync": "^4.1.9",
    "aws-sdk": "^2.1113.0",


are this (aws-sdk) and aws-sdk-mock able to be removed as dependencies now?

Oversight ... probably means Q and I missed a few things. Working on it.

ShadowCat567 · 2025-09-24T14:20:50Z

packages/amplify-util-mock/src/__tests__/utils/dynamo-db/helper.test.ts

    await waitTillTableStateIsActivePromise;
-    expect(describeTableMock.mock.calls[0][0]).toEqual({ TableName: 'table1' });
+
+    expect(ddbMock.commandCalls(DescribeTableCommand)[0].args[0].input).toEqual({ TableName: 'table1' });


you should be able to use some stuff from this package aws-sdk-client-mock-jest, which is a part of the aws-sdk-client-mock package to make this look cleaner (https://github.com/m-radzikowski/aws-sdk-client-mock?tab=readme-ov-file#custom-matchers)

ShadowCat567 · 2025-09-24T14:23:00Z

packages/amplify-util-mock/src/__tests__/utils/dynamo-db/helper.test.ts

+
+    const calls = ddbMock.commandCalls(DescribeTableCommand);
+    expect(calls.length).toBe(4);
+    expect(calls[0].args[0].input).toEqual({ TableName: 'table1' });


similar idea where this can look cleaner/be easier to understand with aws-sdk-client-mock-jest (in this case:
EX: expect(ddbMock.commandCalls(DescribeTableCommand)).toHaveLength(4); expect(ddbMock).toHaveReceivedNthCommandWith(1, DescribeTableCommand, { TableName: 'table1' });
...
expect(ddbMock).toHaveReceivedNthCommandWith(4, DescribeTableCommand, { TableName: 'table1' });)
This and the previous comment related to this are not really blocking, more ways of rewriting this that makes it a bit easier to tell what we are expecting to see with this test

ShadowCat567 · 2025-09-24T14:32:05Z

packages/amplify-util-mock/src/__tests__/utils/dynamo-db/util.test.ts

                  {
                    AttributeName: 'createdAt',
-                    KeyType: 'SORT',
+                    KeyType: 'RANGE',


why was this changed?

I think this is v3 being more strict on KeyType. The correct values for this field are HASH or RANGE, as far as I can tell.

ShadowCat567 · 2025-09-24T15:03:21Z

scripts/e2e-test-manager.ts

@@ -0,0 +1,402 @@
+#!/usr/bin/env ts-node


should this file (scripts/e2e-test-manager) and quick-build-status.sh be removed from this PR? It looks like they are artifacts from Q

I was thinking of keeping these scripts around. Should be useful in general.

ShadowCat567 · 2025-09-24T15:04:32Z

.q/README.md

@@ -0,0 +1,47 @@
+# Q Workspace


Do we want to keep this .q folder in the PR?

Good question. I'm of two minds on this. I'll let this question simmer while I dig more on some of the other oddities.

ShadowCat567 · 2025-09-24T15:11:19Z

packages/amplify-util-mock/src/__e2e__/utils/index.ts

-  const client: DynamoDB = await dynamoEmulator.getClient(emulator);
+
+  // Create SDK v3 client instead of using the v2 client from getClient
+  const client = new DynamoDBClient({


Any particular reason why we are creating the DynamoDB Client here instead of updating the dynamoEmulator to use aws-sdk v3?

Unsure yet. I need to dig on this a little myself.

Oh yeah, I think I let this one slide to limit scope. Keep Q updates restricted to per-package updates -- but, taking a peek at getClient(), it looks like it should be fairly isolated. I'll make that change as long as there's no significant collateral damage.

OK. There will either be some sprawl or a very light temporary compatibility layer if we tug on this in this PR. Would prefer to keep this as-is in the PR.

ShadowCat567 · 2025-09-24T15:13:09Z

packages/amplify-util-mock/src/__e2e__/utils/index.ts


  if (client) {
    await createAndUpdateTable(client, config);
-    config = configureDDBDataSource(config, client.config);


are we unable to extract the config from the aws sdk v3 client?

…rtions - Replace manual commandCalls() checks with toHaveReceivedCommand* matchers - Add aws-sdk-client-mock-jest import to test files - Improve test readability and maintainability - All tests continue to pass with cleaner assertions

- Update getClient() in amplify-dynamodb-simulator to use DynamoDBClient - Add backward compatibility layer to maintain v2-style API (.promise() methods) - Strip v3 metadata from responses to match v2 response format - Update e2e utils to use dynamoEmulator.getClient() instead of direct creation - Add @aws-sdk/client-dynamodb dependency to amplify-dynamodb-simulator - All tests passing with v3 client under the hood

This reverts commit 80a4b7a.

- Remove aws-sdk and aws-sdk-mock from package.json - Update e2e test to remove AWS SDK v2 credential clearing - Fix KeyType from 'SORT' to 'RANGE' in util.test.ts for SDK v3 compatibility - All tests passing with AWS SDK v3 migration complete

- Update dependency_licenses.txt to reflect removed AWS SDK v2 packages - Update yarn.lock to remove unused dependencies from aws-sdk-mock and aws-sdk v2 - Fixes e2e test failures for verify_yarn_lock and verify_dependency_licenses_extract

svidgen added 11 commits August 20, 2025 09:45

chore: add Q's migration plan

8853dfb

Merge branch 'release-api-plugin-stable' into wirej/gen1-migrate-aws-sdk

3d93089

chore: q self-instructions

40d746f

chore: more q context

a106b62

chore: q discovery

29cdf93

chore: q todo update

5c35def

chore: teaching q to run e2e tests and monitor status ...

102cbcb

chore: fixes to q instructions for failed build retries, cleanse acco…

6868920

…unt ids

chore: q updated todos

a97e7ef

svidgen requested review from a team as code owners September 16, 2025 19:23

svidgen marked this pull request as draft September 16, 2025 19:23

ShadowCat567 mentioned this pull request Sep 16, 2025

Feat: Update Gen1 Data to be compatible with SDK v2->V3 migrations happening in Gen1 CLI #3310

Merged

7 tasks

svidgen added 15 commits September 17, 2025 09:44

Revert "feat: migrate amplify-category-api SSM client to AWS SDK v3"

9ca2eb6

This reverts commit 3de734d.

chore: add reminder to use say command for status updates

577b216

chore: fix import syntax and add e2e workflow documentation

aca6201

- Fix TypeScript import syntax in DynamoDB test files - Add comprehensive e2e testing workflow to AmazonQ.md - Update dependency licenses and yarn.lock files

chore: fix whitespace formatting in e2e utils

08efa03

chore: fixing some Q garbage

7bff046

feat: add branch information to e2e-list command

6c5c7de

- Display source branch for each build batch - Helps identify relevant batches for current working context - Branch shown alongside timestamp for better context matching

feat: increase default e2e-list limit to 20 batches

9d0c5e8

- Change default from 10 to 20 most recent batches - Update help text and documentation - Provides better coverage of recent test activity

chore: fix test error message handling

0d37b50

Merge remote-tracking branch 'origin/release-api-plugin-stable' into …

9bbe371

…wirej/aws-sdk-v3-amplify-category-api

chore: updated dependency_licenses.txt

daa9fc3

chore: update TODO status after successful parent branch merge and te…

3f9c981

…st run

svidgen marked this pull request as ready for review September 24, 2025 14:03

svidgen changed the title ~~[WIP] chore: aws sdk v3 amplify category api~~ chore: aws sdk v3 amplify category api Sep 24, 2025

svidgen changed the title ~~chore: aws sdk v3 amplify category api~~ chore: upgrade aws-sdk to 3 for ddb util mock Sep 24, 2025

svidgen mentioned this pull request Sep 24, 2025

WIP: upgrade aws-sdk to v3 for ddb simulator #3344

Draft

7 tasks

ShadowCat567 reviewed Sep 24, 2025

View reviewed changes

svidgen added 6 commits September 24, 2025 11:25

Revert "feat: migrate dynamoEmulator getClient to AWS SDK v3"

4460d65

This reverts commit 80a4b7a.

chore: fix q docs to use ada --once

5691981

chore: upgrade aws-sdk to 3 for ddb util mock #3337

Are you sure you want to change the base?

chore: upgrade aws-sdk to 3 for ddb util mock #3337

Uh oh!

Conversation

svidgen commented Sep 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description of changes

CDK / CloudFormation Parameters Changed

Issue #, if available

Description of how you validated changes

Checklist

Uh oh!

ShadowCat567 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

svidgen commented Sep 16, 2025 •

edited

Loading