Skip to content

Conversation

skwowet
Copy link
Member

@skwowet skwowet commented Aug 21, 2023

Changes proposed ✍️

What

🤖 Generated by Copilot at 977cacf

Refactored and added a common utility function for string truncation to the @crowd/common package. This function is used to trim strings that exceed a byte length limit for OpenSearch indexing in various services. Updated the imports and usage of the function in aws.ts, member.sync.service.ts, and sentiment.ts.

🤖 Generated by Copilot at 977cacf

To index strings without any trouble
We need to trim them when they're too long
So we added a function
To do the reduction
And used it in aws and sentiment along

Why

How

🤖 Generated by Copilot at 977cacf

  • Move trimUtf8ToMaxByteLength function to @crowd/common package to avoid duplication and improve reusability (link, link)
  • Import trimUtf8ToMaxByteLength function from @crowd/common package in aws.ts, member.sync.service.ts, and sentiment.ts to truncate strings that exceed OpenSearch byte length limit (link, link, link)
  • Export byteLength.ts file from @crowd/common package in index.ts to allow other modules to access trimUtf8ToMaxByteLength function (link)
  • Add maxByteLength constant to MemberSyncService class in member.sync.service.ts to define OpenSearch byte length limit for string attributes (link)
  • Apply trimUtf8ToMaxByteLength function to data of special attribute and value of string attribute in member.sync.service.ts to avoid OpenSearch indexing errors (link, link)

Checklist ✅

  • Label appropriately with Feature, Improvement, or Bug.
  • Add screehshots to the PR description for relevant FE changes
  • New backend functionality has been unit-tested.
  • API documentation has been updated (if necessary) (see docs on API documentation).
  • Quality standards are met.

@skwowet skwowet added the Improvement Created by Linear-GitHub Sync label Aug 21, 2023
@skwowet skwowet self-assigned this Aug 21, 2023
@skwowet skwowet changed the title Avoid exceeding max byte length during OpenSearch indexing Handle max byte limit during OpenSearch member Indexing Aug 21, 2023
@skwowet skwowet merged commit 8de242b into main Aug 21, 2023
@skwowet skwowet deleted the improvement/opensearch-max-byte branch August 21, 2023 16:01
@skwowet skwowet changed the title Handle max byte limit during OpenSearch member Indexing Avoid exceeding max byte length during OpenSearch member indexing Aug 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Improvement Created by Linear-GitHub Sync
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants