Skip to content

Conversation

skwowet
Copy link
Member

@skwowet skwowet commented Aug 24, 2023

Changes proposed ✍️

What

🤖 Generated by Copilot at e5a06fb

This pull request improves the OpenSearch indexing process for the search_sync_worker service. It adds a function to trim long strings to avoid indexing errors, updates the types and settings for the indexes, and uses versioned indexes and aliases for each document type. It also adds new scripts to the package.json file for testing and debugging the indexing process. The changes affect the files activity.sync.service.ts, member.sync.service.ts, opensearch.service.ts, types.ts, package.json, and byteLength.ts.

🤖 Generated by Copilot at e5a06fb

To index data in OpenSearch
We added scripts and functions to fetch
We trimmed the strings
And added some things
To package.json and types.ts

Why

How

🤖 Generated by Copilot at e5a06fb

  • Add new function trimUtf8ToMaxByteLength to common library to truncate strings that exceed OpenSearch byte limit (link)
  • Use trimUtf8ToMaxByteLength to trim string attributes of activity and member documents before indexing them (link, link, link, link, link, link)
  • Add new scripts to package.json for running OpenSearch indexing tasks (link)
  • Modify OpenSearch service to use versioned indexes and aliases for managing documents (link, link, link, link, link, link, link, link, link)
  • Add new constant IndexVersions to types file to store version numbers for each index name (link)
  • Add new property analyzer to prefixedMapping object to specify custom analyzer for lowercasing keywords (link)
  • Add new object OPENSEARCH_INDEX_SETTINGS to types file to define settings for each index name (link)

Checklist ✅

  • Label appropriately with Feature, Improvement, or Bug.
  • Add screehshots to the PR description for relevant FE changes
  • New backend functionality has been unit-tested.
  • API documentation has been updated (if necessary) (see docs on API documentation).
  • Quality standards are met.

skwowet added 30 commits July 28, 2023 15:05
@skwowet skwowet added Bug Created by Linear-GitHub Sync Improvement Created by Linear-GitHub Sync labels Aug 24, 2023
@skwowet skwowet self-assigned this Aug 24, 2023
@skwowet skwowet removed the Bug Created by Linear-GitHub Sync label Aug 24, 2023
@skwowet skwowet merged commit 9748d10 into main Aug 25, 2023
@skwowet skwowet deleted the bugfix/searchFullName branch August 25, 2023 01:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Improvement Created by Linear-GitHub Sync
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant