Skip to content

Conversation

garrrikkotua
Copy link
Contributor

@garrrikkotua garrrikkotua commented Jun 2, 2023

Changes proposed ✍️

What

🤖 Generated by Copilot at a2db0fd

This pull request adds a new feature to allow users to connect their Stack Overflow accounts to the platform and get insights from their questions and answers. It also refactors and optimizes the integration run, data, and stream services to use a new int-run-worker service and a Map cache for the platform configuration. It also updates the frontend and backend components to support the new feature and adds logging and debugging improvements. It introduces new files and functions in the integrations library to handle the Stack Overflow API calls, data processing, and stream generation.

🤖 Generated by Copilot at a2db0fd

To get insights from Stack Overflow
We fetch questions and answers, you know
We use int-run-worker
And integration_stream_worker
And cache the config with a Map, not a var

Why

How

🤖 Generated by Copilot at a2db0fd

  • Add support for Stack Overflow integration, which allows users to connect their Stack Overflow accounts to the platform and get insights from their questions and answers (link, link)
  • Define the member attributes, types, constants, and enums for the Stack Overflow integration, such as reputation, badges, tags, and activity types (link, link)
  • Implement the functions to fetch data from the Stack Overflow API, such as questions, answers, and user profiles, using pagination and keywords (link, link, link, link)
  • Implement the functions to generate, process, and consume streams of data from the Stack Overflow API, using the integration settings and the integration_stream_worker service (link, link, link)
  • Refactor the PLATFORM_CONFIG function in the integration_data_worker, integration_run_worker, and integration_stream_worker services to use a Map instead of a variable for caching the platform-specific configuration values, which improves the performance and memory usage (link, link, link)
  • Replace the creation of an IntegrationRun entity with a call to sendStartIntegrationRunMessage in the integrationService, which sends a message to the int-run-worker service to start processing the integration data, which decouples the integration run logic from the integrationService and uses a message-driven architecture instead (link)
  • Remove the call to sendNodeWorkerMessage in the integrationService, which was used to send a message to the node-worker service to process the integration data, which is replaced by the int-run-worker service, which handles the integration run logic in a more scalable and robust way (link)
  • Add a log message to indicate the start of creating a Stack Overflow integration in the integrationService (link)
  • Add a log message to show the platform settings for the current stream being processed by the integration_stream_worker service, which provides more visibility into the stream processing logic and configuration (link)
  • Add a new configuration entry for the stackoverflow platform in the integration_stream_worker service, which specifies the environment variable CROWD_STACKEXCHANGE_KEY that holds the Stack Overflow API key (link)
  • Update the value of MAX_STACK_OVERFLOW_QUESTIONS_PER_TAG in the stackoverflow-connect-drawer.vue component, which is the maximum number of questions that can be fetched from the Stack Overflow API for a given tag, which allows users to get insights from a large number of questions related to their interests or skills (link)

Checklist ✅

  • Label appropriately with Feature, Improvement, or Bug.
  • Add screehshots to the PR description for relevant FE changes
  • New backend functionality has been unit-tested.
  • API documentation has been updated (if necessary) (see docs on API documentation).
  • Quality standards are met.

@garrrikkotua garrrikkotua changed the title Improvement/move stackoverflow Move Stack Overflow to the new integration framework Jun 2, 2023
@garrrikkotua garrrikkotua requested a review from themarolt June 2, 2023 13:51
@garrrikkotua garrrikkotua merged commit 8f91c42 into main Jun 2, 2023
@garrrikkotua garrrikkotua deleted the improvement/move-stackoverflow branch June 2, 2023 16:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants