Skip to content

ci: minor refactors and improvements #7308

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Feb 24, 2025
Merged

ci: minor refactors and improvements #7308

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
7ea8cf4
ci: remove unused Docker build workflows
SanchithHegde Feb 18, 2025
58285a8
ci: move connector test and UI test workflows to `archive` directory
SanchithHegde Feb 18, 2025
53908ab
ci: update versions of actions
SanchithHegde Feb 18, 2025
594aa32
ci: use `taiki-e/install-action` instead of `baptiste0928/cargo-insta…
SanchithHegde Feb 18, 2025
cabe7ff
ci: remove `CARGO_REGISTRIES_CRATES_IO_PROTOCOL=sparse` environment v…
SanchithHegde Feb 19, 2025
a952d03
ci: use `postgres:alpine` and `redis:alpine` images for service conta…
SanchithHegde Feb 19, 2025
f2898f2
ci: remove unnecessary comments and if conditions
SanchithHegde Feb 19, 2025
786d0e6
ci(postman-collection-runner): remove unused `workflow_dispatch` trigger
SanchithHegde Feb 19, 2025
9215f31
ci(CI-push): set environment variables to handle flaky CI failures du…
SanchithHegde Feb 19, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 15 additions & 21 deletions .github/workflows/CI-pr.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,6 @@ name: CI-pr

on:
pull_request:
# paths:
# - ".github/workflows/**"
# - "crates/**"
# - "examples/**"
# - "Cargo.lock"
# - "Cargo.toml"

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
Expand Down Expand Up @@ -127,24 +121,24 @@ jobs:
repo-token: ${{ secrets.GITHUB_TOKEN }}

# - name: Install sccache
# uses: taiki-e/install-action@v2.33.28
# uses: taiki-e/install-action@v2
# with:
# tool: sccache
# checksum: true

- name: Install rust cache
uses: Swatinem/[email protected]
with:
save-if: false
with:
save-if: false

- name: Install cargo-hack
uses: taiki-e/install-action@v2.33.28
uses: taiki-e/install-action@v2
with:
tool: cargo-hack
checksum: true

- name: Install just
uses: taiki-e/install-action@v2.41.10
uses: taiki-e/install-action@v2
with:
tool: just
checksum: true
Expand Down Expand Up @@ -235,24 +229,24 @@ jobs:
repo-token: ${{ secrets.GITHUB_TOKEN }}

# - name: Install sccache
# uses: taiki-e/install-action@v2.33.28
# uses: taiki-e/install-action@v2
# with:
# tool: sccache
# checksum: true

- name: Install rust cache
uses: Swatinem/[email protected]
with:
save-if: false
with:
save-if: false

- name: Install cargo-hack
uses: taiki-e/install-action@v2.33.28
uses: taiki-e/install-action@v2
with:
tool: cargo-hack
checksum: true

- name: Install just
uses: taiki-e/install-action@v2.41.10
uses: taiki-e/install-action@v2
with:
tool: just
checksum: true
Expand All @@ -262,7 +256,7 @@ jobs:
run: .github/scripts/install-jq.sh

# - name: Install cargo-nextest
# uses: taiki-e/install-action@v2.33.28
# uses: taiki-e/install-action@v2
# with:
# tool: cargo-nextest
# checksum: true
Expand Down Expand Up @@ -326,11 +320,11 @@ jobs:

- name: Install rust cache
uses: Swatinem/[email protected]
with:
save-if: false
with:
save-if: false

- name: Install just
uses: taiki-e/install-action@v2.41.10
uses: taiki-e/install-action@v2
with:
tool: just
checksum: true
Expand Down
52 changes: 35 additions & 17 deletions .github/workflows/CI-push.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,12 +4,6 @@ on:
push:
branches:
- main
# paths:
# - ".github/workflows/**"
# - "crates/**"
# - "examples/**"
# - "Cargo.lock"
# - "Cargo.toml"

merge_group:
types:
Expand All @@ -19,6 +13,27 @@ concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

env:
# Disable incremental compilation.
#
# Incremental compilation is useful as part of an edit-build-test-edit cycle,
# as it lets the compiler avoid recompiling code that hasn't changed. However,
# on CI, we're not making small edits; we're almost always building the entire
# project from scratch. Thus, incremental compilation on CI actually
# introduces *additional* overhead to support making future builds
# faster...but no future builds will ever occur in any given CI environment.
#
# See https://matklad.github.io/2021/09/04/fast-rust-builds.html#ci-workflow
# for details.
CARGO_INCREMENTAL: 0
# Allow more retries for network requests in cargo (downloading crates) and
# rustup (installing toolchains). This should help to reduce flaky CI failures
# from transient network timeouts or other issues.
CARGO_NET_RETRY: 10
RUSTUP_MAX_RETRIES: 10
# Don't emit giant backtraces in the CI logs.
RUST_BACKTRACE: short

jobs:
formatting:
name: Check formatting
Expand Down Expand Up @@ -77,9 +92,9 @@ jobs:
uses: arduino/setup-protoc@v3
with:
repo-token: ${{ secrets.GITHUB_TOKEN }}

# - name: Install sccache
# uses: taiki-e/install-action@v2.33.28
# uses: taiki-e/install-action@v2
# with:
# tool: sccache
# checksum: true
Expand All @@ -89,12 +104,13 @@ jobs:
save-if: ${{ github.event_name == 'push' }}

- name: Install cargo-hack
uses: baptiste0928/cargo-install@v2.2.0
uses: taiki-e/install-action@v2
with:
crate: cargo-hack
tool: cargo-hack
checksum: true

- name: Install just
uses: taiki-e/install-action@v2.41.10
uses: taiki-e/install-action@v2
with:
tool: just
checksum: true
Expand Down Expand Up @@ -171,18 +187,19 @@ jobs:
repo-token: ${{ secrets.GITHUB_TOKEN }}

# - name: Install sccache
# uses: taiki-e/install-action@v2.33.28
# uses: taiki-e/install-action@v2
# with:
# tool: sccache
# checksum: true

- name: Install cargo-hack
uses: baptiste0928/cargo-install@v2.2.0
uses: taiki-e/install-action@v2.41.10
with:
crate: cargo-hack
tool: cargo-hack
checksum: true

- name: Install just
uses: taiki-e/install-action@v2.41.10
uses: taiki-e/install-action@v2
with:
tool: just
checksum: true
Expand All @@ -192,9 +209,10 @@ jobs:
run: .github/scripts/install-jq.sh

# - name: Install cargo-nextest
# uses: baptiste0928/cargo-install@v2.2.0
# uses: taiki-e/install-action@v2
# with:
# crate: cargo-nextest
# tool: cargo-nextest
# checksum: true
Comment on lines 211 to +215
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can just remove the commented ones since they're unused? or do we have it kept as a reference?


- uses: Swatinem/[email protected]
with:
Expand Down
4 changes: 1 addition & 3 deletions .github/workflows/connector-sanity-tests.yml → ...kflows/archive/connector-sanity-tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,8 +36,6 @@ env:
RUSTUP_MAX_RETRIES: 10
# Don't emit giant backtraces in the CI logs.
RUST_BACKTRACE: short
# Use cargo's sparse index protocol
CARGO_REGISTRIES_CRATES_IO_PROTOCOL: sparse

jobs:
test_connectors:
Expand Down Expand Up @@ -87,7 +85,7 @@ jobs:
toolchain: stable 2 weeks ago

- uses: Swatinem/[email protected]
with:
with:
save-if: false

- name: Decrypt connector auth file
Expand Down
12 changes: 5 additions & 7 deletions ...b/workflows/connector-ui-sanity-tests.yml → ...ows/archive/connector-ui-sanity-tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,8 +31,6 @@ env:
RUSTUP_MAX_RETRIES: 10
# Don't emit giant backtraces in the CI logs.
RUST_BACKTRACE: short
# Use cargo's sparse index protocol
CARGO_REGISTRIES_CRATES_IO_PROTOCOL: sparse
RUST_MIN_STACK: 10485760

jobs:
Expand Down Expand Up @@ -77,7 +75,7 @@ jobs:
if: (github.event_name == 'merge_group') || (github.event.pull_request.head.repo.full_name != github.event.pull_request.base.repo.full_name)
shell: bash
run: |
echo "Skipped tests as the event is merge_group or pull_request from external repository"
echo "Skipped tests as the event is merge_group or pull_request from external repository"
exit 0

- name: Checkout repository
Expand Down Expand Up @@ -123,16 +121,16 @@ jobs:

- name: Build and Cache Rust Dependencies
uses: Swatinem/[email protected]
with:
save-if: false
with:
save-if: false

- name: Install Diesel CLI with Postgres Support
uses: baptiste0928/cargo-install@v2.2.0
uses: baptiste0928/cargo-install@v3.3.0
if: ${{ (github.event_name == 'pull_request') && (github.event.pull_request.head.repo.full_name == github.event.pull_request.base.repo.full_name) }}
with:
crate: diesel_cli
features: postgres
args: "--no-default-features"
args: --no-default-features

- name: Diesel migration run
if: ${{ (github.event_name == 'pull_request') && (github.event.pull_request.head.repo.full_name == github.event.pull_request.base.repo.full_name) }}
Expand Down
59 changes: 0 additions & 59 deletions .github/workflows/auto-release-tag.yml
View file
Open in desktop

This file was deleted.

3 changes: 1 addition & 2 deletions .github/workflows/create-hotfix-branch.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,6 @@ jobs:

steps:
- name: Generate a token
if: ${{ github.event.pull_request.head.repo.full_name == github.event.pull_request.base.repo.full_name }}
id: generate_token
uses: actions/create-github-app-token@v1
with:
Expand Down Expand Up @@ -43,4 +42,4 @@ jobs:
else
echo "::error::Failed to create hotfix branch"
exit 1
fi
fi
11 changes: 5 additions & 6 deletions .github/workflows/create-hotfix-tag.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,6 @@ jobs:

steps:
- name: Generate a token
if: ${{ github.event.pull_request.head.repo.full_name == github.event.pull_request.base.repo.full_name }}
id: generate_token
uses: actions/create-github-app-token@v1
with:
Expand All @@ -23,10 +22,10 @@ jobs:
token: ${{ steps.generate_token.outputs.token }}

- name: Install git-cliff
uses: baptiste0928/cargo-install@v2.2.0
uses: taiki-e/install-action@v2
with:
crate: git-cliff
version: 1.2.0
tool: git-cliff
checksum: true

- name: Check if the input is valid hotfix branch
shell: bash
Expand Down Expand Up @@ -55,7 +54,7 @@ jobs:
local previous_tag="${1}"
local previous_hotfix_number
local next_tag

previous_hotfix_number="$(echo "${previous_tag}" | awk -F. '{ print $4 }' | sed -E 's/([0-9]+)(-hotfix([0-9]+))?/\3/')"

if [[ -z "${previous_hotfix_number}" ]]; then
Expand All @@ -66,7 +65,7 @@ jobs:
local hotfix_number=$((previous_hotfix_number + 1))
next_tag="${previous_tag/%${previous_hotfix_number}/${hotfix_number}}"
fi

echo "${next_tag}"
}

Expand Down
Loading
Loading