Skip to content

Conversation

sebrandon1
Copy link
Contributor

@sebrandon1 sebrandon1 commented Jun 16, 2025

Changes:

  • Forked and created my own maintained version of the grab repo and stripped all extraneous/outdated things out of it.
  • Old repo was stuck on Go 1.14 with multiple go.mod files within different folders. Very confusing and an old way of organizing a Go project.
  • New repo organization puts all public functions inside of lib/ and accurately documents how to use it. Changed the references in crc to use the new lib.
  • New repo has a binary that you can build and download files via grab.
  • New repo has dependencies managed by dependabot.

Dependency Updates:

  • go.mod: Removed the dependency github.com/cavaliergopher/grab/v3 and added github.com/sebrandon1/grab v0.0.4, indicating a shift to a forked version of the library. [1] [2]

Code Adjustments:

  • pkg/download/download.go: Updated the import statement to use the forked library grab "github.com/sebrandon1/grab/lib" instead of github.com/cavaliergopher/grab/v3.

Summary by Sourcery

Replace the outdated cavaliergopher/grab/v3 dependency with a custom, maintained fork sebrandon1/grab and update code references and vendor files accordingly.

Enhancements:

  • Migrate to a streamlined fork of the grab library under github.com/sebrandon1/grab.
  • Refresh the vendor directory to include the new grab fork and remove old package files.

Chores:

  • Remove the old cavaliergopher/grab/v3 dependency from go.mod and add github.com/sebrandon1/grab v0.0.4.
  • Update import paths in pkg/download/download.go to use the new grab library.

Summary by CodeRabbit

  • New Features

    • Added a simplified batch download API to start concurrent downloads and receive results asynchronously.
  • Bug Fixes

    • Improved error handling and cleanup for downloads (closers, truncation, partial-file removal).
  • Chores

    • Swapped vendored download library for an alternate implementation.
    • Removed transfer-rate tracking to streamline transfers.
  • Documentation

    • Added README with usage examples for the new download library.

Copy link

sourcery-ai bot commented Jun 16, 2025

Reviewer's Guide

Replaces the outdated cavaliergopher/grab/v3 dependency with a maintained fork (sebrandon1/grab v0.0.1), updates import paths in the download logic, and refreshes the vendor directory to reflect the new library structure.

Class diagram for updated grab dependency usage

classDiagram
    class grab.Client {
    }
    class grab.Request {
    }
    class grab.Response {
    }
    class Download {
        +doRequest(client: grab.Client, req: grab.Request) string, error
    }
    Download --> grab.Client
    Download --> grab.Request
    Download --> grab.Response

    %% Note: The grab package is now imported from github.com/sebrandon1/grab/lib instead of github.com/cavaliergopher/grab/v3
Loading

File-Level Changes

Change Details Files
Updated module dependencies to use forked grab
  • Removed github.com/cavaliergopher/grab/v3 from go.mod
  • Added github.com/sebrandon1/grab v0.0.1 to go.mod
go.mod
Adjusted import in download package to new library path
  • Replaced import of cavaliergopher/grab/v3
  • Imported sebrandon1/grab/lib under alias grab
pkg/download/download.go
Refreshed vendor directory for the new fork
  • Removed old grab/v3 files from vendor
  • Added vendor files for sebrandon1/grab/lib, including README and download.go
  • Updated vendor/modules.txt to include the new dependency
vendor/github.com/cavaliergopher/grab/v3/*
vendor/github.com/sebrandon1/grab/lib/*
vendor/modules.txt
go.sum

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

Copy link

openshift-ci bot commented Jun 16, 2025

Hi @sebrandon1. Thanks for your PR.

I'm waiting for a crc-org member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

Copy link

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @sebrandon1 - I've reviewed your changes and they look great!


Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

@sebrandon1 sebrandon1 force-pushed the replace_grab branch 3 times, most recently from d0c3911 to 3712ea0 Compare June 16, 2025 22:26
@praveenkumar
Copy link
Member

/ok-to-test

@sebrandon1 sebrandon1 force-pushed the replace_grab branch 2 times, most recently from f4c9c1e to 98f1d62 Compare June 23, 2025 22:56
Copy link

coderabbitai bot commented Jun 23, 2025

Walkthrough

Replaced vendored dependency github.com/cavaliergopher/grab/v3 with github.com/sebrandon1/grab v0.0.4, updated imports to .../grab/lib, added a new lib package (including DownloadBatch), improved resource/error handling, and removed the prior BPS gauge and its vendor code.

Changes

Cohort / File(s) Change Summary
Module files
go.mod, vendor/modules.txt
Replaced module github.com/cavaliergopher/grab/v3 with github.com/sebrandon1/grab (v0.0.4); updated vendor/module listings.
Project import
pkg/download/download.go
Switched import from github.com/cavaliergopher/grab/v3 to github.com/sebrandon1/grab/lib (aliased grab); no other logic changes.
Removed vendor: bps
vendor/github.com/cavaliergopher/grab/v3/pkg/bps/*
Deleted BPS package files (bps.go, sma.go) — removed Gauge, SampleFunc, Watch, and SMA implementation.
Added vendored grab/lib package
vendor/github.com/sebrandon1/grab/lib/...
Added new package files and README; package renamed to lib; introduced DownloadBatch and DownloadResponse; many files updated for error handling and package rename.
Transfer / rate logic
vendor/.../grab/lib/transfer.go
Removed dependency on external bps package, added internal gauge interface, set gauge to nil, and removed the background BPS Watch goroutine.
Client & resource handling
vendor/.../grab/lib/client.go, .../response.go, .../download.go
Improved error capture on closing bodies/writers, checked/trapped Truncate and remove errors, updated seek constants, and added DownloadBatch helper.
Package docs & renames
vendor/.../grab/lib/doc.go, error.go, grab.go, rate_limiter.go, request.go, util.go, README.md
Changed package declaration from grablib, adjusted imports/usages, and added README documenting new lib API.
Tests / helpers
vendor/.../grab/lib/test_helpers.go
Added testing utilities for mock HTTP client, temp directory management, response builders, and test client helpers.

Sequence Diagram(s)

sequenceDiagram
    autonumber
    participant Caller
    participant grabLib as grab/lib
    participant GetBatch
    participant Worker as Downloader
    participant FS as FileSystem

    Caller->>grabLib: DownloadBatch(ctx, urls)
    grabLib->>GetBatch: GetBatch(workers=..., dst=".", urls...)
    note right of GetBatch #DDEBF7: starts worker pool
    GetBatch->>Worker: start HTTP download (concurrent)
    Worker->>FS: write file data
    FS-->>Worker: write complete / error
    Worker-->>GetBatch: send *Response
    GetBatch-->>grabLib: responses channel
    grabLib-->>Caller: forward DownloadResponse via channel
    note right of Caller #E6F4EA: Caller consumes async results
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

I hopped through modules, clipped an old vine,
Switched my grab-paths to a friendlier line.
I tidy the bytes and guard every close,
Batch downloads now sprout where the carrot grows.
🐇📦

Pre-merge checks (1 passed, 2 warnings)

❌ Failed checks (2 warnings)
Check name Status Explanation Resolution
Description Check ⚠️ Warning The pull request description does not follow the required repository template: it lacks the “## Description” heading with issue links (Fixes/Relates to), the “## Type of change” section with checkboxes, the “## Proposed changes” breakdown, the “## Testing” plan, and the contribution checklist. Please update the description to use the repository’s template by adding a “## Description” section summarizing the PR and linking relevant issues, fill in the “Fixes” or “Relates to” lines, complete the “## Type of change” checkboxes, provide “## Proposed changes” and “## Testing” details, and include the contribution checklist.
Docstring Coverage ⚠️ Warning Docstring coverage is 63.64% which is insufficient. The required threshold is 80.00%. You can run @coderabbitai generate docstrings to improve docstring coverage.
✅ Passed checks (1 passed)
Check name Status Explanation
Title Check ✅ Passed The title succinctly conveys the central change—replacing the grab dependency in go.mod—and is clear, concise, and focused on the main purpose of the changeset.

Tip

👮 Agentic pre-merge checks are now available in preview!

Pro plan users can now enable pre-merge checks in their settings to enforce checklists before merging PRs.

  • Built-in checks – Quickly apply ready-made checks to enforce title conventions, require pull request descriptions that follow templates, validate linked issues for compliance, and more.
  • Custom agentic checks – Define your own rules using CodeRabbit’s advanced agentic capabilities to enforce organization-specific policies and workflows. For example, you can instruct CodeRabbit’s agent to verify that API documentation is updated whenever API schema files are modified in a PR. Note: Upto 5 custom checks are currently allowed during the preview period. Pricing for this feature will be announced in a few weeks.

Example:

reviews:
  pre_merge_checks:
    custom_checks:
      - name: "Undocumented Breaking Changes"
        mode: "warning"
        instructions: |
          Pass/fail criteria: All breaking changes to public APIs, CLI flags, environment variables, configuration keys, database schemas, or HTTP/GraphQL endpoints must be documented in the "Breaking Change" section of the PR description and in CHANGELOG.md. Exclude purely internal or private changes (e.g., code not exported from package entry points or explicitly marked as internal).

Please share your feedback with us on this Discord post.

✨ Finishing Touches
  • 📝 Generate Docstrings
🧪 Generate unit tests
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🔭 Outside diff range comments (1)
vendor/github.com/sebrandon1/grab/lib/util.go (1)

37-38: Tighten directory permissions to the conventional 0755.
os.MkdirAll(dir, 0777) grants world-writable perms, unexpected in most download scenarios. 0755 is more typical and still lets the current user write.

-        if err := os.MkdirAll(dir, 0777); err != nil {
+        if err := os.MkdirAll(dir, 0o755); err != nil {
🧹 Nitpick comments (3)
vendor/github.com/sebrandon1/grab/lib/util.go (1)

30-37: Avoid shadowing the imported path package.
The parameter path string in mkdirp hides the imported "path" package, which can trip up future refactors (e.g. adding path.* helpers inside the function). Renaming the parameter keeps the namespace clear.

-func mkdirp(path string) error {
-    dir := filepath.Dir(path)
+func mkdirp(dst string) error {
+    dir := filepath.Dir(dst)
vendor/github.com/sebrandon1/grab/lib/README.md (1)

37-90: Replace hard tabs with spaces in code examples for better markdown compatibility.

The code examples use hard tabs which can cause inconsistent rendering across different markdown viewers and platforms. Consider using spaces (typically 4 spaces) for indentation in markdown code blocks.

vendor/github.com/sebrandon1/grab/lib/client.go (1)

558-559: Remove redundant error check.

There's a redundant check for resp.err != nil on line 558 when it was just assigned on line 556.

 		if err := closer.Close(); err != nil {
 			resp.err = fmt.Errorf("cannot close writer for %q: %w", resp.Filename, err)
-			// if we cannot close the writer, we cannot continue
-			if resp.err != nil && !resp.Request.NoStore && resp.Request.deleteOnError {
+			if !resp.Request.NoStore && resp.Request.deleteOnError {
 				// if we cannot close the writer, we cannot continue
 				if err := os.Remove(resp.Filename); err != nil {
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 47be8d0 and 98f1d62.

⛔ Files ignored due to path filters (1)
  • go.sum is excluded by !**/*.sum
📒 Files selected for processing (16)
  • go.mod (1 hunks)
  • pkg/download/download.go (1 hunks)
  • vendor/github.com/cavaliergopher/grab/v3/pkg/bps/bps.go (0 hunks)
  • vendor/github.com/cavaliergopher/grab/v3/pkg/bps/sma.go (0 hunks)
  • vendor/github.com/sebrandon1/grab/lib/README.md (1 hunks)
  • vendor/github.com/sebrandon1/grab/lib/client.go (7 hunks)
  • vendor/github.com/sebrandon1/grab/lib/doc.go (2 hunks)
  • vendor/github.com/sebrandon1/grab/lib/download.go (1 hunks)
  • vendor/github.com/sebrandon1/grab/lib/error.go (1 hunks)
  • vendor/github.com/sebrandon1/grab/lib/grab.go (1 hunks)
  • vendor/github.com/sebrandon1/grab/lib/rate_limiter.go (1 hunks)
  • vendor/github.com/sebrandon1/grab/lib/request.go (1 hunks)
  • vendor/github.com/sebrandon1/grab/lib/response.go (5 hunks)
  • vendor/github.com/sebrandon1/grab/lib/transfer.go (2 hunks)
  • vendor/github.com/sebrandon1/grab/lib/util.go (1 hunks)
  • vendor/modules.txt (1 hunks)
💤 Files with no reviewable changes (2)
  • vendor/github.com/cavaliergopher/grab/v3/pkg/bps/bps.go
  • vendor/github.com/cavaliergopher/grab/v3/pkg/bps/sma.go
🧰 Additional context used
🪛 LanguageTool
vendor/github.com/sebrandon1/grab/lib/README.md

[uncategorized] ~11-~11: Loose punctuation mark.
Context: ... of the file downloaded. - Err error: Any error encountered during download (...

(UNLIKELY_OPENING_PUNCTUATION)

🪛 markdownlint-cli2 (0.17.2)
vendor/github.com/sebrandon1/grab/lib/README.md

41-41: Hard tabs
Column: 1

(MD010, no-hard-tabs)


42-42: Hard tabs
Column: 1

(MD010, no-hard-tabs)


43-43: Hard tabs
Column: 1

(MD010, no-hard-tabs)


47-47: Hard tabs
Column: 1

(MD010, no-hard-tabs)


48-48: Hard tabs
Column: 1

(MD010, no-hard-tabs)


49-49: Hard tabs
Column: 1

(MD010, no-hard-tabs)


50-50: Hard tabs
Column: 1

(MD010, no-hard-tabs)


51-51: Hard tabs
Column: 1

(MD010, no-hard-tabs)


52-52: Hard tabs
Column: 1

(MD010, no-hard-tabs)


53-53: Hard tabs
Column: 1

(MD010, no-hard-tabs)


54-54: Hard tabs
Column: 1

(MD010, no-hard-tabs)


55-55: Hard tabs
Column: 1

(MD010, no-hard-tabs)


56-56: Hard tabs
Column: 1

(MD010, no-hard-tabs)


57-57: Hard tabs
Column: 1

(MD010, no-hard-tabs)


58-58: Hard tabs
Column: 1

(MD010, no-hard-tabs)


68-68: Hard tabs
Column: 1

(MD010, no-hard-tabs)


69-69: Hard tabs
Column: 1

(MD010, no-hard-tabs)


73-73: Hard tabs
Column: 1

(MD010, no-hard-tabs)


74-74: Hard tabs
Column: 1

(MD010, no-hard-tabs)


75-75: Hard tabs
Column: 1

(MD010, no-hard-tabs)


76-76: Hard tabs
Column: 1

(MD010, no-hard-tabs)


77-77: Hard tabs
Column: 1

(MD010, no-hard-tabs)


78-78: Hard tabs
Column: 1

(MD010, no-hard-tabs)


79-79: Hard tabs
Column: 1

(MD010, no-hard-tabs)


80-80: Hard tabs
Column: 1

(MD010, no-hard-tabs)


81-81: Hard tabs
Column: 1

(MD010, no-hard-tabs)


82-82: Hard tabs
Column: 1

(MD010, no-hard-tabs)


83-83: Hard tabs
Column: 1

(MD010, no-hard-tabs)


84-84: Hard tabs
Column: 1

(MD010, no-hard-tabs)


85-85: Hard tabs
Column: 1

(MD010, no-hard-tabs)


86-86: Hard tabs
Column: 1

(MD010, no-hard-tabs)


87-87: Hard tabs
Column: 1

(MD010, no-hard-tabs)


88-88: Hard tabs
Column: 1

(MD010, no-hard-tabs)

⏰ Context from checks skipped due to timeout of 90000ms (17)
  • GitHub Check: build-qe (linux, arm64)
  • GitHub Check: build (windows-2022, 1.23)
  • GitHub Check: build-qe (darwin, amd64)
  • GitHub Check: build-qe (windows, amd64)
  • GitHub Check: build-qe (linux, amd64)
  • GitHub Check: build-qe (darwin, arm64)
  • GitHub Check: build-installer (windows-2022, 1.23)
  • GitHub Check: build (macOS-14, 1.23)
  • GitHub Check: build (macOS-13, 1.23)
  • GitHub Check: build (macOS-13, 1.23)
  • GitHub Check: build (ubuntu-22.04, 1.23)
  • GitHub Check: build (ubuntu-latest, 1.23)
  • GitHub Check: build (windows-2022, 1.23)
  • GitHub Check: build (macOS-14, 1.23)
  • GitHub Check: verify-devcontainer
  • GitHub Check: Run OKD bundle with crc (1.23)
  • GitHub Check: build (ubuntu-latest, 1.23)
🔇 Additional comments (17)
vendor/github.com/sebrandon1/grab/lib/util.go (1)

1-1: Package rename is consistent with new import path – looks good.
The switch to package lib aligns with the new module path github.com/sebrandon1/grab/lib and avoids the historical /v3 import confusion.

vendor/github.com/sebrandon1/grab/lib/error.go (1)

1-1: Package rename acknowledged.
The change to package lib is consistent across the vendored fork – no issues spotted.

vendor/github.com/sebrandon1/grab/lib/rate_limiter.go (1)

1-1: Package rename looks correct.
Nothing else changed in this file; interface remains compatible.

vendor/github.com/sebrandon1/grab/lib/grab.go (1)

1-1: Package declaration updated successfully.
Importers already reference github.com/sebrandon1/grab/lib with alias grab, so the rename won’t break callers.

vendor/github.com/sebrandon1/grab/lib/request.go (1)

1-1: Package rename verified.
All downstream types (Client, Response, etc.) remain in the same package, so type references are intact.

go.mod (1)

45-45: Verify the new fork's maintenance and legitimacy.

The migration to github.com/sebrandon1/grab v0.0.3 introduces a dependency on a personal fork. Please ensure:

  • The fork is actively maintained and trustworthy
  • Version v0.0.3 is stable and production-ready
  • The fork provides clear advantages over the original library
#!/bin/bash
# Description: Verify the new grab fork's maintenance status and compare with original

# Check the new fork's repository details
gh repo view sebrandon1/grab --json createdAt,updatedAt,stargazerCount,forkCount,hasIssuesEnabled

# Check recent activity
gh api repos/sebrandon1/grab/commits --jq '.[0:5] | .[] | {date: .commit.author.date, message: .commit.message}'

# Compare with original repository
gh repo view cavaliergopher/grab --json createdAt,updatedAt,stargazerCount,forkCount
pkg/download/download.go (1)

22-22: LGTM! Import change correctly implemented.

The import statement properly:

  • Uses an explicit alias to maintain API compatibility
  • Points to the new fork's /lib subpackage
  • Aligns with the dependency change in go.mod
vendor/github.com/sebrandon1/grab/lib/doc.go (1)

2-2: LGTM! Package rename correctly implemented.

The package documentation and declaration have been properly updated from grab to lib, maintaining consistency with the new import path structure.

Also applies to: 63-63

vendor/modules.txt (1)

665-667: LGTM! Vendor file correctly updated.

The vendor modules file properly reflects the migration to the new grab fork with the correct module path and subpackage structure.

vendor/github.com/sebrandon1/grab/lib/response.go (6)

1-1: LGTM! Package name correctly updated.

The package declaration has been properly changed to lib to match the new package structure.


6-6: Good addition of fmt import.

The fmt import is needed for the enhanced error logging in the defer functions.


172-172: Excellent modernization using time.Since().

Using time.Since(c.Start) is more idiomatic and readable than time.Now().Sub(c.Start).


207-207: Good migration from deprecated ioutil.

Correctly updated from ioutil.NopCloser to io.NopCloser following Go's deprecation of the ioutil package.


228-235: Excellent improvement in error handling.

The enhanced defer function properly handles and logs file close errors instead of silently ignoring them. The migration from ioutil.ReadAll to io.ReadAll is also correct.


249-251: Good error handling improvement.

The defer function now explicitly handles the file close operation, which is better than the previous simple defer f.Close() that ignored potential errors.

vendor/github.com/sebrandon1/grab/lib/client.go (2)

84-84: Good improvement: buffered Done channel prevents potential deadlocks.

Making the Done channel buffered is a good defensive programming practice that prevents potential deadlocks if the channel is closed before all readers are ready.


350-356: Excellent error handling improvements throughout the file.

The consistent use of fmt.Errorf with %w verb for error wrapping provides better error context and maintains the error chain. The updated seek constants and proper resource cleanup in error paths improve code robustness.

Also applies to: 463-466, 512-516, 555-567, 580-584

Comment on lines +10 to +13
type gauge interface {
Sample(t time.Time, n int64)
BPS() float64
}
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

The gauge interface is defined but never implemented, leaving transfer rate tracking non-functional.

The code defines a gauge interface and includes a BPS() method that depends on it, but the gauge is always initialized to nil. This makes the transfer rate tracking functionality non-operational. If this is intentional, consider removing the gauge-related code entirely. Otherwise, implement the gauge interface to restore transfer rate tracking.

Would you like me to help implement the gauge interface or remove the unused code?

Also applies to: 28-28, 96-101

🤖 Prompt for AI Agents
In vendor/github.com/sebrandon1/grab/lib/transfer.go around lines 10-13, 28, and
96-101, the gauge interface is defined but never implemented or initialized,
causing transfer rate tracking to be non-functional. To fix this, either
implement the gauge interface with the required Sample and BPS methods and
initialize the gauge variable properly to enable transfer rate tracking, or
remove all gauge-related code if this functionality is not needed to avoid dead
code.

Comment on lines +14 to +30
func DownloadBatch(ctx context.Context, urls []string) (<-chan DownloadResponse, error) {
ch := make(chan DownloadResponse)
respch, err := GetBatch(0, ".", urls...)
if err != nil {
return nil, err
}
go func() {
for resp := range respch {
ch <- DownloadResponse{
Filename: resp.Filename,
Err: resp.Err(),
}
}
close(ch)
}()
return ch, nil
}
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Add context cancellation handling to prevent potential goroutine leaks.

The goroutine that forwards responses doesn't handle context cancellation. If the context is cancelled while downloads are in progress, the goroutine will continue running until all responses are processed, potentially causing resource leaks.

Consider this improved implementation:

 func DownloadBatch(ctx context.Context, urls []string) (<-chan DownloadResponse, error) {
 	ch := make(chan DownloadResponse)
 	respch, err := GetBatch(0, ".", urls...)
 	if err != nil {
 		return nil, err
 	}
 	go func() {
-		for resp := range respch {
-			ch <- DownloadResponse{
-				Filename: resp.Filename,
-				Err:      resp.Err(),
+		defer close(ch)
+		for {
+			select {
+			case resp, ok := <-respch:
+				if !ok {
+					return
+				}
+				select {
+				case ch <- DownloadResponse{
+					Filename: resp.Filename,
+					Err:      resp.Err(),
+				}:
+				case <-ctx.Done():
+					return
+				}
+			case <-ctx.Done():
+				return
 			}
 		}
-		close(ch)
 	}()
 	return ch, nil
 }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
func DownloadBatch(ctx context.Context, urls []string) (<-chan DownloadResponse, error) {
ch := make(chan DownloadResponse)
respch, err := GetBatch(0, ".", urls...)
if err != nil {
return nil, err
}
go func() {
for resp := range respch {
ch <- DownloadResponse{
Filename: resp.Filename,
Err: resp.Err(),
}
}
close(ch)
}()
return ch, nil
}
func DownloadBatch(ctx context.Context, urls []string) (<-chan DownloadResponse, error) {
ch := make(chan DownloadResponse)
respch, err := GetBatch(0, ".", urls...)
if err != nil {
return nil, err
}
go func() {
defer close(ch)
for {
select {
case resp, ok := <-respch:
if !ok {
return
}
select {
case ch <- DownloadResponse{
Filename: resp.Filename,
Err: resp.Err(),
}:
case <-ctx.Done():
return
}
case <-ctx.Done():
return
}
}
}()
return ch, nil
}
🤖 Prompt for AI Agents
In vendor/github.com/sebrandon1/grab/lib/download.go lines 14 to 30, the
goroutine forwarding responses from respch to ch does not handle context
cancellation, risking goroutine leaks if the context is cancelled. Modify the
goroutine to select on both the context's Done channel and the respch channel,
exiting early and closing ch if the context is cancelled to prevent resource
leaks.

@sebrandon1
Copy link
Contributor Author

/retest

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (2)
pkg/download/download.go (2)

117-124: Add bounded cancellation to InMemory to avoid indefinite hangs.

If the remote stalls, this path has no context/timeout. Consider a conservative timeout.

Apply this diff:

   grabReq, err := grab.NewRequest("", uri)
   if err != nil {
     return nil, fmt.Errorf("failed to create grab request: %w", err)
   }
   // do not write the downloaded file to disk
   grabReq.NoStore = true
+  // bound the request to avoid indefinite hangs
+  ctx, cancel := context.WithTimeout(context.Background(), 2*time.Minute)
+  defer cancel()
+  grabReq = grabReq.WithContext(ctx)

75-79: Avoid panic on nil ctx; fallback to context.Background().

Prevents surprising crashes and keeps behavior predictable.

-  if ctx == nil {
-    panic("ctx is nil, this should not happen")
-  }
-  req = req.WithContext(ctx)
+  if ctx == nil {
+    ctx = context.Background()
+  }
+  req = req.WithContext(ctx)
📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between fa4b8e1 and b8490df.

⛔ Files ignored due to path filters (1)
  • go.sum is excluded by !**/*.sum
📒 Files selected for processing (2)
  • go.mod (1 hunks)
  • pkg/download/download.go (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • go.mod
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (17)
  • GitHub Check: build (macOS-14, 1.24)
  • GitHub Check: build (macOS-13, 1.24)
  • GitHub Check: build-qe (linux, arm64)
  • GitHub Check: Run OKD bundle with crc (1.24)
  • GitHub Check: build-qe (linux, amd64)
  • GitHub Check: build-qe (windows, amd64)
  • GitHub Check: build-qe (darwin, arm64)
  • GitHub Check: build-installer (windows-2022, 1.24)
  • GitHub Check: build (macOS-14, 1.24)
  • GitHub Check: build-qe (darwin, amd64)
  • GitHub Check: build (ubuntu-22.04, 1.24)
  • GitHub Check: build (ubuntu-latest, 1.24)
  • GitHub Check: build (windows-2022, 1.24)
  • GitHub Check: build (macOS-13, 1.24)
  • GitHub Check: build (ubuntu-latest, 1.24)
  • GitHub Check: build (windows-2022, 1.24)
  • GitHub Check: verify-devcontainer
🔇 Additional comments (2)
pkg/download/download.go (2)

22-22: Import switch to the fork looks correct.

Changing to grab "github.com/sebrandon1/grab/lib" preserves the alias, minimizing churn. No functional differences are introduced in this file by the import change itself.


25-33: grab fork API parity verified
All required symbols (Client.UserAgent/HTTPClient, Request.WithContext/SetChecksum/NoStore, Response.Done/Size/BytesComplete/Filename/Err/Open) are present in the vendored fork and go.mod correctly requires github.com/sebrandon1/grab v0.0.4 with no leftover imports or replace directives.

@cfergeau
Copy link
Contributor

cfergeau commented Sep 9, 2025

I force pushed to rebase the PR.
Imo we could give it a go (ie merge it)

@sebrandon1
Copy link
Contributor Author

@cfergeau I'll do a rebase and update to my branch quick.

@cfergeau
Copy link
Contributor

cfergeau commented Sep 9, 2025

@cfergeau I'll do a rebase and update to my branch quick.

I’ve done the rebase. I’ve seen there’s a 0.0.5 release, but the update to go 1.25 is not something we want to do in crc just yet, so I’ve kept 0.0.4

Copy link

openshift-ci bot commented Sep 9, 2025

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign cfergeau for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (6)
vendor/github.com/sebrandon1/grab/lib/README.md (2)

41-60: Silence markdownlint noise from tabs in code blocks (MD010).

These lines contain hard tabs inside fenced Go code. Prefer excluding vendor/** from markdownlint or disabling MD010 for fenced code to avoid failing CI on vendored docs rather than editing upstream text.

If you keep linting vendor, update the linter config accordingly; otherwise confirm vendor/** is excluded.


68-90: Same MD010 tab issue in the second code block.

Apply the same linter exclusion/override here rather than modifying vendored content.

vendor/github.com/sebrandon1/grab/lib/test_helpers.go (4)

1-10: Test helpers live in non-test file under vendor.

These helpers import testing and compile into the vendored package. Not harmful, but ideally they’d be *_test.go upstream so they don’t ship in normal builds. No action needed in this PR; consider proposing upstream.


44-53: Minor consistency: derive ContentLength from body.

The default response hardcodes 12. Consider using len("test content") for consistency with createMockHTTPResponse.

-        ContentLength: 12,
+        ContentLength: int64(len("test content")),

145-148: Include status code in Response.Status string.

Status typically includes both code and text (e.g., "404 Not Found"). Current code sets only the text.

-func createErrorResponse(statusCode int, content string) *http.Response {
-    status := http.StatusText(statusCode)
-    return createMockHTTPResponse(status, statusCode, content, nil)
-}
+func createErrorResponse(statusCode int, content string) *http.Response {
+    status := fmt.Sprintf("%d %s", statusCode, http.StatusText(statusCode))
+    return createMockHTTPResponse(status, statusCode, content, nil)
+}

Note: would require fmt import.


150-165: Prefer t.Cleanup for restoring globals to be subtest/parallel-friendly.

Defer works, but t.Cleanup composes better with subtests and early returns.

 func withMockClient(t *testing.T, mockClient *mockHTTPClient, fn func()) {
   t.Helper()

   originalClient := DefaultClient
   DefaultClient = &Client{
     HTTPClient: mockClient,
     UserAgent:  "test-agent",
   }

-  defer func() {
-    DefaultClient = originalClient
-  }()
+  t.Cleanup(func() { DefaultClient = originalClient })

   fn()
 }
📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between b8490df and 9e92ae3.

⛔ Files ignored due to path filters (1)
  • go.sum is excluded by !**/*.sum
📒 Files selected for processing (17)
  • go.mod (1 hunks)
  • pkg/download/download.go (1 hunks)
  • vendor/github.com/cavaliergopher/grab/v3/pkg/bps/bps.go (0 hunks)
  • vendor/github.com/cavaliergopher/grab/v3/pkg/bps/sma.go (0 hunks)
  • vendor/github.com/sebrandon1/grab/lib/README.md (1 hunks)
  • vendor/github.com/sebrandon1/grab/lib/client.go (7 hunks)
  • vendor/github.com/sebrandon1/grab/lib/doc.go (2 hunks)
  • vendor/github.com/sebrandon1/grab/lib/download.go (1 hunks)
  • vendor/github.com/sebrandon1/grab/lib/error.go (1 hunks)
  • vendor/github.com/sebrandon1/grab/lib/grab.go (1 hunks)
  • vendor/github.com/sebrandon1/grab/lib/rate_limiter.go (1 hunks)
  • vendor/github.com/sebrandon1/grab/lib/request.go (1 hunks)
  • vendor/github.com/sebrandon1/grab/lib/response.go (5 hunks)
  • vendor/github.com/sebrandon1/grab/lib/test_helpers.go (1 hunks)
  • vendor/github.com/sebrandon1/grab/lib/transfer.go (2 hunks)
  • vendor/github.com/sebrandon1/grab/lib/util.go (1 hunks)
  • vendor/modules.txt (1 hunks)
💤 Files with no reviewable changes (2)
  • vendor/github.com/cavaliergopher/grab/v3/pkg/bps/bps.go
  • vendor/github.com/cavaliergopher/grab/v3/pkg/bps/sma.go
🚧 Files skipped from review as they are similar to previous changes (13)
  • vendor/github.com/sebrandon1/grab/lib/util.go
  • vendor/github.com/sebrandon1/grab/lib/request.go
  • vendor/github.com/sebrandon1/grab/lib/response.go
  • vendor/github.com/sebrandon1/grab/lib/transfer.go
  • vendor/github.com/sebrandon1/grab/lib/download.go
  • vendor/github.com/sebrandon1/grab/lib/doc.go
  • pkg/download/download.go
  • vendor/github.com/sebrandon1/grab/lib/rate_limiter.go
  • vendor/github.com/sebrandon1/grab/lib/error.go
  • vendor/github.com/sebrandon1/grab/lib/grab.go
  • vendor/github.com/sebrandon1/grab/lib/client.go
  • vendor/modules.txt
  • go.mod
🧰 Additional context used
🧬 Code graph analysis (1)
vendor/github.com/sebrandon1/grab/lib/test_helpers.go (1)
vendor/github.com/sebrandon1/grab/lib/client.go (3)
  • DefaultClient (63-63)
  • Client (30-47)
  • HTTPClient (17-19)
🪛 markdownlint-cli2 (0.17.2)
vendor/github.com/sebrandon1/grab/lib/README.md

41-41: Hard tabs
Column: 1

(MD010, no-hard-tabs)


42-42: Hard tabs
Column: 1

(MD010, no-hard-tabs)


43-43: Hard tabs
Column: 1

(MD010, no-hard-tabs)


47-47: Hard tabs
Column: 1

(MD010, no-hard-tabs)


48-48: Hard tabs
Column: 1

(MD010, no-hard-tabs)


49-49: Hard tabs
Column: 1

(MD010, no-hard-tabs)


50-50: Hard tabs
Column: 1

(MD010, no-hard-tabs)


51-51: Hard tabs
Column: 1

(MD010, no-hard-tabs)


52-52: Hard tabs
Column: 1

(MD010, no-hard-tabs)


53-53: Hard tabs
Column: 1

(MD010, no-hard-tabs)


54-54: Hard tabs
Column: 1

(MD010, no-hard-tabs)


55-55: Hard tabs
Column: 1

(MD010, no-hard-tabs)


56-56: Hard tabs
Column: 1

(MD010, no-hard-tabs)


57-57: Hard tabs
Column: 1

(MD010, no-hard-tabs)


58-58: Hard tabs
Column: 1

(MD010, no-hard-tabs)


68-68: Hard tabs
Column: 1

(MD010, no-hard-tabs)


69-69: Hard tabs
Column: 1

(MD010, no-hard-tabs)


73-73: Hard tabs
Column: 1

(MD010, no-hard-tabs)


74-74: Hard tabs
Column: 1

(MD010, no-hard-tabs)


75-75: Hard tabs
Column: 1

(MD010, no-hard-tabs)


76-76: Hard tabs
Column: 1

(MD010, no-hard-tabs)


77-77: Hard tabs
Column: 1

(MD010, no-hard-tabs)


78-78: Hard tabs
Column: 1

(MD010, no-hard-tabs)


79-79: Hard tabs
Column: 1

(MD010, no-hard-tabs)


80-80: Hard tabs
Column: 1

(MD010, no-hard-tabs)


81-81: Hard tabs
Column: 1

(MD010, no-hard-tabs)


82-82: Hard tabs
Column: 1

(MD010, no-hard-tabs)


83-83: Hard tabs
Column: 1

(MD010, no-hard-tabs)


84-84: Hard tabs
Column: 1

(MD010, no-hard-tabs)


85-85: Hard tabs
Column: 1

(MD010, no-hard-tabs)


86-86: Hard tabs
Column: 1

(MD010, no-hard-tabs)


87-87: Hard tabs
Column: 1

(MD010, no-hard-tabs)


88-88: Hard tabs
Column: 1

(MD010, no-hard-tabs)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (15)
  • GitHub Check: build-qe (darwin, arm64)
  • GitHub Check: build-qe (linux, amd64)
  • GitHub Check: build-qe (linux, arm64)
  • GitHub Check: build-installer (windows-2022, 1.24)
  • GitHub Check: build (windows-2022, 1.24)
  • GitHub Check: Run OKD bundle with crc (1.24)
  • GitHub Check: build (windows-2022, 1.24)
  • GitHub Check: build (macOS-14, 1.24)
  • GitHub Check: build (ubuntu-latest, 1.24)
  • GitHub Check: verify-devcontainer
  • GitHub Check: build (macOS-14, 1.24)
  • GitHub Check: build (ubuntu-latest, 1.24)
  • GitHub Check: build (macOS-13, 1.24)
  • GitHub Check: build (ubuntu-22.04, 1.24)
  • GitHub Check: build (macOS-13, 1.24)
🔇 Additional comments (1)
vendor/github.com/sebrandon1/grab/lib/README.md (1)

41-44: All legacy imports removed
No instances of github.com/cavaliergopher/grab (or /v3) remain in the repo after running the provided regex search.

Copy link

openshift-ci bot commented Sep 9, 2025

@sebrandon1: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/security 9e92ae3 link false /test security
ci/prow/e2e-crc 9e92ae3 link true /test e2e-crc

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants