Skip to content

Conversation

markphelps
Copy link
Collaborator

@markphelps markphelps commented Aug 20, 2025

Summary

This PR fixes an issue where local filesystem git repository initialization would either fail with "repository does not exist" or cause "reference not found" errors. The solution uses a hybrid approach that selects the most appropriate git operation based on the directory state.

The Problem

The original code used git.Open(storage, worktree) for all cases, which failed for existing filesystem repositories because it expected pre-initialized storage with git data. Later attempts with PlainInit caused "reference not found" errors because the remote tracking references weren't properly set up for the environments system.

The Solution: Hybrid Approach

Logic Flow for Local Path Storage

  1. If .git directory exists (existing repository):

    • Uses git.PlainOpen to open the existing repository
    • PlainOpen handles filesystem repositories correctly and preserves all existing git state
    • ✅ Works correctly
  2. If directory has files but no .git directory:

    • Uses git.PlainInit to create a new repository
    • PlainInit properly handles existing files (leaves them untracked)
    • Creates the .git directory structure correctly
    • ✅ Works correctly and preserves existing files
  3. If directory is empty or doesn't exist:

    • Uses git.Init with filesystem storage
    • This approach maintains proper remote tracking references for the environments system
    • ✅ Works correctly and integrates with remote features

Key Changes

  1. Hybrid approach based on directory state:

    • Detects whether .git directory exists
    • Uses the most appropriate git operation for each scenario
  2. PlainOpen for existing repositories:

    • Handles filesystem repositories better than git.Open with storage
    • Preserves all existing git history and configuration
  3. PlainInit for directories with existing files:

    • Creates repository while preserving user files
    • Files remain untracked until Flipt explicitly modifies them
  4. git.Init for empty directories:

    • Maintains compatibility with remote tracking and environments system
    • Sets up proper reference structure that other parts of Flipt expect
  5. Preserve existing README.md files:

    • Added check to prevent overwriting user's existing README.md
    • Only creates README.md if it doesn't already exist

Why This Hybrid Approach Is Correct

  • PlainOpen vs git.Open: PlainOpen is designed specifically for filesystem repositories and handles all the storage setup internally
  • PlainInit benefits: Properly creates .git directory and handles existing files safely
  • git.Init benefits: Maintains remote tracking references that the environments system requires
  • Safety: Only files that Flipt creates or modifies get tracked in git, preventing accidental commits of user files

Fixes Both Issues

"repository does not exist" error: Fixed by using PlainOpen for existing repos
"reference not found" error: Fixed by using git.Init for empty directories to maintain proper reference structure
File preservation: All approaches preserve existing user files safely

Testing

Added comprehensive tests covering all scenarios:

  • Empty directory → creates new repo with README (git.Init path)
  • Existing git repository → opens and uses it correctly (PlainOpen path)
  • Directory with files but no git → initializes repo, preserves files (PlainInit path)
  • Existing README.md → preserves user's content, doesn't overwrite

Best Practices Followed

✅ Use the right git operation for each specific scenario
✅ Don't auto-track existing user files
✅ Only track files that Flipt explicitly creates/modifies
✅ Preserve existing user content (like README.md)
✅ Maintain compatibility with existing remote/environments features
✅ Handle all three scenarios without errors

The hybrid approach ensures Flipt works reliably with local filesystem storage in all scenarios while being safe about user data and maintaining system compatibility.

@markphelps markphelps requested a review from a team as a code owner August 20, 2025 17:49
@markphelps markphelps added the v2 Flipt v2 label Aug 20, 2025
@dosubot dosubot bot added the size:L This PR changes 100-499 lines, ignoring generated files. label Aug 20, 2025
Copy link

codecov bot commented Aug 20, 2025

Codecov Report

❌ Patch coverage is 58.33333% with 15 lines in your changes missing coverage. Please review.
✅ Project coverage is 64.87%. Comparing base (6c44f8f) to head (97ed1fe).
⚠️ Report is 2 commits behind head on v2.

Files with missing lines Patch % Lines
internal/storage/git/repository.go 58.33% 11 Missing and 4 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##               v2    #4588      +/-   ##
==========================================
+ Coverage   64.80%   64.87%   +0.06%     
==========================================
  Files         131      131              
  Lines       14776    14797      +21     
==========================================
+ Hits         9576     9599      +23     
+ Misses       4549     4545       -4     
- Partials      651      653       +2     
Flag Coverage Δ
integrationtests 38.61% <36.11%> (+0.19%) ⬆️
unittests 54.51% <58.33%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Fixes local storage backend compatibility with go-git v6 by switching
from git.Open(storage, nil) to git.PlainOpen(path) when opening existing
local Git repositories.

The issue occurred because go-git v6 changed the API and git.Open() with
a nil worktree parameter no longer works properly for filesystem repositories.
git.PlainOpen() is the recommended approach for opening repositories from
a filesystem path.

This fix allows Flipt v2 to properly work with the local storage backend
in Docker and other deployment scenarios where users mount existing Git
repositories.

Changes:
- Replace git.Open(storage, nil) with git.PlainOpen(r.localPath) for existing repos
- Add comprehensive unit tests covering the fixed scenario
- Test specifically validates the "repository does not exist" error was resolved

Fixes the error: "initializing environment store: environment 'default': repository does not exist"

Signed-off-by: Mark Phelps <[email protected]>
Extends the PlainOpen fix to automatically initialize Git repositories
when directories contain files but no .git directory. This makes Flipt
more user-friendly by eliminating the need for manual Git setup.

Features:
- Automatically calls git.PlainInit() when git.PlainOpen() fails
- Preserves all existing files and directory structure
- Provides clear debug logging for the auto-initialization process
- Graceful error handling with descriptive error messages

Testing:
- Added comprehensive unit tests using t.Cleanup
- Tests verify file preservation during auto-initialization
- Tests verify error handling for invalid paths
- All existing functionality continues to work

This enhancement allows users to mount any directory with flag files
and Flipt will automatically set up Git tracking without manual
repository initialization.

Signed-off-by: Mark Phelps <[email protected]>
- Change file permissions from 0644 to 0600 to satisfy gosec
- Use require.Error instead of assert.Error for testifylint
- Use require.NoError instead of assert.NoError for critical path checks

Signed-off-by: Mark Phelps <[email protected]>
Signed-off-by: Mark Phelps <[email protected]>
@markphelps markphelps force-pushed the fix/local-storage-plainopen branch from 1ca92cf to 457cc7c Compare August 20, 2025 18:55
@markphelps markphelps requested review from erka and Copilot August 20, 2025 19:13
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR fixes the local storage backend compatibility with go-git v6 by replacing the deprecated git.Open() method with the recommended git.PlainOpen() for filesystem repositories. It also enhances the user experience by automatically initializing Git repositories for directories containing files but lacking a .git directory.

  • Replaced git.Open(storage, nil) with git.PlainOpen(r.localPath) for existing repositories
  • Added automatic Git repository initialization when no .git directory exists
  • Enhanced error handling with descriptive messages and graceful fallback behavior

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

File Description
internal/storage/git/repository.go Core fix to use PlainOpen instead of git.Open and add auto-initialization fallback
internal/storage/git/repository_test.go Comprehensive test coverage for all scenarios including existing repos and auto-initialization

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.


// Create a test file and commit it to make it a "real" repository
testFile := filepath.Join(tempDir, "features.yml")
err = os.WriteFile(testFile, []byte("namespace: default\nflags: []\n"), 0600)
Copy link
Preview

Copilot AI Aug 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] The file is created with restrictive permissions (0600) but this may be inconsistent with typical YAML file permissions. Consider using 0644 for better compatibility unless security requirements specifically mandate 0600.

Suggested change
err = os.WriteFile(testFile, []byte("namespace: default\nflags: []\n"), 0600)
err = os.WriteFile(testFile, []byte("namespace: default\nflags: []\n"), 0644)

Copilot uses AI. Check for mistakes.


configFile := filepath.Join(tempDir, "config.yml")
configContent := "version: v1\nnamespace: default\n"
err = os.WriteFile(configFile, []byte(configContent), 0600)
Copy link
Preview

Copilot AI Aug 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] The file permissions 0600 are restrictive and may not reflect typical usage patterns. Consider using 0644 for consistency with standard file permissions unless there's a specific security requirement.

Suggested change
err = os.WriteFile(configFile, []byte(configContent), 0600)
err = os.WriteFile(flagsFile, []byte(flagsContent), 0644)
require.NoError(t, err)
configFile := filepath.Join(tempDir, "config.yml")
configContent := "version: v1\nnamespace: default\n"
err = os.WriteFile(configFile, []byte(configContent), 0644)

Copilot uses AI. Check for mistakes.


configFile := filepath.Join(tempDir, "config.yml")
configContent := "version: v1\nnamespace: default\n"
err = os.WriteFile(configFile, []byte(configContent), 0600)
Copy link
Preview

Copilot AI Aug 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] The file permissions 0600 are restrictive and may not reflect typical usage patterns. Consider using 0644 for consistency with standard file permissions unless there's a specific security requirement.

Suggested change
err = os.WriteFile(configFile, []byte(configContent), 0600)
err = os.WriteFile(flagsFile, []byte(flagsContent), 0644)
require.NoError(t, err)
configFile := filepath.Join(tempDir, "config.yml")
configContent := "version: v1\nnamespace: default\n"
err = os.WriteFile(configFile, []byte(configContent), 0644)

Copilot uses AI. Check for mistakes.


segmentFile := filepath.Join(subDir, "segments.yml")
segmentContent := "namespace: default\nsegments: []\n"
err = os.WriteFile(segmentFile, []byte(segmentContent), 0600)
Copy link
Preview

Copilot AI Aug 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] The file permissions 0600 are restrictive and may not reflect typical usage patterns. Consider using 0644 for consistency with standard file permissions unless there's a specific security requirement.

Suggested change
err = os.WriteFile(segmentFile, []byte(segmentContent), 0600)
err = os.WriteFile(segmentFile, []byte(segmentContent), 0644)

Copilot uses AI. Check for mistakes.

…preservation

- Use PlainOpen for existing git repositories (more reliable than git.Open)
- Fall back to PlainInit for directories with files but no git repo
- Preserve existing README.md files instead of overwriting them
- Add comprehensive tests for all initialization scenarios
- Ensure existing user files remain untracked (safe default)

The fix ensures Flipt can work with:
1. Empty directories (creates new repo with README)
2. Existing git repositories (opens and uses them)
3. Directories with files but no git (initializes repo, preserves files)
…tion

- Use PlainOpen for existing repositories (.git directory exists)
- Use PlainInit for directories with files but no .git (preserves files)
- Use git.Init with storage for empty directories (maintains remote references)

This approach fixes the "reference not found" error while maintaining
compatibility with all three scenarios:

1. Empty directory → git.Init creates proper repository structure
2. Existing git repo → PlainOpen handles filesystem repos correctly
3. Files but no .git → PlainInit preserves existing files

The hybrid approach ensures remote tracking references are properly
created while using the most appropriate git operation for each scenario.
…pen/git.Init

Use a simple two-step approach:
1. Try git.Open for existing repositories
2. Fall back to git.Init for new repositories (ensure .git dir exists first)

This approach:
- Uses consistent storage abstraction for both cases
- Handles all scenarios: empty dirs, existing repos, dirs with files
- Maintains proper remote tracking references
- Preserves existing files (they remain untracked)
- Much simpler than mixing multiple git library approaches

Fixes both "repository does not exist" and "reference not found" errors.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
size:L This PR changes 100-499 lines, ignoring generated files. v2 Flipt v2
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

1 participant