[RFE] provider build container image #74
Description
Problem
Friction in developing native providers is a blocker to growing beyond our 14 native providers in the registry today.
Affected area/feature
- Native Provider Development UX
Recommendation
Create a multi-arch containerized provider build environment to satisfy the many dependencies required to build multi language providers.
Future Opportunities
Adding a containerized build image simplifies future enhancements including:
- daily/weekly boilerplate repository ci testing
- simpler native provider developer docs
- improved provider build reliability
Additional Information
LOGS: Click To Expand
~/pulumi-provider-boilerplate ❯ docker build -t provider -f docker/Dockerfile .
[+] Building 1.1s (14/14) FINISHED
=> [internal] load .dockerignore 0.0s
=> => transferring context: 653B 0.0s
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 4.91kB 0.0s
=> [internal] load metadata for docker.io/library/ubuntu:22.04 1.1s
=> [auth] library/ubuntu:pull token for registry-1.docker.io 0.0s
=> [internal] load build context 0.0s
=> => transferring context: 742B 0.0s
=> [1/8] FROM docker.io/library/ubuntu:22.04@sha256:9b8dec3bf938bc80fbe758d856e96fdfab5f56c39d44b0cff351e847bb1b01ea 0.0s
=> CACHED [2/8] ADD docker/rootfs / 0.0s
=> CACHED [3/8] RUN set -ex && apt-get update && apt-get install git curl gnupg python3 python3-pip dotnet-sdk-7.0 ca-certificates build-essential dotnet-runtime-7.0 && update-alternatives --install /usr/bin/python python /usr/bin/python3 1 0.0s
=> CACHED [4/8] RUN set -ex && export arch=$(uname -m | awk '{print ($1 == "x86_64") ? "amd64" : ($1 == "aarch64" ? "arm64" : "unknown")}') && export goversion="$(curl -s https://go.dev/dl/?mode=json | awk -F'[":go]' '/ "version"/{print $8}' | head -n1)" 0.0s
=> CACHED [5/8] RUN set -ex && export NODE_MAJOR=20 && curl -fsSL https://deb.nodesource.com/gpgkey/nodesource-repo.gpg.key | gpg --dearmor -o /etc/apt/keyrings/nodesource.gpg && echo "deb [signed-by=/etc/apt/keyrings/nodesource.gpg] https://deb.nodesource.co 0.0s
=> CACHED [6/8] RUN set -ex && export arch=$(uname -m | awk '{print ($1 == "x86_64") ? "x64" : ($1 == "aarch64" ? "arm64" : "unknown")}') && export urlPulumiRelease="https://api.github.com/repos/pulumi/pulumi/releases/latest" && export urlPulumiVersion=$(curl -s ${ur 0.0s
=> CACHED [7/8] RUN set -ex && export arch=$(uname -m | awk '{print ($1 == "x86_64") ? "amd64" : ($1 == "aarch64" ? "arm64" : "unknown")}') && export urlPulumiRelease="https://api.github.com/repos/pulumi/pulumictl/releases/latest" && export urlPulumiVersion=$(curl -s 0.0s
=> CACHED [8/8] WORKDIR /provider 0.0s
=> exporting to image 0.0s
=> => exporting layers 0.0s
=> => writing image sha256:2375457dd3a19fbf32aa59226bfae190a621c4b15fa5c48fed8c659f14e6c7e9 0.0s
=> => naming to docker.io/library/provider:latest 0.0s
~/pulumi-provider-boilerplate ❯ docker run -it --rm -v $PWD:/provider provider
(cd provider && go build -o /provider/bin/pulumi-resource-xyz -ldflags "-X github.com/pulumi/pulumi-xyz/provider.Version=0.0.1-alpha.1697065379+97813b07.dirty" github.com/pulumi/pulumi-xyz/provider/cmd/pulumi-resource-xyz)
go: downloading github.com/pulumi/pulumi-go-provider v0.11.1
go: downloading github.com/pulumi/pulumi/sdk/v3 v3.79.0
go: downloading github.com/hashicorp/go-multierror v1.1.1
go: downloading github.com/pulumi/pulumi/pkg/v3 v3.79.0
go: downloading google.golang.org/grpc v1.57.0
go: downloading google.golang.org/protobuf v1.31.0
go: downloading github.com/hashicorp/errwrap v1.1.0
go: downloading github.com/golang/protobuf v1.5.3
go: downloading github.com/pkg/errors v0.9.1
go: downloading github.com/blang/semver v3.5.1+incompatible
go: downloading golang.org/x/net v0.14.0
go: downloading lukechampine.com/frand v1.4.2
go: downloading github.com/grpc-ecosystem/grpc-opentracing v0.0.0-20180507213350-8e809c8a8645
go: downloading github.com/hashicorp/hcl/v2 v2.18.0
go: downloading github.com/opentracing/opentracing-go v1.2.0
go: downloading github.com/rivo/uniseg v0.4.4
go: downloading github.com/spf13/cobra v1.7.0
go: downloading github.com/uber/jaeger-client-go v2.30.0+incompatible
go: downloading golang.org/x/term v0.11.0
go: downloading sourcegraph.com/sourcegraph/appdash v0.0.0-20211028080628-e2786a622600
go: downloading github.com/golang/glog v1.1.2
go: downloading github.com/pkg/term v1.1.0
go: downloading google.golang.org/genproto/googleapis/rpc v0.0.0-20230822172742-b8732ec3820d
go: downloading github.com/aead/chacha20 v0.0.0-20180709150244-8b13a72661da
go: downloading gopkg.in/yaml.v3 v3.0.1
go: downloading golang.org/x/crypto v0.12.0
go: downloading github.com/cheggaaa/pb v1.0.29
go: downloading github.com/djherbis/times v1.5.0
go: downloading github.com/go-git/go-git/v5 v5.8.1
go: downloading github.com/rogpeppe/go-internal v1.11.0
go: downloading github.com/santhosh-tekuri/jsonschema/v5 v5.3.1
go: downloading github.com/texttheater/golang-levenshtein v1.0.1
go: downloading github.com/tweekmonster/luser v0.0.0-20161003172636-3fa38070dbd7
go: downloading github.com/agext/levenshtein v1.2.3
go: downloading github.com/apparentlymart/go-textseg/v15 v15.0.0
go: downloading github.com/mitchellh/go-wordwrap v1.0.1
go: downloading github.com/zclconf/go-cty v1.14.0
go: downloading github.com/spf13/pflag v1.0.5
go: downloading golang.org/x/sys v0.11.0
go: downloading github.com/uber/jaeger-lib v2.4.1+incompatible
go: downloading go.uber.org/atomic v1.11.0
go: downloading github.com/gogo/protobuf v1.3.2
go: downloading github.com/opentracing/basictracer-go v1.1.0
go: downloading github.com/mattn/go-runewidth v0.0.15
go: downloading github.com/sabhiram/go-gitignore v0.0.0-20210923224102-525f6e181f06
go: downloading github.com/kevinburke/ssh_config v1.2.0
go: downloading dario.cat/mergo v1.0.0
go: downloading github.com/ProtonMail/go-crypto v0.0.0-20230828082145-3c4c8a2d2371
go: downloading github.com/go-git/go-billy/v5 v5.4.1
go: downloading github.com/sergi/go-diff v1.3.1
go: downloading github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da
go: downloading github.com/skeema/knownhosts v1.2.0
go: downloading github.com/xanzy/ssh-agent v0.3.3
go: downloading github.com/pjbgf/sha1cd v0.3.0
go: downloading github.com/go-git/gcfg v1.5.1-0.20230307220236-3a3c6141e376
go: downloading github.com/emirpasic/gods v1.18.1
go: downloading github.com/jbenet/go-context v0.0.0-20150711004518-d14ea06fba99
go: downloading golang.org/x/text v0.12.0
go: downloading github.com/cloudflare/circl v1.3.3
go: downloading gopkg.in/warnings.v0 v0.1.2
go: downloading github.com/pgavlin/goldmark v1.1.33-0.20200616210433-b5eb04559386
go: downloading github.com/natefinch/atomic v1.0.1
go: downloading github.com/edsrzf/mmap-go v1.1.0
go: downloading github.com/segmentio/encoding v0.3.6
go: downloading github.com/segmentio/asm v1.2.0
rm -rf sdk/dotnet
pulumi package gen-sdk /provider/bin/pulumi-resource-xyz --language dotnet
cd sdk/dotnet/&& \
echo "0.0.1-alpha.1697065379+97813b07.dirty" >version.txt && \
dotnet build /p:Version=0.0.1-alpha.1697065379+97813b07.dirty
Welcome to .NET 7.0!
---------------------
SDK Version: 7.0.112
----------------
Installed an ASP.NET Core HTTPS development certificate.
To trust the certificate run 'dotnet dev-certs https --trust' (Windows and macOS only).
Learn about HTTPS: https://aka.ms/dotnet-https
----------------
Write your first app: https://aka.ms/dotnet-hello-world
Find out what's new: https://aka.ms/dotnet-whats-new
Explore documentation: https://aka.ms/dotnet-docs
Report issues and find source on GitHub: https://github.com/dotnet/core
Use 'dotnet --help' to see available commands or visit: https://aka.ms/dotnet-cli
-----------------------------------------------------------------------------
MSBuild version 17.4.8+6918b863a for .NET
Determining projects to restore...
Restored /provider/sdk/dotnet/Pulumi.Xyz.csproj (in 3.81 sec).
Pulumi.Xyz -> /provider/sdk/dotnet/bin/Debug/net6.0/Pulumi.Xyz.dll
Successfully created package '/provider/sdk/dotnet/bin/Debug/Pulumi.Xyz.0.0.1-alpha.1697065379.nupkg'.
Build succeeded.
0 Warning(s)
0 Error(s)
Time Elapsed 00:00:06.02
rm -rf sdk/go
pulumi package gen-sdk /provider/bin/pulumi-resource-xyz --language go
rm -rf sdk/nodejs
pulumi package gen-sdk /provider/bin/pulumi-resource-xyz --language nodejs
cd sdk/nodejs/ && \
yarn install && \
yarn run tsc && \
cp ../../README.md ../../LICENSE package.json yarn.lock bin/ && \
sed -i.bak 's/${VERSION}/v0.0.1-alpha.1697065379+97813b07.dirty/g' bin/package.json && \
rm ./bin/package.json.bak
yarn install v1.22.19
warning package.json: No license field
info No lockfile found.
warning @pulumi/xyz@${VERSION}: No license field
[1/4] Resolving packages...
warning @pulumi/pulumi > [email protected]: The functionality that this package provided is now in @npmcli/arborist
warning @pulumi/pulumi > @opentelemetry/instrumentation > @opentelemetry/[email protected]: Please use @opentelemetry/api >= 1.3.0
warning @pulumi/pulumi > @opentelemetry/instrumentation-grpc > @opentelemetry/[email protected]: Please use @opentelemetry/api >= 1.3.0
warning @pulumi/pulumi > read-package-tree > [email protected]: This functionality has been moved to @npmcli/fs
warning @pulumi/pulumi > read-package-tree > readdir-scoped-modules > [email protected]: Package no longer supported. Contact Support at https://www.npmjs.com/support for more info.
[2/4] Fetching packages...
[3/4] Linking dependencies...
[4/4] Building fresh packages...
success Saved lockfile.
Done in 10.55s.
yarn run v1.22.19
warning package.json: No license field
$ /provider/sdk/nodejs/node_modules/.bin/tsc
Done in 1.82s.
rm -rf sdk/python
pulumi package gen-sdk /provider/bin/pulumi-resource-xyz --language python
cp README.md sdk/python/
cd sdk/python/ && \
python3 setup.py clean --all 2>/dev/null && \
rm -rf ./bin/ ../python.bin/ && cp -R . ../python.bin && mv ../python.bin ./bin && \
sed -i.bak -e 's/^VERSION = .*/VERSION = "0.0.1a1697065379+dirty"/g' -e 's/^PLUGIN_VERSION = .*/PLUGIN_VERSION = "0.0.1-alpha.1697065379+97813b07.dirty"/g' ./bin/setup.py && \
rm ./bin/setup.py.bak && \
cd ./bin && python3 setup.py build sdist
running clean
running build
running build_py
creating build
creating build/lib
creating build/lib/pulumi_xyz
copying pulumi_xyz/provider.py -> build/lib/pulumi_xyz
copying pulumi_xyz/__init__.py -> build/lib/pulumi_xyz
copying pulumi_xyz/random.py -> build/lib/pulumi_xyz
copying pulumi_xyz/_utilities.py -> build/lib/pulumi_xyz
copying pulumi_xyz/py.typed -> build/lib/pulumi_xyz
copying pulumi_xyz/pulumi-plugin.json -> build/lib/pulumi_xyz
running sdist
running egg_info
creating pulumi_xyz.egg-info
writing pulumi_xyz.egg-info/PKG-INFO
writing dependency_links to pulumi_xyz.egg-info/dependency_links.txt
writing requirements to pulumi_xyz.egg-info/requires.txt
writing top-level names to pulumi_xyz.egg-info/top_level.txt
writing manifest file 'pulumi_xyz.egg-info/SOURCES.txt'
reading manifest file 'pulumi_xyz.egg-info/SOURCES.txt'
writing manifest file 'pulumi_xyz.egg-info/SOURCES.txt'
running check
warning: check: missing required meta-data: url
warning: check: missing meta-data: either (author and author_email) or (maintainer and maintainer_email) should be supplied
creating pulumi_xyz-0.0.1a1697065379+dirty
creating pulumi_xyz-0.0.1a1697065379+dirty/pulumi_xyz
creating pulumi_xyz-0.0.1a1697065379+dirty/pulumi_xyz.egg-info
copying files to pulumi_xyz-0.0.1a1697065379+dirty...
copying README.md -> pulumi_xyz-0.0.1a1697065379+dirty
copying setup.py -> pulumi_xyz-0.0.1a1697065379+dirty
copying pulumi_xyz/__init__.py -> pulumi_xyz-0.0.1a1697065379+dirty/pulumi_xyz
copying pulumi_xyz/_utilities.py -> pulumi_xyz-0.0.1a1697065379+dirty/pulumi_xyz
copying pulumi_xyz/provider.py -> pulumi_xyz-0.0.1a1697065379+dirty/pulumi_xyz
copying pulumi_xyz/pulumi-plugin.json -> pulumi_xyz-0.0.1a1697065379+dirty/pulumi_xyz
copying pulumi_xyz/py.typed -> pulumi_xyz-0.0.1a1697065379+dirty/pulumi_xyz
copying pulumi_xyz/random.py -> pulumi_xyz-0.0.1a1697065379+dirty/pulumi_xyz
copying pulumi_xyz.egg-info/PKG-INFO -> pulumi_xyz-0.0.1a1697065379+dirty/pulumi_xyz.egg-info
copying pulumi_xyz.egg-info/SOURCES.txt -> pulumi_xyz-0.0.1a1697065379+dirty/pulumi_xyz.egg-info
copying pulumi_xyz.egg-info/dependency_links.txt -> pulumi_xyz-0.0.1a1697065379+dirty/pulumi_xyz.egg-info
copying pulumi_xyz.egg-info/not-zip-safe -> pulumi_xyz-0.0.1a1697065379+dirty/pulumi_xyz.egg-info
copying pulumi_xyz.egg-info/requires.txt -> pulumi_xyz-0.0.1a1697065379+dirty/pulumi_xyz.egg-info
copying pulumi_xyz.egg-info/top_level.txt -> pulumi_xyz-0.0.1a1697065379+dirty/pulumi_xyz.egg-info
Writing pulumi_xyz-0.0.1a1697065379+dirty/setup.cfg
creating dist
Creating tar archive
removing 'pulumi_xyz-0.0.1a1697065379+dirty' (and everything under it)