Add cache built OpenSSL/wolfSSL deps in PR builders to speedup builds in CI#412
Add cache built OpenSSL/wolfSSL deps in PR builders to speedup builds in CI#412aidangarske wants to merge 1 commit into
Conversation
aidangarske
changed the title
padelsbach
left a comment
padelsbach
left a comment
There was a problem hiding this comment.
This is a very challenging nut to crack (I've tried). Lots of landmines lurking, even assuming GH cache behavior is stable.
Zooming out, we really only need to build openssl and wolfssl once per tag/commit. Could we have a job that builds and pushes to oras? Or maybe even bakes it into different docker containers?
| uses: actions/cache@v4 | ||
| with: | ||
| path: openssl-install | ||
| key: openssl-install-${{ inputs.variant }}-${{ steps.openssl-ref.outputs.sha }}-${{ hashFiles('scripts/utils-openssl.sh', 'scripts/utils-wolfssl.sh', 'scripts/build-wolfprovider.sh') }} |
There was a problem hiding this comment.
AI tells me we may want to include the patch files and patching scripts in the hashFiles
| GITHUB_TOKEN: ${{ inputs.github_token }} | ||
| run: echo "sha=$("${GITHUB_WORKSPACE}/scripts/resolve-ref.sh" "${{ inputs.wolfssl_ref }}" wolfssl/wolfssl)" >> "$GITHUB_OUTPUT" | ||
|
|
||
| - name: Cache OpenSSL install |
There was a problem hiding this comment.
Recommend redoing the openssl install if the openssl source is a cache miss, otherwise we might see unexpected stuff. Could happen with various cache eviction by Github.
Same with wolfssl source+install.
| path: openssl-source | ||
| key: openssl-source-${{ inputs.variant }}-${{ steps.openssl-ref.outputs.sha }}-${{ hashFiles('scripts/utils-openssl.sh', 'scripts/utils-wolfssl.sh', 'scripts/build-wolfprovider.sh') }} | ||
|
|
||
| - name: Cache wolfSSL install |
There was a problem hiding this comment.
AI pointed out that even wolfSSL depends on the openssl source (since we use --enable-opensslcoexist in utils-wolfssl.sh. So this step needs everything from the previous steps.
| variant: simple${{ matrix.replace_default != '' && '-rd' || '' }} | ||
| openssl_ref: ${{ matrix.openssl_ref }} | ||
| wolfssl_ref: ${{ matrix.wolfssl_ref }} | ||
| cache_openssl_source: ${{ matrix.replace_default != '' }} |
There was a problem hiding this comment.
IIRC, the WP build scripts will pull/download missing deps (eg openssl source) if not present, potentially bypassing the cache or causing a header-vs-binary mismatch. We really the workflow to fail if the cache step didn't work
3 participants
Description
Adds a reusable composite action
.github/actions/cache-build-depsthat cachesthe built
openssl-install/wolfssl-install(andopenssl-sourceforpatched builds) keyed by resolved commit SHA, so
build-wolfprovider.shskipsthe OpenSSL/wolfSSL configure+make on a cache hit. Wired into the normal-PR
from-source builders:
cmdline,simple,smoke-test,seed-src,fips-ready.Pattern mirrors the existing
multi-compiler.ymlcache (resolve-ref -> SHA ->actions/cache): a stable tag resolves to a fixed SHA (cached across runs);masterresolves to current HEAD (rebuilt only when upstream moves). The unitunder test (
wolfprov-install) is never cached.Measured effect (verified on this PR: cold run 1 -> warm run 2)
Per heavy config the from-scratch dep build is ~5:30-5:50; on a warm cache the
build step drops to ~30-50 s (about 5 min saved per cached job). All builders
passed on both the cold and warm runs.
Validation
masterrows hit on the resolved HEAD SHA("Cache hit ... Cache restored successfully"): master is reused when the commit
is unchanged and rebuilds when it moves; stable tags hit unconditionally.
simple.yml) also cacheopenssl-sourceso the source-reading mismatch checks pass on a hit.
replace-default/seed-src/fips) so no wrong-config cache hit.
wgetonly runs on a cache miss.