Do not clone repositories when there is no config and onboarding is disabled

[aentwist]

How are you running Renovate?

Self-hosted

If you're self-hosting Renovate, tell us what version of Renovate you run.

37.43.4

If you're self-hosting Renovate, select which platform you are using.

GitLab self-hosted

Was this something which used to work for you, and then stopped?

No

Wanted end result.

I am running renovate with autodiscover and without onboarding. I want it to only run on repos that are configured / enabled. I expect renovate to determine whether repos are enabled before starting work on them. I do not want it to clone repos that are not enabled as this is very expensive, but I am under the impression that is not currently the case. How can I set cloning to be lazy?

What you tried so far.

Having trouble finding info on this. Haven't tried anything, open to suggestions.

Relevant debug logs

For example

INFO: Repository is disabled - skipping (repository=andersonentwistle/sticky-scroll)
 INFO: Repository finished (repository=andersonentwistle/sticky-scroll)
       "cloned": true,
       "durationMs": 1066

There are many repos, and some take a full 2 minutes to be cloned.

[rarkins]

Renovate supports a lot of config file names, so you'd need to limit yourself to one (eg renovate.json) to avoid doing too many API calls. Then there could be a variant if optimizeForDisabled adapted for the case where onboarding=false

[aentwist]

I see... if onboarding is disabled and the config file doesn't exist but there is a config file by a different name, the repo would be skipped where it wasn't before. This wouldn't be the case if we read the tree with 1-2 requests and checked for all config files... so I guess that is actually the ask. That is not breaking... right? And if the breaking optimization were to be added after, then it would also not be breaking?

But that is a lot of work. So for now I guess the existing way to do this is to disable all in the global config and then reenable? If I understand that correctly then I will try that, because this will speed up my runs by an order of magnitude.

Just adding another config option is starting to sound pretty good 😅

---

By the way, this is a pretty weird piece of code

renovate/lib/workers/repository/init/apis.ts

Lines 47 to 51 in f4f7a89

	if (
	renovateConfig?.extends?.includes(':enableRenovate') ??
	renovateConfig?.ignorePresets?.includes(':disableRenovate') ??
	renovateConfig?.enabled
	) {

Note that if renovateConfig?.extends?.includes(':enableRenovate') ?? evaluates to false (renovate config exists and has extends but extends doesn't include enableRenovate) then the nullish operator will evaluate to the LHS (false is not null or undefined) without evaluating the other conditions. Same story for the next condition. Not sure if that is intentional but I'd be amazed if so. Hard to read / maintain - I literally cannot tell what is happening there. Relying hard on that comment and trusting the documentation.

---

If I get some spare time I can try to look into requesting git trees and matching for config files if you are interested. No guarantees though.

[aentwist]

@rarkins I played around with the GitLab GraphQL API for an hour and came up with this:

{
  project(fullPath: "andersonentwistle/vue-template") {
    repository {
      rootDir: tree {
        blobs {
          nodes {
            path
          }
        }
      }
      githubDir: tree(path: ".github") {
        blobs {
          nodes {
            path
          }
        }
      }
      gitlabDir: tree(path: ".gitlab") {
        blobs {
          nodes {
            path
          }
        }
      }
    }
  }
}

Which gives

{
  "data": {
    "project": {
      "repository": {
        "rootDir": {
          "blobs": {
            "nodes": [
              {
                "path": ".commitlintrc.json"
              },
              {
                "path": ".eslintrc.cjs"
              },
              {
                "path": ".gitattributes"
              },
              {
                "path": ".gitignore"
              },
              {
                "path": ".gitlab-ci.yml"
              },
              {
                "path": ".pre-commit-config.yaml"
              },
              {
                "path": ".prettierrc.json"
              },
              {
                "path": ".releaserc.json"
              },
              {
                "path": ".renovaterc.json"
              },
              {
                "path": ".tool-versions"
              },
              {
                "path": "CONTRIBUTING.md"
              },
              {
                "path": "README.md"
              },
              {
                "path": "env.d.ts"
              },
              {
                "path": "index.html"
              },
              {
                "path": "package-lock.json"
              },
              {
                "path": "package.json"
              },
              {
                "path": "template.md"
              },
              {
                "path": "tsconfig.app.json"
              },
              {
                "path": "tsconfig.config.json"
              },
              {
                "path": "tsconfig.json"
              },
              {
                "path": "tsconfig.vitest.json"
              },
              {
                "path": "vite.config.ts"
              }
            ]
          }
        },
        "githubDir": {
          "blobs": {
            "nodes": []
          }
        },
        "gitlabDir": {
          "blobs": {
            "nodes": []
          }
        }
      }
    }
  }
}

Here are some notes:

• I have intentionally chosen GraphQL over REST here for the powerful request multiple resources and filtering capabilities
• I understand this could be a large request and limit the scope/bandwidth to minimize it
  • I limit the tree requests to root, .github, and .gitlab because that is where the config files live
  • I limit the data to the relative path path. We cannot just use name because only specific config files can live in the .github and .gitlab folders
  • I have not found a way to tell them to do the search on their side
• Optimizing for package.json config sections would require special handling to check whether those sections actually exist within package.json files

So vendor specific, do this, loop and match path to ... hm, this https://regex101.com/r/Cujd5Z/1. What do you think?

[aentwist]

Here is a crappy WIP query for GitHub

{
  repository(owner: "aentwist", name: "pre-commit-mirrors-commitlint") {
    object(expression: "HEAD:") {
      ... on Tree {
        entries {
          path
        }
      }
    }
  }
}

https://docs.github.com/en/graphql/overview/explorer

[rarkins]

No GraphQL for this - it is much more heavily rate limited than REST

[aentwist]

Then it's just the same, except with 3 REST requests + over double the bandwidth on each. That is fine. I see there are nontrivial calculations to determine how much rate allowance each GitHub GQL request takes :( maybe GitLab is better for this, I'd hope.

[jamietanna]

optimizeForDisabled was added for this

[aentwist]

As discussed in great detail above, optimizeForDisabled did not cover it. Maybe it has since been updated.