git error: RPC failed; HTTP 400 curl 22 The requested URL returned error: 400

[mtelvers]

Some job logs report RPC failed.

2024-05-16 06:53.54: Waiting for worker…
2024-05-16 06:53.54: Got resource from pool OCluster
Building on summer
error: RPC failed; HTTP 400 curl 22 The requested URL returned error: 400
fatal: unable to write request to remote: Broken pipe
git-fetch failed with exit-code 128
2024-05-16 06:53.58: Job failed: Failed: Build failed

On the FreeBSD system, in /var/log/messages, the failure relates to git fetch:

May 16 06:58:13 summer root[92057]: 2024-05-16 06:58.13         worker [INFO] Exec(git-fetch): "git" "-C"
May 16 06:58:13 summer root[92057]:                                           "/obuilder/state/git/opam-repository.git-b322f7b18d276db68487afc3dfc5a5befdd45818"
May 16 06:58:13 summer root[92057]:                                           "fetch" "-q" "--update-head-ok"
May 16 06:58:13 summer root[92057]:                                           "--recurse-submodules=no" "origin"
May 16 06:58:14 summer root[92057]: 2024-05-16 06:58.14         worker [INFO] Job failed: git-fetch failed with exit-code 128

Running git -C /obuilder/state/git/opam-repository.git-b322f7b18d276db68487afc3dfc5a5befdd45818 fetch -q --update-head-ok --recurse-submodules=no origin on the command line gives the same error.

Running git config --global http.postBuffer 524288000 as suggested on Stack overflow and then copying the .gitconfig to / seems to work around this issue. Copying to / is required as git is running as root without a $HOME environment variable.

[shonfeder]

Am I right in thinking that this is a configuration change we can make in our ansible configuration so it will be part of provisioning and configuration for all our machines?

[mtelvers]

It is. However, it only seems necessary on FreeBSD machines. I wonder whether the workaround is masking some other issue which we haven't yet identified. I would be reluctant to apply this across the whole infrastructure at this point.

[shonfeder]

Right. Good point. Echoing our sync conversation, it makes good sense o try to identify the root cause here before pushing out configuration changes to machines that may not need it.

Let's just be sure not to forget to add this to our absolve config if it turns out we need it as a permanent config change to any of the machines! :)