All the implementations of DNS discovery were done in this epic:
https://gitlab.com/groups/gitlab-org/-/epics/8971. Gitaly allows clients
to configure DNS discovery via dial option. This MR adds the exposed
dial options to client connection creation in Gitlab-shell.

Issue: https://gitlab.com/gitlab-org/gitaly/-/issues/4722
Changelog: added

What
---
Make the retryableHTTP client introduced in
https://gitlab.com/gitlab-org/gitlab-shell/-/merge_requests/703 the
default HTTP client.

Why
---
In
https://gitlab.com/gitlab-com/gl-infra/production/-/issues/7979#note_1254964426
we've seen a 99% error reduction on `git` commands from `gitlab-shell`
when the retryableHTTP client is used.

This has been running in production for over 2 weeks in `us-east1-b` and
5 days fleet-wide so we should be confident that this client works as
expected.

Reference: https://gitlab.com/gitlab-com/gl-infra/production/-/issues/7979
Signed-off-by: Steve Azzopardi <sazzopardi@gitlab.com>

What
---
Change the default `HTTP.Client` to
`github.com/hashicorp/go-retryablehttp.Client` to get automatic retries
and exponential backoff.

We retry the request 2 times resulting in 3 attempts of sending the
request, the min retry wait is 1 second, and the maximum is 15
seconds.

Hide the retry logic behind a temporary feature flag
`FF_GITLAB_SHELL_RETRYABLE_HTTP` to easily roll this out in GitLab.com.
When we verify that this works as expected we will remove
`FF_GITLAB_SHELL_RETRYABLE_HTTP` and have the retry logic as the default
logic.

Why
---
In https://gitlab.com/gitlab-com/gl-infra/production/-/issues/7979 users
end up seeing the following errors when trying to `git-clone(1)` a
repository locally on in CI.

```shell
remote: ===============================
remote:
remote: ERROR: Internal API unreachable
remote:
remote: ================================
```

When we look at the application logs we see the following error:

```json
{ "err": "http://gitlab-webservice-git.gitlab.svc:8181/api/v4/internal/allowed":
dial tcp 10.69.184.120:8181: connect: connection refused", "msg":
"Internal API unreachable"}
```

In
https://gitlab.com/gitlab-com/gl-infra/production/-/issues/7979#note_1222670120
we've correlated these `connection refused` errors with infrastructure
events that remove the git pods that are hosting
`gitlab-webservice-git` service. We could try to make the underlying
infrastructure more reactive to these changes as suggested in
https://gitlab.com/gitlab-com/gl-infra/production/-/issues/7979#note_1225164944
but we can still end up serving bad requests.

Implementing retry logic for 5xx or other errors would allow users to
still be able to `git-clone(1)` reposirories, although it being slower.
This is espically important during CI runs so users don't have to retry
jobs themselves.

Reference: https://gitlab.com/gitlab-com/gl-infra/production/-/issues/7979
Closes: https://gitlab.com/gitlab-org/gitlab-shell/-/issues/604
Signed-off-by: Steve Azzopardi <sazzopardi@gitlab.com>

This commit also excludes gitlab-shell from dependencies:

Gitaly specifies Gitlab Shell as a dependency as well in order
to use gitlabnet client to perform API endpoints to Gitlab Rails.
As a result, Gitlab Shell requires Gitaly -> Gitaly requires an
older version of Gitlab Shell -> that version requires an older
version of Gitlab Shell, etc. Let's use exclude to break the
chain earlier

While gitlab-shell currently has a major version of v14, the module path
it exposes is not using that major version like it is required by the Go
standard. This makes it impossible for dependents to import gitlab-shell
as a dependency without using a commit as version.

Fix this by changing the module path of gitlab-shell to instead be
`gitlab.com/gitlab-org/gitlab-shell/v14` and adjust all imports
accordingly.

Changelog: fixed

If the GitLab API returns an allowed response with use_sidechannel set
to true, gitlab-shell will establish a sidechannel connection and use
SSHUploadPackWithSidechannel instead of SSHUploadPack. This is an
efficiency improvement.

This updates the Gitaly client go.mod dependency to Gitaly commit
2e398afa0490ccdf5a82e1a7c7d824ae491eba16. This causes a grpc-go
version bump, and hence a minor change in some of our test code.

Gitaly project now properly respects module release flow
and includes a module suffix in the package name. It requires
to re-write all non-suffixed imports with suffixed of a specific
version of tha module. With proper module versioning we don't
need to use a 'replace' directive to point to specific commit
and can use semantic versioning for the gitaly dependency.

Part of: https://gitlab.com/gitlab-org/gitaly/-/issues/3177