GitLab

Upgrade Sidekiq and Redis gems

See merge request gitlab-org/gitlab-exporter!154

Upgrades the gems to match those used in the gitlab-org/gitlab repo.

Add query to count registry online GC tasks that are overdue

See merge request gitlab-org/gitlab-exporter!153

Add a query to count rows in the registry namespaces table

See merge request gitlab-org/gitlab-exporter!152

Add registry-specific database queries

See merge request gitlab-org/gitlab-exporter!151

Update Ruby to 2.7.4

See merge request gitlab-org/gitlab-exporter!150

Add Ruby 3 build

Closes #84

See merge request gitlab-org/gitlab-exporter!148

Adds probe_future_sets to report on delays in scheduled and retry sets

See merge request gitlab-org/gitlab-exporter!147

A long description is available at https://gitlab.com/gitlab-com/gl-infra/scalability/-/issues/1171 but in short, this now reports how far "behind" Sidekiq is in processing the Scheduled or Retry sets; jobs in those sets are expected to normally be processed within a short time (seconds, not minutes) of their nominal scheduled time, but in bad times we can sometimes get behind, and being able to observe the magnitude of the backlog is desirable.

Adds both a delay time metric (how old is the next job that will be processed) and a backlog count (how many jobs are pending)

Remove final mentions of the LUA script

See merge request gitlab-org/gitlab-exporter!146

This also means we need to change the basic 'is this thing on?' connection check.  PING is the usual suspect.  Note that this is *not* an 'adminstrative' command in the context of 'redis-namespace', so we can go direct to 'conn' and remove the scary comment

Remove Sidekiq `probe_jobs` probe

See merge request gitlab-org/gitlab-exporter!145

This was generally fast but had two unfortunate effects that made it
impractical in production:

1. The script got slower the larger the queue, as it scanned all jobs in
   queue.
2. Because it was a Lua script, it blocked Redis execution entirely
   while running.

The combination of these two can lead to a catastrophic case. When a
queue is very long, that's precisely the time when we want jobs to be
processed quickly. But we'd see that jobs were blocked by the metrics
script, which is the opposite of what we need at that time.

Add probe_jobs_limit probe for Sidekiq

See merge request gitlab-org/gitlab-exporter!143

With large job sizes, we can spend a lot of time in both Ruby and Redis
getting 10,000 jobs. 1,000 gives us an order of magnitude of headroom.

This does the same as #probe_jobs, but only looks at the first
PROBE_JOBS_LIMIT jobs in each queue. This means that we run a single
LRANGE command for each queue, which does not block other commands. For
queues over PROBE_JOBS_LIMIT in size, this means that we will not have
completely accurate statistics, but the probe performance will also not
degrade as the queue gets larger.

DO NOT USE this and probe_jobs together, as they export the same
metric (sidekiq_enqueued_jobs).

Add explicit dast stage

See merge request gitlab-org/gitlab-exporter!144

Remove Unicorn references

Closes #81

See merge request gitlab-org/gitlab-exporter!142

Pin gem dependencies

Closes #82

See merge request gitlab-org/gitlab-exporter!141

So that we can more selectively update them
in the future.

Bump Puma to 5.3.2

See merge request gitlab-org/gitlab-exporter!140