Commit 80b923d7 authored by Toon Claes's avatar Toon Claes 🎨

Revert "Add script to fill event log gaps manually"

This reverts commit bdbe3dd6.
parent bdbe3dd6
Pipeline #89120 passed with stage
in 1 minute and 7 seconds
......@@ -337,8 +337,6 @@ state of the secondary to converge.
* You can use `sudo gitlab-rake geo:status` instead if the UI is non-compliant
* If failures appear, see Rails console commands to resync repos/wikis: https://gitlab.com/snippets/1713152
* On staging, this may not complete
* Fill event log gaps manually
* `/opt/gitlab-migration/migration/bin/scripts/02_failover/060_go/p03/010-fill-event-log-gaps.sh`
1. [ ] 🐺 {+ Coordinator +}: Wait for all repositories and wikis to become verified
* Staging: https://dashboards.gitlab.net/d/YoKVGxSmk/gcp-failover-gcp?orgId=1&var-environment=gstg
* Production: https://dashboards.gitlab.net/d/YoKVGxSmk/gcp-failover-gcp?orgId=1&var-environment=gprd
......@@ -351,7 +349,7 @@ state of the secondary to converge.
* In GCP: `Geo::EventLogState.last_processed.id`
* The two numbers should be the same
1. [ ] 🐺 {+ Coordinator +}: Ensure the prospective failover target in GCP is up to date
* `/opt/gitlab-migration/migration/bin/scripts/02_failover/060_go/p03/020-check-wal-secondary-sync.sh`
* `/opt/gitlab-migration/migration/bin/scripts/02_failover/060_go/p03/check-wal-secondary-sync.sh`
* Assuming the clocks are in sync, this value should be close to 0
* If this is a large number, GCP may not have some data that is in Azure
1. [ ] 🐺 {+ Coordinator +}: Now disable all sidekiq-cron jobs on the secondary
......
#!/usr/bin/gitlab-rails runner
GEO_EVENT_LOG_GAPS = 'geo:event_log:gaps'.freeze
def with_redis
::Gitlab::Redis::SharedState.with { |redis| yield redis }
end
# Get gaps tracked in redis
gap_ids = with_redis { |redis| redis.zrangebyscore(GEO_EVENT_LOG_GAPS, '-inf', '+inf', with_scores: true) }; nil
gap_ids.each { |gap| puts "Found event log gap with id #{gap.first} @ #{Time.now.to_i - gap.second} seconds ago" }; gap_ids.count
# Fetch the events from database, if they exist
gap_events = Geo::EventLog.where(id: gap_ids.map(&:first)); gap_events.count
# Use the daemon to process the gap events
daemon = Gitlab::Geo::LogCursor::Daemon.new; nil
gap_events.each { |gap_event| daemon.send(:handle_single_event, gap_event.first) }; nil
# Delete the gaps from redis
with_redis { |redis| redis.zrem(GEO_EVENT_LOG_GAPS, gap_events.map(&:id)) }
puts "Filled #{gap_events.count} gaps out of #{gaps_ids.count}"
#!/usr/bin/env bash
set -euo pipefail
IFS=$'\n\t'
SCRIPT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
UNSYMLINKED_SCRIPT_DIR="$(greadlink -f "${SCRIPT_DIR}" || readlink "${SCRIPT_DIR}" || echo "${SCRIPT_DIR}")"
# shellcheck disable=SC1091,SC1090
source "${UNSYMLINKED_SCRIPT_DIR}/../../../../workflow-script-commons.sh"
# --------------------------------------------------------------
if [[ ${FAILOVER_ENVIRONMENT} == "stg" ]]; then
DEPLOY_HOST=deploy.stg.gitlab.com
elif [[ ${FAILOVER_ENVIRONMENT} == "prd" ]]; then
DEPLOY_HOST=deploy.gitlab.com
fi
# remote_gitlab_runner will log the command internally
remote_gitlab_runner "${DEPLOY_HOST}" "${SCRIPT_DIR}/010-fill-event-log-gaps.rb"
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment