# Gitaly unusual activity alert

## First and foremost

*Don't Panic*

## Symptoms

* Alert on Slack: _Unusual Gitaly activity for a project has been detected. Review the runbook at https://gitlab.com/gitlab-com/runbooks/tree/master/troubleshooting/gitaly-unusual-activity.md for more details_

## 1. Review the suspicious activity

- **Check out the abuse dashboard**: https://dashboards.gitlab.net/d/9T-wXWbik/abuse-dashboard?orgId=1&panelId=2&from=now-1h&to=now
- **Review the abuse reporting data in Kibana**: https://log.gitlab.net/goto/6636a49add992f6326862df0afc6ae54
- **Review the abuse dashboard**:  https://log.gitlab.net/app/kibana#/dashboard/AWSIfVZhTIzC7JP6Xxn1
- Keep in mind that this is an open-ended alert, so it alerts to suspicious activity, rather than pin-pointing an issue.
- Use this as an informational alert, combine it with other signals

## 2. Evaluate impact

- If the affected Gitaly server is under load due to the activity this project is generating, consider disabling the project:

    1. **Archive the project** - this is especially useful if the project name or description itself contains links or reference to scams/spam/malware, as it de-lists the project from search as well.

    1. **Delete the project** - as far as we can tell, this is the only way to shut down a project that's publishing to gitlab pages.

- If the traffic is being generated by anonymous users accessing a public project, consider making the project private.

    1. We tend to do for people using GitLAb as a CDN, highly trafficked repos, etc.  This doesn't always help, at least one project has included authentication to access the private repos.

    1. Go through the UI: Settings -> General -> Permissions -> Project Visibility