summaryrefslogtreecommitdiff
path: root/doc/administration/monitoring
diff options
context:
space:
mode:
Diffstat (limited to 'doc/administration/monitoring')
-rw-r--r--doc/administration/monitoring/gitlab_instance_administration_project/index.md36
-rw-r--r--doc/administration/monitoring/index.md3
-rw-r--r--doc/administration/monitoring/ip_whitelist.md20
-rw-r--r--doc/administration/monitoring/performance/gitlab_configuration.md6
-rw-r--r--doc/administration/monitoring/performance/grafana_configuration.md49
-rw-r--r--doc/administration/monitoring/performance/img/performance_bar.pngbin99331 -> 33642 bytes
-rw-r--r--doc/administration/monitoring/performance/img/performance_bar_gitaly_calls.pngbin91275 -> 83212 bytes
-rw-r--r--doc/administration/monitoring/performance/img/performance_bar_line_profiling.pngbin93063 -> 0 bytes
-rw-r--r--doc/administration/monitoring/performance/img/performance_bar_redis_calls.pngbin0 -> 70859 bytes
-rw-r--r--doc/administration/monitoring/performance/img/performance_bar_rugged_calls.pngbin0 -> 105305 bytes
-rw-r--r--doc/administration/monitoring/performance/img/performance_bar_sql_queries.pngbin128337 -> 143074 bytes
-rw-r--r--doc/administration/monitoring/performance/img/request_profile_result.pngbin3216 -> 11451 bytes
-rw-r--r--doc/administration/monitoring/performance/influxdb_configuration.md2
-rw-r--r--doc/administration/monitoring/performance/performance_bar.md15
-rw-r--r--doc/administration/monitoring/performance/request_profiling.md4
-rw-r--r--doc/administration/monitoring/prometheus/gitlab_metrics.md10
-rw-r--r--doc/administration/monitoring/prometheus/gitlab_monitor_exporter.md6
-rw-r--r--doc/administration/monitoring/prometheus/index.md142
-rw-r--r--doc/administration/monitoring/prometheus/node_exporter.md6
-rw-r--r--doc/administration/monitoring/prometheus/pgbouncer_exporter.md6
-rw-r--r--doc/administration/monitoring/prometheus/postgres_exporter.md6
-rw-r--r--doc/administration/monitoring/prometheus/redis_exporter.md6
22 files changed, 216 insertions, 101 deletions
diff --git a/doc/administration/monitoring/gitlab_instance_administration_project/index.md b/doc/administration/monitoring/gitlab_instance_administration_project/index.md
new file mode 100644
index 00000000000..8e33cea6217
--- /dev/null
+++ b/doc/administration/monitoring/gitlab_instance_administration_project/index.md
@@ -0,0 +1,36 @@
+# GitLab instance administration project
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/56883) in GitLab 12.2.
+
+GitLab has been adding the ability for administrators to see insights into the health of
+their GitLab instance. In order to surface this experience in a native way, similar to how
+you would interact with an application deployed via GitLab, a base project called
+"GitLab Instance Administration" with
+[internal visibility](../../../public_access/public_access.md#internal-projects) will be
+added under a group called "GitLab Instance Administrators" specifically created for
+visualizing and configuring the monitoring of your GitLab instance.
+
+All administrators at the time of creation of the project and group will be added
+as maintainers of the group and project, and as an admin, you'll be able to add new
+members to the group in order to give them maintainer access to the project.
+
+This project will be used for self-monitoring your GitLab instance.
+
+## Connection to Prometheus
+
+The project will be automatically configured to connect to the
+[internal Prometheus](../prometheus/index.md) instance if the Prometheus
+instance is present (should be the case if GitLab was installed via Omnibus
+and you haven't disabled it).
+
+If that's not the case or if you have an external Prometheus instance or an HA setup,
+you should
+[configure it manually](../../../user/project/integrations/prometheus.md#manual-configuration-of-prometheus).
+
+## Taking action on Prometheus alerts **[ULTIMATE]**
+
+You can [add a webhook](../../../user/project/integrations/prometheus.md#external-prometheus-instances)
+to the Prometheus config in order for GitLab to receive notifications of any alerts.
+
+Once the webhook is setup, you can
+[take action on incoming alerts](../../../user/project/integrations/prometheus.md#taking-action-on-incidents-ultimate).
diff --git a/doc/administration/monitoring/index.md b/doc/administration/monitoring/index.md
index fa0459b24ff..2b3daec42bd 100644
--- a/doc/administration/monitoring/index.md
+++ b/doc/administration/monitoring/index.md
@@ -2,6 +2,9 @@
Explore our features to monitor your GitLab instance:
+- [GitLab self-monitoring](gitlab_instance_administration_project/index.md): The
+ GitLab instance administration project helps to monitor the GitLab instance and
+ take action on alerts.
- [Performance monitoring](performance/index.md): GitLab Performance Monitoring makes it possible to measure a wide variety of statistics of your instance.
- [Prometheus](prometheus/index.md): Prometheus is a powerful time-series monitoring service, providing a flexible platform for monitoring GitLab and other software products.
- [GitHub imports](github_imports.md): Monitor the health and progress of GitLab's GitHub importer with various Prometheus metrics.
diff --git a/doc/administration/monitoring/ip_whitelist.md b/doc/administration/monitoring/ip_whitelist.md
index ad2773de132..6bb2fe81b2c 100644
--- a/doc/administration/monitoring/ip_whitelist.md
+++ b/doc/administration/monitoring/ip_whitelist.md
@@ -12,9 +12,9 @@ hosts or use IP ranges:
1. Open `/etc/gitlab/gitlab.rb` and add or uncomment the following:
- ```ruby
- gitlab_rails['monitoring_whitelist'] = ['127.0.0.0/8', '192.168.0.1']
- ```
+ ```ruby
+ gitlab_rails['monitoring_whitelist'] = ['127.0.0.0/8', '192.168.0.1']
+ ```
1. Save the file and [reconfigure] GitLab for the changes to take effect.
@@ -24,13 +24,13 @@ hosts or use IP ranges:
1. Edit `config/gitlab.yml`:
- ```yaml
- monitoring:
- # by default only local IPs are allowed to access monitoring resources
- ip_whitelist:
- - 127.0.0.0/8
- - 192.168.0.1
- ```
+ ```yaml
+ monitoring:
+ # by default only local IPs are allowed to access monitoring resources
+ ip_whitelist:
+ - 127.0.0.0/8
+ - 192.168.0.1
+ ```
1. Save the file and [restart] GitLab for the changes to take effect.
diff --git a/doc/administration/monitoring/performance/gitlab_configuration.md b/doc/administration/monitoring/performance/gitlab_configuration.md
index 771584268d9..9a25cc04ee7 100644
--- a/doc/administration/monitoring/performance/gitlab_configuration.md
+++ b/doc/administration/monitoring/performance/gitlab_configuration.md
@@ -8,12 +8,8 @@ The minimum required settings you need to set are the InfluxDB host and port.
Make sure _Enable InfluxDB Metrics_ is checked and hit **Save** to save the
changes.
----
-
![GitLab Performance Monitoring Admin Settings](img/metrics_gitlab_configuration_settings.png)
----
-
Finally, a restart of all GitLab processes is required for the changes to take
effect:
@@ -30,8 +26,6 @@ sudo service gitlab restart
When any migrations are pending, the metrics are disabled until the migrations
have been performed.
----
-
Read more on:
- [Introduction to GitLab Performance Monitoring](introduction.md)
diff --git a/doc/administration/monitoring/performance/grafana_configuration.md b/doc/administration/monitoring/performance/grafana_configuration.md
index 187fb2f73a1..95be0d5fd88 100644
--- a/doc/administration/monitoring/performance/grafana_configuration.md
+++ b/doc/administration/monitoring/performance/grafana_configuration.md
@@ -1,6 +1,6 @@
# Grafana Configuration
-[Grafana](http://grafana.org/) is a tool that allows you to visualize time
+[Grafana](https://grafana.com/) is a tool that allows you to visualize time
series metrics through graphs and dashboards. It supports several backend
data stores, including InfluxDB. GitLab writes performance data to InfluxDB
and Grafana will allow you to query to display useful graphs.
@@ -13,7 +13,7 @@ services.
[GitLab Omnibus can help you install Grafana (recommended)](https://docs.gitlab.com/omnibus/settings/grafana.html)
or Grafana supplies package repositories (Yum/Apt) for easy installation.
-See [Grafana installation documentation](http://docs.grafana.org/installation/)
+See [Grafana installation documentation](https://grafana.com/docs/installation/)
for detailed steps.
NOTE: **Note:**
@@ -103,6 +103,51 @@ repository for more information on this process.
[grafana-dashboards]: https://gitlab.com/gitlab-org/grafana-dashboards
+## Integration with GitLab UI
+
+> [Introduced](https://gitlab.com/gitlab-org/gitlab-ce/issues/61005) in GitLab 12.1.
+
+If you have set up Grafana, you can enable a link to access it easily from the sidebar:
+
+1. Go to the admin area under **Settings > Metrics and profiling**
+ and expand "Metrics - Grafana".
+1. Check the "Enable access to Grafana" checkbox.
+1. If Grafana is enabled through Omnibus GitLab and on the same server,
+ leave "Grafana URL" unchanged. In any other case, enter the full URL
+ path of the Grafana instance.
+1. Click **Save changes**.
+1. The new link will be available in the admin area under **Monitoring > Metrics Dashboard**.
+
+## Security Update
+
+Users running GitLab version 12.0 or later should immediately upgrade to one of the following security releases due to a known vulnerability with the embedded Grafana dashboard:
+
+- 12.0.6
+- 12.1.6
+
+After upgrading, the Grafana dashboard will be disabled and the location of your existing Grafana data will be changed from `/var/opt/gitlab/grafana/data/` to `/var/opt/gitlab/grafana/data.bak.#{Date.today}/`.
+
+To prevent the data from being relocated, you can run the following command prior to upgrading:
+
+```sh
+echo "0" > /var/opt/gitlab/grafana/CVE_reset_status
+```
+
+To reinstate your old data, move it back into its original location:
+
+```
+sudo mv /var/opt/gitlab/grafana/data.bak.xxxx/ /var/opt/gitlab/grafana/data/
+```
+
+However, you should **not** reinstate your old data _except_ under one of the following conditions:
+
+1. If you are certain that you changed your default admin password when you enabled Grafana
+1. If you run GitLab in a private network, accessed only by trusted users, and your Grafana login page has not been exposed to the internet
+
+If you require access to your old Grafana data but do not meet one of these criteria, you may consider reinstating it temporarily, [exporting the dashboards](https://grafana.com/docs/reference/export_import/#exporting-a-dashboard) you need, then refreshing the data and [re-importing your dashboards](https://grafana.com/docs/reference/export_import/#importing-a-dashboard). Note that this poses a temporary vulnerability while your old Grafana data is in use, and the decision to do so should be weighed carefully with your need to access existing data and dashboards.
+
+For more information and further mitigation details, please refer to our [blog post on the security release](https://about.gitlab.com/2019/08/12/critical-security-release-gitlab-12-dot-1-dot-6-released/).
+
---
Read more on:
diff --git a/doc/administration/monitoring/performance/img/performance_bar.png b/doc/administration/monitoring/performance/img/performance_bar.png
index 2bf686f9017..d1187fd879a 100644
--- a/doc/administration/monitoring/performance/img/performance_bar.png
+++ b/doc/administration/monitoring/performance/img/performance_bar.png
Binary files differ
diff --git a/doc/administration/monitoring/performance/img/performance_bar_gitaly_calls.png b/doc/administration/monitoring/performance/img/performance_bar_gitaly_calls.png
index 7af6d401d1d..2c43201cbd0 100644
--- a/doc/administration/monitoring/performance/img/performance_bar_gitaly_calls.png
+++ b/doc/administration/monitoring/performance/img/performance_bar_gitaly_calls.png
Binary files differ
diff --git a/doc/administration/monitoring/performance/img/performance_bar_line_profiling.png b/doc/administration/monitoring/performance/img/performance_bar_line_profiling.png
deleted file mode 100644
index a55ce753101..00000000000
--- a/doc/administration/monitoring/performance/img/performance_bar_line_profiling.png
+++ /dev/null
Binary files differ
diff --git a/doc/administration/monitoring/performance/img/performance_bar_redis_calls.png b/doc/administration/monitoring/performance/img/performance_bar_redis_calls.png
new file mode 100644
index 00000000000..ecb2dffbf8d
--- /dev/null
+++ b/doc/administration/monitoring/performance/img/performance_bar_redis_calls.png
Binary files differ
diff --git a/doc/administration/monitoring/performance/img/performance_bar_rugged_calls.png b/doc/administration/monitoring/performance/img/performance_bar_rugged_calls.png
new file mode 100644
index 00000000000..210f80a713d
--- /dev/null
+++ b/doc/administration/monitoring/performance/img/performance_bar_rugged_calls.png
Binary files differ
diff --git a/doc/administration/monitoring/performance/img/performance_bar_sql_queries.png b/doc/administration/monitoring/performance/img/performance_bar_sql_queries.png
index b3219b4fa94..6b571f4e85c 100644
--- a/doc/administration/monitoring/performance/img/performance_bar_sql_queries.png
+++ b/doc/administration/monitoring/performance/img/performance_bar_sql_queries.png
Binary files differ
diff --git a/doc/administration/monitoring/performance/img/request_profile_result.png b/doc/administration/monitoring/performance/img/request_profile_result.png
index 1b06e240fa0..9176a0b49fd 100644
--- a/doc/administration/monitoring/performance/img/request_profile_result.png
+++ b/doc/administration/monitoring/performance/img/request_profile_result.png
Binary files differ
diff --git a/doc/administration/monitoring/performance/influxdb_configuration.md b/doc/administration/monitoring/performance/influxdb_configuration.md
index fa281f47ed8..cf6728510fe 100644
--- a/doc/administration/monitoring/performance/influxdb_configuration.md
+++ b/doc/administration/monitoring/performance/influxdb_configuration.md
@@ -187,7 +187,7 @@ Read more on:
[influxdb documentation]: https://docs.influxdata.com/influxdb/v0.9/
[influxdb cli]: https://docs.influxdata.com/influxdb/v0.9/tools/shell/
[udp]: https://docs.influxdata.com/influxdb/v0.9/write_protocols/udp/
-[influxdb]: https://influxdata.com/time-series-platform/influxdb/
+[influxdb]: https://www.influxdata.com/products/influxdb-overview/
[tsm tree]: https://influxdata.com/blog/new-storage-engine-time-structured-merge-tree/
[tsm1-commit]: https://github.com/influxdata/influxdb/commit/15d723dc77651bac83e09e2b1c94be480966cb0d
[influx-admin]: https://docs.influxdata.com/influxdb/v0.9/administration/authentication_and_authorization/#create-a-new-admin-user
diff --git a/doc/administration/monitoring/performance/performance_bar.md b/doc/administration/monitoring/performance/performance_bar.md
index 95f497a1470..02f4b78bd60 100644
--- a/doc/administration/monitoring/performance/performance_bar.md
+++ b/doc/administration/monitoring/performance/performance_bar.md
@@ -8,16 +8,14 @@ activated, it looks as follows:
It allows you to see (from left to right):
- the current host serving the page
-- the timing of the page (backend, frontend)
- time taken and number of DB queries, click through for details of these queries
![SQL profiling using the Performance Bar](img/performance_bar_sql_queries.png)
- time taken and number of [Gitaly] calls, click through for details of these calls
![Gitaly profiling using the Performance Bar](img/performance_bar_gitaly_calls.png)
-- profile of the code used to generate the page, line by line. In the profile view, the numbers in the left panel represent wall time, cpu time, and number of calls (based on [rblineprof](https://github.com/tmm1/rblineprof)).
- ![Line profiling using the Performance Bar](img/performance_bar_line_profiling.png)
-- time taken and number of calls to Redis
-- time taken and number of background jobs created by Sidekiq
-- time taken and number of Ruby GC calls
+- time taken and number of [Rugged] calls, click through for details of these calls
+ ![Rugged profiling using the Performance Bar](img/performance_bar_rugged_calls.png)
+- time taken and number of Redis calls, click through for details of these calls
+ ![Redis profiling using the Performance Bar](img/performance_bar_redis_calls.png)
On the far right is a request selector that allows you to view the same metrics
(excluding the page timing and line profiler) for any requests made while the
@@ -40,10 +38,7 @@ display it.
You can toggle the Bar using the same shortcut.
----
-
![GitLab Performance Bar Admin Settings](img/performance_bar_configuration_settings.png)
----
-
[Gitaly]: ../../gitaly/index.md
+[Rugged]: ../../high_availability/nfs.md#improving-nfs-performance-with-gitlab
diff --git a/doc/administration/monitoring/performance/request_profiling.md b/doc/administration/monitoring/performance/request_profiling.md
index 726882fbb87..9f671e0db11 100644
--- a/doc/administration/monitoring/performance/request_profiling.md
+++ b/doc/administration/monitoring/performance/request_profiling.md
@@ -5,9 +5,9 @@
1. Grab the profiling token from **Monitoring > Requests Profiles** admin page
(highlighted in a blue in the image below).
![Profile token](img/request_profiling_token.png)
-1. Pass the header `X-Profile-Token: <token>` to the request you want to profile. You can use:
+1. Pass the header `X-Profile-Token: <token>` and `X-Profile-Mode: <mode>`(where <mode> can be `execution` or `memory`) to the request you want to profile. You can use:
- Browser extensions. For example, [ModHeader](https://chrome.google.com/webstore/detail/modheader/idgpnmonknjnojddfkpgkljpfnnfcklj) Chrome extension.
- - `curl`. For example, `curl --header 'X-Profile-Token: <token>' https://gitlab.example.com/group/project`.
+ - `curl`. For example, `curl --header 'X-Profile-Token: <token>' --header 'X-Profile-Mode: <mode>' https://gitlab.example.com/group/project`.
1. Once request is finished (which will take a little longer than usual), you can
view the profiling output from **Monitoring > Requests Profiles** admin page.
![Profiling output](img/request_profile_result.png)
diff --git a/doc/administration/monitoring/prometheus/gitlab_metrics.md b/doc/administration/monitoring/prometheus/gitlab_metrics.md
index 84b71ae6f1c..ec26c0b2e7e 100644
--- a/doc/administration/monitoring/prometheus/gitlab_metrics.md
+++ b/doc/administration/monitoring/prometheus/gitlab_metrics.md
@@ -34,6 +34,9 @@ The following metrics are available:
| filesystem_writable | Gauge | 9.4 | Whether or not the filesystem is writable |
| filesystem_read_latency_seconds | Gauge | 9.4 | Read latency of a specific filesystem |
| filesystem_readable | Gauge | 9.4 | Whether or not the filesystem is readable |
+| gitlab_cache_misses_total | Counter | 10.2 | Cache read miss |
+| gitlab_cache_operation_duration_seconds | Histogram | 10.2 | Cache access time |
+| gitlab_cache_operations_total | Counter | 12.2 | Cache operations by controller/action |
| http_requests_total | Counter | 9.4 | Rack request count |
| http_request_duration_seconds | Histogram | 9.4 | HTTP response time from rack middleware |
| pipelines_created_total | Counter | 9.4 | Counter of pipelines created |
@@ -49,7 +52,7 @@ The following metrics are available:
| unicorn_queued_connections | Gauge | 11.0 | The number of queued Unicorn connections |
| unicorn_workers | Gauge | 12.0 | The number of Unicorn workers |
-## Sidekiq Metrics available for Geo **[PREMIUM]**
+## Sidekiq Metrics available for Geo **(PREMIUM)**
Sidekiq jobs may also gather metrics, and these metrics can be accessed if the Sidekiq exporter is enabled (e.g. via
the `monitoring.sidekiq_exporter` configuration option in `gitlab.yml`.
@@ -104,11 +107,11 @@ Some basic Ruby runtime metrics are available:
| ruby_process_cpu_seconds_total | Gauge | 12.0 | Total amount of CPU time per process |
| ruby_process_max_fds | Gauge | 12.0 | Maximum number of open file descriptors per process |
| ruby_process_resident_memory_bytes | Gauge | 12.0 | Memory usage by process, measured in bytes |
-| ruby_process_start_time_seconds | Gauge | 12.0 | The elapsed time between system boot and the process started, measured in seconds |
+| ruby_process_start_time_seconds | Gauge | 12.0 | UNIX timestamp of process start time |
[GC.stat]: https://ruby-doc.org/core-2.3.0/GC.html#method-c-stat
-## Puma Metrics **[EXPERIMENTAL]**
+## Puma Metrics **(EXPERIMENTAL)**
When Puma is used instead of Unicorn, following metrics are available:
@@ -117,7 +120,6 @@ When Puma is used instead of Unicorn, following metrics are available:
| puma_workers | Gauge | 12.0 | Total number of workers |
| puma_running_workers | Gauge | 12.0 | Number of booted workers |
| puma_stale_workers | Gauge | 12.0 | Number of old workers |
-| puma_phase | Gauge | 12.0 | Phase number (increased during phased restarts) |
| puma_running | Gauge | 12.0 | Number of running threads |
| puma_queued_connections | Gauge | 12.0 | Number of connections in that worker's "todo" set waiting for a worker thread |
| puma_active_connections | Gauge | 12.0 | Number of threads processing a request |
diff --git a/doc/administration/monitoring/prometheus/gitlab_monitor_exporter.md b/doc/administration/monitoring/prometheus/gitlab_monitor_exporter.md
index f68b03d1ade..9aa4dfa5ab7 100644
--- a/doc/administration/monitoring/prometheus/gitlab_monitor_exporter.md
+++ b/doc/administration/monitoring/prometheus/gitlab_monitor_exporter.md
@@ -12,9 +12,9 @@ To enable the GitLab monitor exporter:
1. Edit `/etc/gitlab/gitlab.rb`
1. Add or find and uncomment the following line, making sure it's set to `true`:
- ```ruby
- gitlab_monitor['enable'] = true
- ```
+ ```ruby
+ gitlab_monitor['enable'] = true
+ ```
1. Save the file and [reconfigure GitLab][reconfigure] for the changes to
take effect
diff --git a/doc/administration/monitoring/prometheus/index.md b/doc/administration/monitoring/prometheus/index.md
index ce65d77274b..c8968c51393 100644
--- a/doc/administration/monitoring/prometheus/index.md
+++ b/doc/administration/monitoring/prometheus/index.md
@@ -39,9 +39,9 @@ To disable Prometheus and all of its exporters, as well as any added in the futu
1. Edit `/etc/gitlab/gitlab.rb`
1. Add or find and uncomment the following line, making sure it's set to `false`:
- ```ruby
- prometheus_monitoring['enable'] = false
- ```
+ ```ruby
+ prometheus_monitoring['enable'] = false
+ ```
1. Save the file and [reconfigure GitLab][reconfigure] for the changes to
take effect.
@@ -61,19 +61,19 @@ To change the address/port that Prometheus listens on:
1. Edit `/etc/gitlab/gitlab.rb`
1. Add or find and uncomment the following line:
- ```ruby
- prometheus['listen_address'] = 'localhost:9090'
- ```
+ ```ruby
+ prometheus['listen_address'] = 'localhost:9090'
+ ```
- Replace `localhost:9090` with the address/port you want Prometheus to
- listen on. If you would like to allow access to Prometheus to hosts other
- than `localhost`, leave out the host, or use `0.0.0.0` to allow public access:
+ Replace `localhost:9090` with the address/port you want Prometheus to
+ listen on. If you would like to allow access to Prometheus to hosts other
+ than `localhost`, leave out the host, or use `0.0.0.0` to allow public access:
- ```ruby
- prometheus['listen_address'] = ':9090'
- # or
- prometheus['listen_address'] = '0.0.0.0:9090'
- ```
+ ```ruby
+ prometheus['listen_address'] = ':9090'
+ # or
+ prometheus['listen_address'] = '0.0.0.0:9090'
+ ```
1. Save the file and [reconfigure GitLab][reconfigure] for the changes to
take effect
@@ -90,22 +90,22 @@ To use an external Prometheus server:
1. Edit `/etc/gitlab/gitlab.rb`.
1. Disable the bundled Prometheus:
- ```ruby
- prometheus['enable'] = false
- ```
+ ```ruby
+ prometheus['enable'] = false
+ ```
1. Set each bundled service's [exporter](#bundled-software-metrics) to listen on a network address, for example:
- ```ruby
- gitlab_monitor['listen_address'] = '0.0.0.0'
- sidekiq['listen_address'] = '0.0.0.0'
- gitlab_monitor['listen_port'] = '9168'
- node_exporter['listen_address'] = '0.0.0.0:9100'
- redis_exporter['listen_address'] = '0.0.0.0:9121'
- postgres_exporter['listen_address'] = '0.0.0.0:9187'
- gitaly['prometheus_listen_addr'] = "0.0.0.0:9236"
- gitlab_workhorse['prometheus_listen_addr'] = "0.0.0.0:9229"
- ```
+ ```ruby
+ gitlab_monitor['listen_address'] = '0.0.0.0'
+ sidekiq['listen_address'] = '0.0.0.0'
+ gitlab_monitor['listen_port'] = '9168'
+ node_exporter['listen_address'] = '0.0.0.0:9100'
+ redis_exporter['listen_address'] = '0.0.0.0:9121'
+ postgres_exporter['listen_address'] = '0.0.0.0:9187'
+ gitaly['prometheus_listen_addr'] = "0.0.0.0:9236"
+ gitlab_workhorse['prometheus_listen_addr'] = "0.0.0.0:9229"
+ ```
1. Install and set up a dedicated Prometheus instance, if necessary, using the [official installation instructions](https://prometheus.io/docs/prometheus/latest/installation/).
1. Add the Prometheus server IP address to the [monitoring IP whitelist](../ip_whitelist.html). For example:
@@ -117,14 +117,14 @@ To use an external Prometheus server:
1. To scrape nginx metrics, you'll also need to configure nginx to allow the Prometheus server
IP. For example:
- ```ruby
- nginx['status']['options'] = {
- "server_tokens" => "off",
- "access_log" => "off",
- "allow" => "192.168.0.1",
- "deny" => "all",
- }
- ```
+ ```ruby
+ nginx['status']['options'] = {
+ "server_tokens" => "off",
+ "access_log" => "off",
+ "allow" => "192.168.0.1",
+ "deny" => "all",
+ }
+ ```
1. [Reconfigure GitLab][reconfigure] to apply the changes
1. Edit the Prometheus server's configuration file.
@@ -132,19 +132,59 @@ To use an external Prometheus server:
[scrape target configuration](https://prometheus.io/docs/prometheus/latest/configuration/configuration/#%3Cscrape_config%3E).
For example, a sample snippet using `static_configs`:
- ```yaml
- scrape_configs:
- - job_name: 'gitlab_exporters'
- static_configs:
- - targets: ['1.1.1.1:9168', '1.1.1.1:9236', '1.1.1.1:9236', '1.1.1.1:9100', '1.1.1.1:9121', '1.1.1.1:9187']
-
- - job_name: 'gitlab_metrics'
- metrics_path: /-/metrics
- static_configs:
- - targets: ['1.1.1.1:443']
- ```
-
-1. Restart the Prometheus server.
+ ```yaml
+ scrape_configs:
+ - job_name: nginx
+ static_configs:
+ - targets:
+ - 1.1.1.1:8060
+ - job_name: redis
+ static_configs:
+ - targets:
+ - 1.1.1.1:9121
+ - job_name: postgres
+ static_configs:
+ - targets:
+ - 1.1.1.1:9187
+ - job_name: node
+ static_configs:
+ - targets:
+ - 1.1.1.1:9100
+ - job_name: gitlab-workhorse
+ static_configs:
+ - targets:
+ - 1.1.1.1:9229
+ - job_name: gitlab-rails
+ metrics_path: "/-/metrics"
+ static_configs:
+ - targets:
+ - 1.1.1.1:8080
+ - job_name: gitlab-sidekiq
+ static_configs:
+ - targets:
+ - 1.1.1.1:8082
+ - job_name: gitlab_monitor_database
+ metrics_path: "/database"
+ static_configs:
+ - targets:
+ - 1.1.1.1:9168
+ - job_name: gitlab_monitor_sidekiq
+ metrics_path: "/sidekiq"
+ static_configs:
+ - targets:
+ - 1.1.1.1:9168
+ - job_name: gitlab_monitor_process
+ metrics_path: "/process"
+ static_configs:
+ - targets:
+ - 1.1.1.1:9168
+ - job_name: gitaly
+ static_configs:
+ - targets:
+ - 1.1.1.1:9236
+ ```
+
+1. Reload the Prometheus server.
## Viewing performance metrics
@@ -241,9 +281,9 @@ To disable the monitoring of Kubernetes:
1. Edit `/etc/gitlab/gitlab.rb`.
1. Add or find and uncomment the following line and set it to `false`:
- ```ruby
- prometheus['monitor_kubernetes'] = false
- ```
+ ```ruby
+ prometheus['monitor_kubernetes'] = false
+ ```
1. Save the file and [reconfigure GitLab][reconfigure] for the changes to
take effect.
diff --git a/doc/administration/monitoring/prometheus/node_exporter.md b/doc/administration/monitoring/prometheus/node_exporter.md
index aef7758a88f..bcacfaa3be5 100644
--- a/doc/administration/monitoring/prometheus/node_exporter.md
+++ b/doc/administration/monitoring/prometheus/node_exporter.md
@@ -13,9 +13,9 @@ To enable the node exporter:
1. Edit `/etc/gitlab/gitlab.rb`
1. Add or find and uncomment the following line, making sure it's set to `true`:
- ```ruby
- node_exporter['enable'] = true
- ```
+ ```ruby
+ node_exporter['enable'] = true
+ ```
1. Save the file and [reconfigure GitLab][reconfigure] for the changes to
take effect
diff --git a/doc/administration/monitoring/prometheus/pgbouncer_exporter.md b/doc/administration/monitoring/prometheus/pgbouncer_exporter.md
index d76834fdbea..6183594c69c 100644
--- a/doc/administration/monitoring/prometheus/pgbouncer_exporter.md
+++ b/doc/administration/monitoring/prometheus/pgbouncer_exporter.md
@@ -12,9 +12,9 @@ To enable the PgBouncer exporter:
1. Edit `/etc/gitlab/gitlab.rb`
1. Add or find and uncomment the following line, making sure it's set to `true`:
- ```ruby
- pgbouncer_exporter['enable'] = true
- ```
+ ```ruby
+ pgbouncer_exporter['enable'] = true
+ ```
1. Save the file and [reconfigure GitLab][reconfigure] for the changes to
take effect.
diff --git a/doc/administration/monitoring/prometheus/postgres_exporter.md b/doc/administration/monitoring/prometheus/postgres_exporter.md
index 8e2d3162f88..3ad15b65497 100644
--- a/doc/administration/monitoring/prometheus/postgres_exporter.md
+++ b/doc/administration/monitoring/prometheus/postgres_exporter.md
@@ -12,9 +12,9 @@ To enable the postgres exporter:
1. Edit `/etc/gitlab/gitlab.rb`
1. Add or find and uncomment the following line, making sure it's set to `true`:
- ```ruby
- postgres_exporter['enable'] = true
- ```
+ ```ruby
+ postgres_exporter['enable'] = true
+ ```
1. Save the file and [reconfigure GitLab][reconfigure] for the changes to
take effect
diff --git a/doc/administration/monitoring/prometheus/redis_exporter.md b/doc/administration/monitoring/prometheus/redis_exporter.md
index d54d409dbb6..1520115a38d 100644
--- a/doc/administration/monitoring/prometheus/redis_exporter.md
+++ b/doc/administration/monitoring/prometheus/redis_exporter.md
@@ -13,9 +13,9 @@ To enable the Redis exporter:
1. Edit `/etc/gitlab/gitlab.rb`
1. Add or find and uncomment the following line, making sure it's set to `true`:
- ```ruby
- redis_exporter['enable'] = true
- ```
+ ```ruby
+ redis_exporter['enable'] = true
+ ```
1. Save the file and [reconfigure GitLab][reconfigure] for the changes to
take effect