How to install and kick off minikube on Ubuntu?

Install kubectl:

$ sudo apt-get update
$ sudo apt-get install apt-transport-https
$ curl -sSL https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -
$ echo "deb http://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee -a /etc/apt/sources.list.d/kubernetes.list
$ sudo apt-get update
$ sudo apt-get install kubectl

Install virtualbox:

$ sudo apt-get update
$ sudo apt-get install virtualbox

Install minikube (replace v0.30.0 with the latest version available for you):

$ wget https://github.com/kubernetes/minikube/releases/download/v0.30.0/minikube-linux-amd64
$ mv minikube-linux-amd64 minikube
$ chmod +x minikube
$ sudo cp minikube /usr/local/bin/
$ rm minikube

Create the VM and start the cluster (this will create the file ~/.kube/config):

$ minikube start  # use `minikube start --help` for more options
$ kubectl cluster-info

To get the IP address of the cluster:

$ minikube ip
$ sudo vi /etc/hosts  # if you want to assign names to this IP address

To show the Kubernetes dashboard:

$ minikube dashboard

To stop the cluster (shuts down the VM and preserve all states):


$ minikube stop

To restart the VM and cluster:

$ minikube start

To delete the cluster (and the VM):

$ minikube delete
$ rm ~/.kube/config

How to do a basic install of Prometheus on Ubuntu?

Machine requirement

I will assume your machine is Ubuntu >= 16.04.

Please make sure only ports 80 and 443 are open.

I assume there are two DNS entries to your machine, such as prometheus.blablabla.com and grafana.blablabla.com (replace as appropriate throughout the tutorial).

Install Prometheus

Follow the steps (you can replace version 2.4.3 with whatever latest version is available for you):

$ sudo useradd --no-create-home --shell /bin/false prometheus
$ wget https://github.com/prometheus/prometheus/releases/download/v2.4.3/prometheus-2.4.3.linux-amd64.tar.gz
$ tar xf prometheus-2.4.3.linux-amd64.tar.gz
$ cd prometheus-2.4.3.linux-amd64
$ sudo cp prometheus promtool /usr/local/bin
$ sudo mkdir /etc/prometheus /var/lib/prometheus
$ sudo cp -r prometheus.yml consoles console_libraries /etc/prometheus
$ sudo chown prometheus.prometheus /var/lib/prometheus

Create the file /etc/systemd/system/prometheus.service and put the following content in it:

[Unit]
Description=Prometheus
Wants=network-online.target
After=network-online.target

[Service]
User=prometheus
Group=prometheus
Type=simple
ExecStart=/usr/local/bin/prometheus \
    --config.file /etc/prometheus/prometheus.yml \
    --storage.tsdb.path /var/lib/prometheus \
    --web.console.templates /etc/prometheus/consoles \
    --web.console.libraries /etc/prometheus/console_libraries
Restart=always

[Install]
WantedBy=multi-user.target

Then run:

$ sudo systemctl daemon-reload
$ sudo systemctl enable prometheus
$ sudo systemctl start prometheus

Install Grafana

To install Grafana:

$ echo "deb https://packagecloud.io/grafana/stable/debian/ stretch main" | sudo tee -a /etc/apt/sources.list.d/grafana.list
$ curl -sSL https://packagecloud.io/gpg.key | sudo apt-key add -
$ sudo apt-get update
$ sudo apt-get install grafana
$ sudo systemctl daemon-reload
$ sudo systemctl enable grafana-server
$ sudo systemctl start grafana-server

Do not try to configure Grafana yet! We need to do all the SSL stuff first.

Reverse-proxy and SSL certificates

Install certbot

Follow the steps:

$ sudo apt-get install software-properties-common
$ sudo add-apt-repository ppa:certbot/certbot
$ sudo apt-get update
$ sudo apt-get install certbot python-certbot-nginx

Reverse-proxy for Prometheus

Prometheus web interface has no authentication mechanism, so we will add one with nginx.

To install nginx as a reverse-proxy for SSL termination and authentication for Prometheus (replace USERNAME and PASSWORD with a username and password of your choice):

$ sudo apt-get install nginx
$ sudo rm /etc/nginx/sites-enabled/default
$ echo "USERNAME:$(openssl passwd -apr1 PASSWORD)" | tee -a /etc/nginx/prometheus.htpasswd

Then edit the file /etc/nginx/sites-available/prometheus to look like this (replace prometheus.blablabla.com with your real domain name; NB: more SSL config will be added by certbot):

server {
  listen 80;
  server_name prometheus.blablabla.com;
}

server {
  listen 443 ssl;
  server_name prometheus.blablabla.com;

  location / {
    proxy_pass http://localhost:9090;
    proxy_set_header Host $host;
    auth_basic "My Prometheus";
    auth_basic_user_file /etc/nginx/prometheus.htpasswd;
  }
}

Reload nginx:

$ sudo ln -s /etc/nginx/sites-available/prometheus /etc/nginx/sites-enabled/
$ sudo systemctl reload nginx

Create an SSL certificate for prometheus.blablabla.com:

$ sudo certbot --nginx -d prometheus.blablabla.com

Certbot will start an interactive session. Answer as follows:

  • When asked for your email address, enter it
  • Agree to the terms of service
  • Agree or not to the marketing, that has no incidence on your SSL certificate
  • When asked to redirect all HTTP traffic to HTTPS, choose to redirect

Reverse-proxy for Grafana

Edit the file /etc/nginx/sites-available/grafana to look like this (NB: more SSL config will be added by certbot):

server {
  listen 80;
  server_name grafana.blablabla.com;
}

server {
  listen 443 ssl;
  server_name grafana.blablabla.com;

  location / {
    proxy_pass http://localhost:3000;
    proxy_set_header Host $host;
  }
}

Reload nginx:

$ sudo ln -s /etc/nginx/sites-available/grafana /etc/nginx/sites-enabled/
$ sudo systemctl reload nginx

Create an SSL certificate for grafana.blablabla.com:

$ sudo certbot --nginx -d grafana.blablabla.com

Answer the interactive prompts in the same manner as previously.

Configure Grafana

Open your favourite browser to http://grafana.blablabla.com. The Grafana login page should come up. Enter admin for both the username and password; the next page should ask you for a more secure password.

Then click on the 2nd icon “Add data source” and enter the following details:

  • Name: Whatever you like
  • Type: Prometheus
  • URL: http://localhost:9090
  • Leave the rest untouched
  • Click “Save & Test”
  • You should see a message at the bottom of the page “Data source is working”

Install and configure Alertmanager

Follow the steps (you can replace version 0.15.2 with whatever latest version is available for you):

$ sudo useradd --no-create-home --shell /bin/false alertmanager
$ wget //github.com/prometheus/alertmanager/releases/download/v0.15.2/alertmanager-0.15.2.linux-amd64.tar.gz
$ tar xf alertmanager-0.15.2.linux-amd64.tar.gz
$ cd alertmanager-0.15.2.linux-amd64
$ sudo cp alertmanager amtool /usr/local/bin
$ sudo mkdir -p /etc/alertmanager/templates /var/lib/alertmanager
$ sudo chown alertmanager.alertmanager /var/lib/alertmanager

Create the file /etc/alertmanager/alertmanager.yml so it looks like this (substitute the correct values for your setup):

global:
  smtp_smarthost: 'localhost:25'
  smtp_from: 'alertmanager@prometheus.blablabla.com'
  smtp_auth_username: ''
  smtp_auth_password: ''
  smtp_require_tls: false

templates:
  - /etc/alertmanager/templates/*.tmpl

route:
  repeat_interval: 1h
  receiver: MYTEAM

receivers:
  - name: MYTEAM
    email_configs:
    - to: MYTEAM@MYORG.com
    slack_configs:
    - api_url: https://hooks.slack.com/services/XXXXXX/XXXXXX/XXXXXX
      send_resolved: true

Create the file /etc/systemd/system/altermanager.service and put the following content in it:

[Unit]
Description=Alertmanager
Wants=network-online.target
After=network-online.target

[Service]
User=alertmanager
Group=alertmanager
Type=simple
ExecStart=/usr/local/bin/alertmanager \
        --config.file /etc/alertmanager/alertmanager.yml \
        --storage.path /var/lib/alertmanager
Restart=always

[Install]
WantedBy=multi-user.target

Then run:

$ sudo systemctl daemon-reload
$ sudo systemctl enable prometheus
$ sudo systemctl start prometheus

You need to tell prometheus that an alertmanager service is available. To do so, edit the /etc/prometheus/prometheus.yml file and add the following at the end:

alerting:
  alertmanagers:
    - static_configs:
      - targets:
        - localhost:9093

Then:

$ sudo systemctl restart prometheus

How to install Docker on Ubuntu

Follow the steps:

$ sudo apt-get install curl software-properties-common
$ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
$ sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
$ sudo apt-get update
$ sudo apt-get install docker-ce
$ sudo usermod -aG docker $(id -un)  # optional, allows you to run docker without `sudo`

How to enable and analyse the slow query log on MySQL?

Set the following variables:

slow_query_log = 1 (means true)
long_query_time = 0 (means 0 seconds)
log_queries_not_using_indexes = 1 (means true)
log_output = FILE (default is TABLE)

NB: Setting `long_query_time` to 0 will instruct MySQL to log all queries to the slow query log. Some queries might not be that slow, but if repeated often enough will have a significant impact on the performance of your MySQL database. Setting `long_query_time` to 0 will allow you to catch that kind of scenario as well.

Let MySQL run for a while, then download pt-query-digest, and run it like so:

$ pt-query-digest slow-query.log > slow-query.log.out

Look at the `slow-query.log.out` file to find out which queries are slowing down your MySQL database.