Tech: Rails

Problem: you have a self-hosted Sentry server behind a firewall and you want to report your frontend errors.

One way to accomplish it is by modifying Sentry dsn to send it to your backend and then proxying them to the Sentry server.

First, let’s set up a new route:

post 'frontend_errors/api/:project_id/store', to: 'frontend_errors#create'

It has to follow a specific pattern to work with the Sentry frontend library. The only thing you can change in the above is frontend_errors – pick whatever name you want. The code above will expect you to have a FrontendErrorsController.

Now, the FrontEndErrorsController needs to redirect to your actual Sentry server in the format that Sentry expects. Let’s create a new class to handle it:

class SentryProxy
  # This could be different based on your Sentry version.
  # Look into raven-sentry gem codebase if this doesn't work
  # Look for http_transport.rb files - https://github.com/getsentry/sentry-ruby/blob/f6625bd12fa5ef86e4ce6a1515e8a8171cea9ece/sentry-ruby/lib/sentry/transport/http_transport.rb
  PROTOCOL_VERSION = '5'
  USER_AGENT = "raven-ruby/#{Raven::VERSION}"

  def initialize(body:, sentry_dsn:)
    @body = body
    @sentry_dsn = sentry_dsn
  end

  def post_to_sentry
    return if @sentry_dsn.blank?

    sentry_connection.post do |faraday|
      faraday.body = @body
    end
  end

  private

  def sentry_connection
    Faraday.new(url: sentry_post_url) do |faraday|
      faraday.headers['X-Sentry-Auth'] = generate_auth_header
      faraday.headers[:user_agent] = "sentry-ruby/#{Raven::VERSION}"
      faraday.adapter(Faraday.default_adapter)
    end
  end

  def sentry_post_url
    key, url = @sentry_dsn.split('@')
    path, project_id = url.split('/')
    http_prefix, _keys = key.split('//')

    "#{http_prefix}//#{path}/api/#{project_id}/store/"
  end

  def generate_auth_header
    now = Time.now.to_i.to_s
    public_key, secret_key = @sentry_dsn.split('//').second.split('@').first.split(':')

    fields = {
      'sentry_version' => PROTOCOL_VERSION,
      'sentry_client' => USER_AGENT,
      'sentry_timestamp' => now,
      'sentry_key' => public_key,
      'sentry_secret' => secret_key
    }
    'Sentry ' + fields.map { |key, value| "#{key}=#{value}" }.join(', ')
  end
end

Now in your controller you can call it like this (assumes you can get your sentry_dsn on the backend):

def create
  SentryProxy.new(body: request.body.read, sentry_dsn: sentry_dsn).post_to_sentry

  head(:no_content)
end

And to make sure your frontend is properly configured, first import Sentry frontend libraries, then initialize them using:

 Sentry.init({
    dsn: `${window.location.protocol}//public_key@${window.location.host}/frontend_errors/0`});

public_key is supposed to be… your public key. You have to supply it in the dsn even if you’re getting the dsn key on the backend, otherwise, the Sentry frontend library will throw errors. 0 is the project id – the same idea, you have to supply it for the Sentry frontend to properly parse it. It doesn’t have to be real, as we’re reconstructing the Sentry url on the backend, and you can get proper keys/project id on the backend.

This should do it. Now you can configure Sentry frontend library to capture all errors, capture specific exceptions or messages.

Redis docs: https://redis.io/topics/lru-cache

OS: Ubuntu 18.04 LTS

When you need to use redis for cache and non-cache entries (e.g., ActionCable, Sidekiq…), the recommended approach is to create a separate redis instance. However, if you want a simpler setup, or just can’t get another instance for reasons, there is an option to use the same redis instance for multiple uses.

We need to make sure that Redis will not evict our important data (e.g., Sidekiq), while at the same time evicting old cache entries. We could use any of the volatile eviction policies:

• volatile-lru – remove least recently used keys where expiry is set
• volatile-random – removes keys at random where expiry is set
• volatile-ttl – evict keys with an expire set, and try to evict keys with a shorter time to live (TTL) first
• volatile-lfu (starting with Redis 4.0) – evict using approximated LFU among the keys with an expire set.

To set up the eviction policy on your redis instance, edit your /etc/systemd/system/redis.conf and set these parameters:

maxmemory 100mb
maxmemory-policy volatile-lfu

Then in your Rails config update your store to use redis cache store, if not using already:

  config.cache_store = :redis_cache_store, {
    url: ENV.fetch('REDIS_URL', 'redis://localhost:6379'),
    expires_in: 24.hours
  }

To import the public key in ruby:

EncryptionError = Class.new(StandardError)

result, stderr, status = Open3.capture3("gpg --import #{@key_path}")
raise EncryptionError.new(stderr_data) unless status.success?

To encrypt data with a public key for a given recipient:

pgp_encrypt_command = "gpg -ear #{recipient} --always-trust --trust-model always --local-user #{recipient} --default-key #{recipient}"

encrypted_data, stderr_data, status = Open3.capture3(pgp_encrypt_command, stdin_data: data)
    raise EncryptionError.new(stderr_data) unless status.success?

This is only one of the reasons why this may happen – Xcode Command Line Tools (CLT) are not installed. This may happen after you upgrade you macOS version.

To fix, run this on the command line:

xcode-select --install

this should trigger a popup with the invitation to install Xcode CLT.

Selected VPS: Linode, 1GB Ram, 20 GB SSD, 1 TB transfer
OS: Ubuntu 18.04 LTS
App server: Passenger
Web Server: Ngnix

If you’d like to try Linode, I would greatly appreciate using this referral link – Linode: SSD Cloud Hosting & Linux Servers

Guide

First step – boot your server from Linode Manager Dashboard and ssh into it

General Server Updates

apt-get update && apt-get upgrade

#If desired to run auto-updates
apt-get install unattended-upgrades
dpkg-reconfigure --priority=low unattended-upgrades

Set timezone:

dpkg-reconfigure tzdata
sudo service rsyslog restart

Security:

fail2ban

sudo apt-get install fail2ban

awk '{ printf "# "; print; }' /etc/fail2ban/jail.conf | sudo tee /etc/fail2ban/jail.local
vim /etc/fail2ban/jail.conf

uncomment sshd section and add
enabled = true

sudo apt-get install sendmail iptables-persistent
sudo service fail2ban start

Firewall ( allow established connections, traffic generated by the server itself, traffic destined for our SSH and web server ports. We will drop all other traffic):

sudo service fail2ban stop
sudo iptables -A INPUT -i lo -j ACCEPT
sudo iptables -A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
sudo iptables -A INPUT -p tcp --dport 22 -j ACCEPT
sudo iptables -A INPUT -p tcp -m multiport --dports 80,443 -j ACCEPT
sudo iptables -A INPUT -j DROP

#easy way to rate-limit ssh with ufw:
# technically, we could do all of the iptables stuff with ufw
ufw enable
ufw limit ssh

If using IPv6:

ip6tables -A INPUT -p tcp --dport 80 -j ACCEPT
ip6tables -A INPUT -p tcp --dport 443 -j ACCEPT
ip6tables -A INPUT -p tcp --dport 22 -j ACCEPT # (replace with your undisclosed port)
ip6tables -A INPUT -p icmpv6 -j ACCEPT
ip6tables -A INPUT -j REJECT
ip6tables -A FORWARD -j REJECT

View iptables rules:

sudo iptables -S

Save iptables rules:

sudo dpkg-reconfigure iptables-persistent
sudo service fail2ban start

Create a deploy user

sudo adduser deploy
sudo adduser deploy sudo
su deploy

Installing Ruby (2.4)

Prereq’s:

sudo apt-get install git-core curl zlib1g-dev build-essential libssl-dev libreadline-dev libyaml-dev libsqlite3-dev sqlite3 libxml2-dev libxslt1-dev libcurl4-openssl-dev python-software-properties libffi-dev nodejs

cd
git clone https://github.com/rbenv/rbenv.git ~/.rbenv
echo 'export PATH="$HOME/.rbenv/bin:$PATH"' >> ~/.bashrc
echo 'eval "$(rbenv init -)"' >> ~/.bashrc
exec $SHELL

git clone https://github.com/rbenv/ruby-build.git ~/.rbenv/plugins/ruby-build
echo 'export PATH="$HOME/.rbenv/plugins/ruby-build/bin:$PATH"' >> ~/.bashrc
exec $SHELL

rbenv install 2.4.0
rbenv global 2.4.0
ruby -v

gem install bundler

Nginx

sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 561F9B9CAC40B2F7
sudo apt-get install -y apt-transport-https ca-certificates

# Add Passenger APT repository
sudo sh -c 'echo deb https://oss-binaries.phusionpassenger.com/apt/passenger xenial main > /etc/apt/sources.list.d/passenger.list'
sudo apt-get update

# Install Passenger & Nginx
sudo apt-get install -y nginx-extras passenger

Nginx update for Ubuntu 17.10 zesty

sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 561F9B9CAC40B2F7
sudo apt-get install -y apt-transport-https ca-certificates

# Add our APT repository
sudo sh -c 'echo deb https://oss-binaries.phusionpassenger.com/apt/passenger zesty main > /etc/apt/sources.list.d/passenger.list'
sudo apt-get update
# Install Passenger + Nginx module
sudo apt-get install -y libnginx-mod-http-passenger libnginx-mod-http-headers-more-filter nginx

Nginx update for Ubuntu 18.04 bionic

sudo sh -c 'echo deb https://oss-binaries.phusionpassenger.com/apt/passenger bionic main > /etc/apt/sources.list.d/passenger.list'

sudo apt-get update
sudo apt-get install -y libnginx-mod-http-passenger

Block anything you don’t want in Nginx

vim /etc/nginx/sites-enabled/default
location ~ ^/(wp-admin|wp-content|wp-login) {
 deny all;
}

Set Cache headers

location ^~ /assets/ {
        gzip_static on;
        expires 1d;
        add_header Cache-Control "public";
}

location ~* \.(?:jpg|jpeg|gif|png|ico|cur|gz|svg|svgz|mp4|ogg|ogv|webm|htc)$ {
  expires 30d;
  access_log off;
  add_header Cache-Control "public";
}

Prevent access by IP

vim /etc/nginx/sites-enabled/default
server {
        listen 80;
        listen 443;
        server_name _put_server_ip_here;
        return 404;
}

Start Nginx:
sudo service nginx start


Nginx config:
sudo vim /etc/nginx/nginx.conf
Uncomment Phusion config – include /etc/nginx/passenger.conf;

Update passenger config:
sudo vim /etc/nginx/passenger.conf

passenger_ruby /home/deploy/.rbenv/shims/ruby;
passenger_root /usr/lib/ruby/vendor_ruby/phusion_passenger/locations.ini;
#passenger_ruby /usr/bin/passenger_free_ruby;

Restart Nginx
sudo service nginx restart

Test Nginx Config with gixy

We will need python-pip in order to install gixy:

apt-get install python-pip
pip install --upgrade pip
pip -V

pip install gixy

gixy

Postgres

sudo apt-get install postgresql postgresql-contrib libpq-dev

Postgres update for Ubuntu 17.10

sudo apt-get install postgresql-9.6
sudo apt-get install python-psycopg2
sudo apt-get install libpq-dev

Postgres update for Ubuntu 18.04

touch /etc/apt/sources.list.d/pgdg.list
sudo sh -c 'echo deb http://apt.postgresql.org/pub/repos/apt/ bionic-pgdg main > /etc/apt/sources.list.d/pgdg.list'

wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | sudo apt-key add -
sudo apt-get update

apt-get install postgresql-10

#Migrate Postresql from 9.6 to 10
sudo pg_dropcluster 10 main --stop
sudo systemctl stop postgresql 
sudo pg_upgradecluster  9.6 main
sudo pg_dropcluster 9.6 main --stop

Ensure UTF-8 support before creating a new database
vim /etc/profile.d/lang.sh
Add the following if not already there:

export LANGUAGE="en_US.UTF-8"
export LANG="en_US.UTF-8"
export LC_ALL="en_US.UTF-8"

Create a Database with a user (Make sure to change the app name):

sudo su - postgres
createuser --pwprompt deploy
sudo su - deploy
psql
create database your_db_name_production_new with owner=deploy encoding='UTF-8';

exit

Creating dumps

pg_dump db_name_production -U deploy -h localhost > db_name_production_backup
# To restore:
psql -d db_name_production -f db_name_production_backup

#if you want to scp it from somewhere else(assuming dumps are in ~/dumps):
scp -r root@server_ip:~/dumps ~/destination_dumps

Capistrano

On your local machine add to your Gemfile:

gem 'capistrano', '~> 3.7', '>= 3.7.1'
gem 'capistrano-passenger', '~> 0.2.0'
gem 'capistrano-rails', '~> 1.2'

group :production do
  gem 'capistrano-rbenv', '~> 2.1'
end

Generate configs:

cap install STAGES=production

Update Capfile:

# If you are using rbenv add these lines:
# require 'capistrano/rbenv'
# set :rbenv_type, :user
# set :rbenv_ruby, '2.4.0'

Update deploy.rb:

set :application, "my_app_name"
set :repo_url, "[email protected]:me/my_repo.git"

set :deploy_to, '/home/deploy/my_app_name'

append :linked_files, "config/database.yml", "config/secrets.yml"
append :linked_dirs, "log", "tmp/pids", "tmp/cache", "tmp/sockets", "vendor/bundle", "public/system", "public/uploads"

Update production.rb with the server IP:
server ‘127.0.0.1’, user: ‘deploy’, roles: %w{app db web}

Had to manually create database.yml and secrets.yml for now in /home/deploy/my_app_name/shared/config and copy connection data. This should be automated later

Update /etc/nginx/sites-enabled/default

server {
        listen 80;
        listen [::]:80 ipv6only=on;

        server_name mydomain.com;
        passenger_enabled on;
        rails_env    production;
        root         /home/deploy/my_app_name/current/public;
        
#action cable config:
        location /cable {
                 passenger_app_group_name app_name_websocket;
                 passenger_force_max_concurrent_requests_per_process 0;
        }
}

Redis

sudo apt-get install redis-server

sudo service nginx restart

Time Synchronization

sudo apt-get update
sudo apt-get install ntp

systemctl start ntp.service
sudo ntpq -p

Let’s Encrypt (optional step)

Install certbot

apt-get install software-properties-common
add-apt-repository ppa:certbot/certbot
apt-get update
apt-get install certbot

Add to /etc/nginx/sites-available/default (this may be necessary for the Webroot plugin, which we’ll use to generate the certs)

location ^~ /.well-known/acme-challenge {
  root /var/www/html;
  default_type "text/plain";
  try_files $uri =404;
}

Verify config and restart:

nginx -t		
systemctl restart nginx

Get your webroot path from /etc/nginx/sites-available/default

Generate certs(replace domain an dweb-root-path):

certbot certonly --webroot --webroot-path=/var/www/html -d example.com -d www.example.com

Add a snippet:
vim /etc/nginx/snippets/ssl-example.com.conf
Paste the following:

ssl_certificate /etc/letsencrypt/live/example.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/example.com/privkey.pem;

And another one for ssl params:
vim /etc/nginx/snippets/ssl-params.conf

ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_prefer_server_ciphers on;
ssl_ciphers "EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH";
ssl_ecdh_curve secp384r1;
ssl_session_cache shared:SSL:10m;
ssl_session_tickets off;
ssl_stapling on;
ssl_stapling_verify on;
resolver 8.8.8.8 8.8.4.4 valid=300s;
resolver_timeout 5s;
add_header X-Frame-Options DENY;
add_header X-Content-Type-Options nosniff;

Update nginx config (this will allow both http and https):
vim /etc/nginx/sites-available/default

server {
    listen 80 default_server;
    listen [::]:80 default_server;
    listen 443 ssl http2 default_server;
    listen [::]:443 ssl http2 default_server;

    server_name example.com www.example.com;
    include snippets/ssl-example.com.conf;
    include snippets/ssl-params.conf;
...

Automate certificate renewal:
crontab -e

15 1 * * * /usr/bin/certbot renew --quiet --renew-hook "/bin/systemctl reload nginx"
20 1 * * * nginx -s reloadv

The above will run at 1:15 am every day, then reloads nginx config at 1:20 am.

To view certs expiration dates:

apt-get install  ssl-cert-check
ssl-cert-check -c /etc/letsencrypt/live/example.com/cert.pem

Additional Mac Setup

brew install ssh-copy-id
ssh-copy-id deploy@IPADDRESS
This will allow you to ssh without a password the next time you ssh.

Useful logs

cat /var/log/redis/redis-server.log
sudo cat /var/log/fail2ban.log
sudo cat /var/log/nginx/error.log

fail2ban grouped by IP:

awk '($(NF-1) = /Ban/){print $NF}' /var/log/fail2ban.log | sort | uniq -c | sort -n

Useful monitoring commands

Ram usage:
free -m
top

Space:
df -ah
du -hs * | sort -h

Another way to get space info
apt-get install ncdu
ncdu /

Network activity:
netstat -tulpn

Linode ipv6 issue

If you can’t get apt-get to work, try to update /etc/gai.conf and uncomment line 54: precedence ::ffff:0:0/96 100

Linode Monitoring – Longview

Linode also provides Longview – web UI to access your server stats. The free version is limited to the last 24 hours.
Here is the official guide
I had to update iptables rules and restart the longview service:

iptables -I INPUT -s longview.linode.com -j ACCEPT
iptables -I OUTPUT -d longview.linode.com -j ACCEPT 
sudo service longview restart

Useful links:

• Deploy ActionCable in production
• Deploy Ruby On Rails on Ubuntu 16.04 Xenial Xerus
• Capistrano Github
• Pagespeed – The tool and optimization guide
• Nifty Nginx config generator

Now, while it’s fun figuring these things out, git push heroku master sounds a lot easier.

As I continue to learn Rails, this is my experience developing a Rails application using Action Cable (allows communication using web sockets)

Finished product: Story Estimate

Assuming we already generated a new Rails app, lets start with authentication using Devise gem.
Add “gem ‘devise'” to your gem file.

Run a few commands to generate your users:
$ rails generate devise:install
$ rails generate devise User
$ rails generate devise:views
$ rails db:migrate

For the sake of this application, I want to enable guest users. Lets allow that:

class ApplicationController < ActionController::Base
  protect_from_forgery with: :exception

  def current_or_guest_user
    current_user || guest_user
  end

  # find guest_user object associated with the current session,
  # creating one as needed
  def guest_user(with_retry = true)
    # Cache the value the first time it's gotten.
    @cached_guest_user ||= User.find(session[:guest_user_id] ||= create_guest_user.id)

  rescue ActiveRecord::RecordNotFound # if session[:guest_user_id] invalid
    session[:guest_user_id] = nil
    guest_user if with_retry
  end

  private

  def create_guest_user
    u = User.create(email: "guest_#{Time.now.to_i}#{rand(100)}@example.com")
    u.save!(validate: false)
    session[:guest_user_id] = u.id
    u
  end
end

Now, if we want to sign in a user as a guest, we’d call sign_in(current_or_guest_user)

Next thing, we want to authorize our Action Cable connections with Devise users. In order to do that, we need to add some hooks in the config/initialazers. This will add current user id to user’s cookie:

config/initialazers/warden_hooks.rb

Warden::Manager.after_set_user do |user,auth,opts|
  scope = opts[:scope]
  auth.cookies.signed["#{scope}.id"] = user.id
end

Warden::Manager.before_logout do |user, auth, opts|
  scope = opts[:scope]
  auth.cookies.signed["#{scope}.id"] = nil
end

With this, we can configure rails connection:

module ApplicationCable
  class Connection < ActionCable::Connection::Base
    identified_by :current_user

    def connect
      self.current_user = find_verified_user
    end

    protected
      def find_verified_user
        if verified_user = User.find_by(id: cookies.signed['user.id'])
          verified_user
        else
          reject_unauthorized_connection
        end
      end
  end
end

Now, time to generate a new channel:
rails g channel channel_name optional_method
It may look someting like this:

class CoolChannel < ApplicationCable::Channel
  BASE_CHANNEL = 'BaseChannel'.freeze

  def subscribed
    stream_from "#{BASE_CHANNEL}#{params['some_key']}"
  end

  def unsubscribed
    # Any cleanup needed when channel is unsubscribed
  end

  def some_action(data)
    value_from_client = data['value_0']
    ActionCable.server.broadcast\
      "#{BASE_CHANNEL}#{data['some_key']}",
      data
  end
end

This will create a new custom-named channel based on whatever we want to send in params from the client. some_action is the method that will be called by the clients to send a message. In this case we just rebroadcast the message to all clients.

To make a call from the client to the server, go to the .js or .coffee file for the earlier generated channel (should be in javascripts/channels). We can call back to the server:

jQuery(document).on 'turbolinks:load', ->
  session_slug = $('#session_slug').val()
  player_name = $('#player_id').val()
  $stats =$('#statistics')

  App.estimation_session = App.cable.subscriptions.create {
    channel: "CoolChannel",
    },
    connected: ->

    disconnected: ->
      # Called when the subscription has been terminated by the server

    received: (data) ->
      value_from_server = data.value_0

    some_action: (value_0, value_1)->
      @perform 'some_action',
               value_0: value_0,
               value_1: value_1

When deploying to Heroku, we need to Add Redis To Go add-in (or any other Redis add-in such as Heroku Redi. Then update cable.yml to use redis:
adapter: redis
url: YOUR_URL

To find Redis connection URL, run:
heroku config | grep redis

Also, we need to add redis gem:
gem ‘redis’, ‘~> 3.2’

git push heroku master

Some useful resources:

• https://www.pluralsight.com/guides/ruby-ruby-on-rails/creating-a-chat-using-rails-action-cable
• https://www.sitepoint.com/create-a-chat-app-with-rails-5-actioncable-and-devise/
• http://exponential.io/blog/2015/02/21/install-postgresql-on-mac-os-x-via-brew/
• https://www.digitalocean.com/community/tutorials/how-to-setup-ruby-on-rails-with-postgres
• https://blog.heroku.com/real_time_rails_implementing_websockets_in_rails_5_with_action_cable