Docker MySQL with a Custom SQL Script for Development

The setup is similar to setting up MariaDB.

Start with standard docker-compose file. If using custom SQL mode, specify the necessary options in the command options:

version: "3.7"
            context: .
            dockerfile: dev.dockerfile
        restart: always
        command: --sql_mode="STRICT_TRANS_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE"
            MYSQL_ROOT_PASSWORD: root_password
            MYSQL_DATABASE: dev
            MYSQL_USER: dev_user
            MYSQL_PASSWORD: dev_password
            - 3306:3306

Add dev.dockerfile:

FROM mysql:8.0.17

ADD init.sql /docker-entrypoint-initdb.d/ddl.sql

Finally, add your init.sql file. Let’s give all privileges to our dev_user and switch the default caching_sha2_password to mysql_native_password (don’t do it unless you rely on older packages that require the less secure au

Finally, add your init.sql file. Let’s give all privileges to our dev_user and switch the default caching_sha2_password to mysql_native_password (don’t do it unless you rely on older packages that require the less secure mysql_native_password authentication method):

GRANT ALL PRIVILEGES ON *.* TO 'dev_user'@'%';
ALTER USER 'dev_user'@'%' IDENTIFIED WITH mysql_native_password BY 'dev_password';

If you want to access the database container from other containers, while running them separately, you can specify host.docker.internal as the host address of your database.

Dockerizing MariaDB with a Custom SQL Script in Development

Start with standard docker-compose file.

version: "3.7"
            context: .
            dockerfile: dev.dockerfile
        restart: always
            MYSQL_ROOT_PASSWORD: password
            MYSQL_DATABASE: db_name
            MYSQL_USER: sql_user
            MYSQL_PASSWORD: password
            - 3306:3306

Add dev.dockerfile:

FROM mariadb:latest

ADD init.sql /docker-entrypoint-initdb.d/ddl.sql

Finally, add your init.sql file. Let’s give all privileges to our sql_user:

GRANT ALL PRIVILEGES ON *.* TO 'sql_user'@'%';

Now, run docker-compose build, then docker-compose up.

Access from another container

If you want to access the database container from other containers, while running them separately, you can specify host.docker.internal as the address of your database.

If you’re on linux, then you need docker engine >= 20.03, and you need to add to your docker-compose file:

      - "host.docker.internal:host-gateway"

If you’re are on Mac ^^ will break your setup unless you are at least on Docker Desktop for Mac 3.3.0. See Support host.docker.internal DNS name to host · Issue #264 · docker/for-linux ( for details.

docker-compose build and deployment for Angular

In this tutorial we’ll make docker-compose files for angular and write a simple deploy script to build and deploy the images from your local machine.


Let’s start with the dev environment. First, add .dockerignore file in the root of your project:


Create .docker directory in the root of your project. Add dev.dockerfile:

FROM node:10

RUN mkdir /home/node/app && chown node:node /home/node/app
RUN mkdir /home/node/app/node_modules && chown node:node /home/node/app/node_modules
WORKDIR  /home/node/app
USER node
COPY --chown=node:node package.json package-lock.json ./
RUN npm ci --quiet
COPY --chown=node:node . .

We are using node 10 image and using a less privileged node user. npm ci “is similar to npm install, except it’s meant to be used in automated environments such as test platforms, continuous integration, and deployment — or any situation where you want to make sure you’re doing a clean install of your dependencies.” – npm-ci | npm Docs (

Create docker-compose.yml file in the root of your project:

# docker-compose
version: '3.7'
    container_name: 'your-container-name'
      context: .
      dockerfile: .docker/dev.dockerfile
    command: sh -c "npm start"
      - 4200:4200
    working_dir: /home/node/app
      - ./:/home/node/app
      - node_modules:/home/node/app/node_modules

With this setup, the node_modules will be overridden when we build a new container. Basically, this means you may have to run docker-compose run app npm install when you need to update your packages. Rebuilding the image is not going to do it for you.

For alternative setups, check out this stackoverflow answer.

In you package.json you should have the definition of the npm start command:

"scripts": {
    "ng": "ng",
    "start": "ng serve --host",
    "build": "ng build"

Run docker-compose build and docker-compose up.


Docker Setup

Let’s add production.dockerfile to .docker directory:

# Stage 1
FROM node:10 as node

RUN mkdir /home/node/app && chown node:node /home/node/app
RUN mkdir /home/node/app/node_modules && chown node:node /home/node/app/node_modules
WORKDIR  /home/node/app
USER node
COPY --chown=node:node package.json package-lock.json ./
RUN npm ci --quiet
COPY --chown=node:node . .

# max_old_space_size is optional but can help when you have a lot of modules
RUN node --max_old_space_size=4096 node_modules/.bin/ng build --prod

# Stage 2
# Using a light-weight nginx image
FROM nginx:alpine

COPY --from=node /home/node/app/dist /usr/share/nginx/html
COPY --from=node /home/node/app/.docker/nginx.conf /etc/nginx/conf.d/default.conf

Add docker-compose.production.yml file:

version: '3.7'
      context: .
      dockerfile: .docker/production.dockerfile
    image: production-image
    container_name: production-container
      - 80:80

Deploy script

We are going to ssh into our destination server and copy the updated image directly. Using a repository has a lot of advantages over this approach, but if you need something simple this will work:


# Build the image locally, upload to your production box and start the new container based on the latest image

    echo "Create an image"
    docker-compose -f docker-compose.yml -f docker-compose.production.yml build

    echo "Upload the latest image"
    echo $(date +"%T")
    docker save production-image:latest | ssh -C [email protected]_server_ip docker load

    echo "Stop and restart containers"
    ssh -C [email protected]_server_ip "echo Stopping container at $(date +'%T'); \
        docker stop production-container || true; \
        docker rm production-container || true; \
        docker container run -d --restart unless-stopped -p 80:80 --name production-container production-image:latest; \
        echo Restarted container at $(date +'%T'); \
        docker image prune -f || true"

    echo "Finished"
    echo $(date +"%T")
} || {
    # catch
    echo "Something went wrong"

We are starting a new container based on the latest uploaded image on our destination host and mapping the host port 80 to the container port 80.

Helpful resources:

Proxy Sentry JS requests to the self-hosted server behind a firewall

Tech: Rails

Problem: you have a self-hosted Sentry server behind a firewall and you want to report your frontend errors.

One way to accomplish it is by modifying Sentry dsn to send it to your backend and then proxying them to the Sentry server.

First, let’s set up a new route:

post 'frontend_errors/api/:project_id/store', to: 'frontend_errors#create'

It has to follow a specific pattern to work with the Sentry frontend library. The only thing you can change in the above is frontend_errors – pick whatever name you want. The code above will expect you to have a FrontendErrorsController.

Now, the FrontEndErrorsController needs to redirect to your actual Sentry server in the format that Sentry expects. Let’s create a new class to handle it:

class SentryProxy
  # This could be different based on your Sentry version.
  # Look into raven-sentry gem codebase if this doesn't work
  # Look for http_transport.rb files -
  USER_AGENT = "raven-ruby/#{Raven::VERSION}"

  def initialize(body:, sentry_dsn:)
    @body = body
    @sentry_dsn = sentry_dsn

  def post_to_sentry
    return if @sentry_dsn.blank? do |faraday|
      faraday.body = @body


  def sentry_connection sentry_post_url) do |faraday|
      faraday.headers['X-Sentry-Auth'] = generate_auth_header
      faraday.headers[:user_agent] = "sentry-ruby/#{Raven::VERSION}"

  def sentry_post_url
    key, url = @sentry_dsn.split('@')
    path, project_id = url.split('/')
    http_prefix, _keys = key.split('//')


  def generate_auth_header
    now =
    public_key, secret_key = @sentry_dsn.split('//').second.split('@').first.split(':')

    fields = {
      'sentry_version' => PROTOCOL_VERSION,
      'sentry_client' => USER_AGENT,
      'sentry_timestamp' => now,
      'sentry_key' => public_key,
      'sentry_secret' => secret_key
    'Sentry ' + { |key, value| "#{key}=#{value}" }.join(', ')

Now in your controller you can call it like this (assumes you can get your sentry_dsn on the backend):

def create, sentry_dsn: sentry_dsn).post_to_sentry


And to make sure your frontend is properly configured, first import Sentry frontend libraries, then initialize them using:

    dsn: `${window.location.protocol}//[email protected]${}/frontend_errors/0`});

public_key is supposed to be… your public key. You have to supply it in the dsn even if you’re getting the dsn key on the backend, otherwise, the Sentry frontend library will throw errors. 0 is the project id – the same idea, you have to supply it for the Sentry frontend to properly parse it. It doesn’t have to be real, as we’re reconstructing the Sentry url on the backend, and you can get proper keys/project id on the backend.

This should do it. Now you can configure Sentry frontend library to capture all errors, capture specific exceptions or messages.

Using the same redis instance for Rails cache and non-cache entries

Redis docs:

OS: Ubuntu 18.04 LTS

When you need to use redis for cache and non-cache entries (e.g., ActionCable, Sidekiq…), the recommended approach is to create a separate redis instance. However, if you want a simpler setup, or just can’t get another instance for reasons, there is an option to use the same redis instance for multiple uses.

We need to make sure that Redis will not evict our important data (e.g., Sidekiq), while at the same time evicting old cache entries. We could use any of the volatile eviction policies:

  • volatile-lru – remove least recently used keys where expiry is set
  • volatile-random – removes keys at random where expiry is set
  • volatile-ttl – evict keys with an expire set, and try to evict keys with a shorter time to live (TTL) first
  • volatile-lfu (starting with Redis 4.0) – evict using approximated LFU among the keys with an expire set.

To set up the eviction policy on your redis instance, edit your /etc/systemd/system/redis.conf and set these parameters:

maxmemory 100mb
maxmemory-policy volatile-lfu

Then in your Rails config update your store to use redis cache store, if not using already:

  config.cache_store = :redis_cache_store, {
    url: ENV.fetch('REDIS_URL', 'redis://localhost:6379'),
    expires_in: 24.hours

GPG Key Encryption in Ruby/Rails

To import the public key in ruby:

EncryptionError =

result, stderr, status = Open3.capture3("gpg --import #{@key_path}")
raise unless status.success?

To encrypt data with a public key for a given recipient:

pgp_encrypt_command = "gpg -ear #{recipient} --always-trust --trust-model always --local-user #{recipient} --default-key #{recipient}"

encrypted_data, stderr_data, status = Open3.capture3(pgp_encrypt_command, stdin_data: data)
    raise unless status.success?

Using Azurite with Active Storage

Install Azurite in your preferred way: npm install azurite

Install Microsoft Azure Storage Explorer

Create some directory to run azurite from: `~/azurite`

Add storage.yml configuration for azurite (using the default dev account and key):

  service: AzureStorage
  storage_account_name: 'devstoreaccount1'
  storage_access_key: 'Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw=='
  container: 'container-name'
  storage_blob_host: ''

Update development.rb to use azurite_emulator:

config.active_storage.service = :azurite_emulator

Start azurite from the directory you created for azurite: azurite --location ~/azurite --debug ~/azurite/debug.log

Start Azure Storage Explorer, connect to local emulator, and create container-name blob container – the same container name you specified in the storage.yml file.

Start uploading to Azurite.

Note for Rails 5.2

Some changes have not been backported as of this post, and you have to monkey-patch ActiveStorage file as described here – – this allows us to work with azurite locally.

If you want to use the newer azure-storage-blob instead of the deprecated azure-storage and you’re on Rails 5.2, you have to do a bit more monkey-patching – otherwise, you’ll start getting No such file to load — azure/storage.rb“:

Add two empty files: lib/azure/storage/core/auth/shared_access_signature.rb, and lib/azure/storage.rb

Add this to config/initializers/active_storage_6_patch.rb (this is the current master version of the ActiveStorage module):

require "azure/storage/blob"
require 'active_storage/service/azure_storage_service'
module ActiveStorage
  # Wraps the Microsoft Azure Storage Blob Service as an Active Storage service.
  # See ActiveStorage::Service for the generic API documentation that applies to all services.
  class Service::AzureStorageService < Service
    attr_reader :client, :container, :signer

    def initialize(storage_account_name:, storage_access_key:, container:, public: false, **options)
      @client = Azure::Storage::Blob::BlobService.create(storage_account_name: storage_account_name, storage_access_key: storage_access_key, **options)
      @signer =, storage_access_key)
      @container = container
      @public = public

    def upload(key, io, checksum: nil, filename: nil, content_type: nil, disposition: nil, **)
      instrument :upload, key: key, checksum: checksum do
        handle_errors do
          content_disposition = content_disposition_with(filename: filename, type: disposition) if disposition && filename

          client.create_block_blob(container, key, IO.try_convert(io) || io, content_md5: checksum, content_type: content_type, content_disposition: content_disposition)

    def download(key, &block)
      if block_given?
        instrument :streaming_download, key: key do
          stream(key, &block)
        instrument :download, key: key do
          handle_errors do
            _, io = client.get_blob(container, key)

    def download_chunk(key, range)
      instrument :download_chunk, key: key, range: range do
        handle_errors do
          _, io = client.get_blob(container, key, start_range: range.begin, end_range: range.exclude_end? ? range.end - 1 : range.end)

    def delete(key)
      instrument :delete, key: key do
        client.delete_blob(container, key)
      rescue Azure::Core::Http::HTTPError => e
        raise unless e.type == "BlobNotFound"
        # Ignore files already deleted

    def delete_prefixed(prefix)
      instrument :delete_prefixed, prefix: prefix do
        marker = nil

        loop do
          results = client.list_blobs(container, prefix: prefix, marker: marker)

          results.each do |blob|

          break unless marker = results.continuation_token.presence

    def exist?(key)
      instrument :exist, key: key do |payload|
        answer = blob_for(key).present?
        payload[:exist] = answer

    def url_for_direct_upload(key, expires_in:, content_type:, content_length:, checksum:)
      instrument :url, key: key do |payload|
        generated_url = signer.signed_uri(
          uri_for(key), false,
          service: "b",
          permissions: "rw",
          expiry: format_expiry(expires_in)

        payload[:url] = generated_url


    def headers_for_direct_upload(key, content_type:, checksum:, filename: nil, disposition: nil, **)
      content_disposition = content_disposition_with(type: disposition, filename: filename) if filename

      { "Content-Type" => content_type, "Content-MD5" => checksum, "x-ms-blob-content-disposition" => content_disposition, "x-ms-blob-type" => "BlockBlob" }

      def private_url(key, expires_in:, filename:, disposition:, content_type:, **)
          uri_for(key), false,
          service: "b",
          permissions: "r",
          expiry: format_expiry(expires_in),
          content_disposition: content_disposition_with(type: disposition, filename: filename),
          content_type: content_type

      def public_url(key, **)

      def uri_for(key)

      def blob_for(key)
        client.get_blob_properties(container, key)
      rescue Azure::Core::Http::HTTPError

      def format_expiry(expires_in)
        expires_in ? expires_in).iso8601 : nil

      # Reads the object for the given key in chunks, yielding each to the block.
      def stream(key)
        blob = blob_for(key)

        chunk_size = 5.megabytes
        offset = 0

        raise ActiveStorage::FileNotFoundError unless blob.present?

        while offset <[:content_length]
          _, chunk = client.get_blob(container, key, start_range: offset, end_range: offset + chunk_size - 1)
          yield chunk.force_encoding(Encoding::BINARY)
          offset += chunk_size

      def handle_errors
      rescue Azure::Core::Http::HTTPError => e
        case e.type
        when "BlobNotFound"
          raise ActiveStorage::FileNotFoundError
        when "Md5Mismatch"
          raise ActiveStorage::IntegrityError

Ad Blocking with ddwrt

This was done on Asus RT-AC68U, running DD-WRT v3.0-r41686 std (12/10/19)

Let’s start with the script. This downloads two adlists and combines them into one. Then we’re restarting the service to pick up the changes. The reason for using curl instead of wget is because wget refused to work with https on my ddwrt build 🤷.

wget -qO /tmp/mvps
curl -k|grep "^" >> /tmp/mvps
killall -HUP dnsmasq
stopservice dnsmasq && startservice dnsmasq

Go to Administration -> Commands. Paste it in there and execute “Run Commands”.

Then, use the same command but execute it as Save Startup. Why? Well, I wanted to use a cron scheduler to run the script on a regular basis, but it just refused to work. Thus, I’m just scheduling a weekly reboot, which will trigger this command to update the ad lists 🤷

All you have left is to enable DNSMasq and Local DNS in Services tab. Then in the Additional Dnsmasq options add this:


Then go to Administration->Keep Alive and schedule a weekly/monthly reboot. Although, if the cron is broken in your build, this may not work either (to check, you can ssh to your router and check the timestamp on the /tmp/mvps file). In that case, you may just have to manually rerun the script from time to time to get the latest ad list.

Update 2020: I finally switched to pfsense and using pfBlockerNG provides a much better experience. Pi-hole is another great option – also much better than tinkering with dd-wrt.

Starting with PySpark – configuration

PySpark is a pain to configure.

For this guide I am using macOS Mojave.
Spark version 2.4.0
Python 3

Start by downloading the Spark Extract wherever – can be your home directory.

Install Java SDK. Important – some later versions don’t seem to be compatible with spark 2.4.0. Version 8 seems to work-

Install pyspark: pip install pyspark

Configure your zshrc/bash_profile – depending on what shell you use:

export SPARK_PATH=~/spark-2.4.0-bin-hadoop2.7
export PYSPARK_DRIVER_PYTHON="jupyter"

export PYSPARK_PYTHON=python3
alias snotebook='$SPARK_PATH/bin/pyspark --master local[2]'

export SPARK_HOME=~/spark-2.4.0-bin-hadoop2.7

export PYSPARK_SUBMIT_ARGS="--master local[2] pyspark-shell"

export JAVA_HOME=$(/usr/libexec/java_home)

Remember to reload your console.

Now, when you enter pyspark on your console, it’ll open a notebook.

You can validate if Spark context is available by entering this in your new notebook:

from pyspark import SparkContext
sc = SparkContext.getOrCreate()