What am I up to?

I recently became interested in cryptocurrency mining. These are my notes on setting up a server. Also, I have been very disappointed at the level of documentation from AMD for their OpenCL implementation on Linux. Specifically, for the Polaris (eg, RX-470) and Vega cards you do not want to install the APP-SDK. Instead, you want to install the amdgpu-pro driver for your card along with the Ubuntu packages opencl-headers and ocl-icd-opencl-dev.

I will be referring to two machines in these instructions. The first is a computer that is already setup for completing tasks (the workstation), the second is a computer that we will be configuring (the server). My workstation runs Ubuntu Mate 16.04, so these instructions will assume a Unix-like environment.

PS - I spend a lot more time writing software than administrating servers. If I have made any horrible errors please let me know: taborkelly@gmail.com.

Hardware

I have the following hardware which I will be setting up to mine XMR. This is my server:

  • ASUS Prime B250M-A motherboard
  • Intel Core i5-7500 CPU
  • 16GB DDR4 RAM
  • EVGA SuperNOVA 1000 G2 power supply
  • SAPPHIRE Radeon RX-470 8GB Mining edition GPU
  • An SSD that I had sitting around

Note: this isn't exactly the hardware that you would normally choose to setup a "mining rig." In particular, this is way more RAM than you need. Also, this is more power supply than you need for running one RX-470. In fact, all setup and mining this hardware is only using 154W.

Note2: CPU mining of XMR is still profitable so we will be using both the CPU and GPU to mine XMR.

Install Ubuntu Server 16.04

We are going to be running Ubuntu server 16.04.03 as recent versions of AMD's OpenCL implementation do not require an X Server to run. So, go ahead and download ubuntu-16.04.3-server-amd64.iso if you do not already have it:

wget http://releases.ubuntu.com/16.04/ubuntu-16.04.3-server-amd64.iso

I personally prefer to copy ISOs to installation media (thumb-drives) using dd. So, plug in your USB thumb-drive. The USB thumb-drive will most likely be auto-mounted and you will need to unmount it. There are lots of ways to figure out which partition(s) to unmount, one ways is to tail /var/log/syslog while you plug in the USB thumb-drive:

tail -f /var/log/syslog | grep Mounted
Jul 11 18:24:50 workstation udisksd[2429]: Mounted /dev/sdb1 at /media/your_username/foo on behalf of uid 1000

Now unmount the partition(s) and use dd to write the iso file to the thumb-drive. Note that while we unmount the partitions /dev/sdAN (where A is a letter and N is a number) we dd to the device /dev/sdA. Be extra careful to dd to the correct device on your system or you could end up overwriting one of your hard-drives.

umount /dev/sdb?
sudo dd bs=4M if=ubuntu-16.04.3-server-amd64.iso of=/dev/sdb
sync

I won't walk you through installing Ubuntu Server as it is pretty self explanatory. However, I would suggest that:

  1. When asked about automatic security updates, choose to take automatic security updates.
  2. When asked what extra software to install, select OpenSSH Server.

Upon first booting into your new server, bring the system completely up to date:

sudo apt-get update
sudo apt-get dist-upgrade

I strongly suggest that you install your ssh public key on the server and disable password based logins. If you do not have an ssh public key you can read about creating one here. On your workstation:

ssh-copy-id server

Now, on the server, edit /etc/ssh/sshd_config and add the line PasswordAuthentication no. This will prevent future password based logins.

Install AMD's OpenCL environment

First, make sure to reboot the server so that if you installed a new kernel during the system update that the AMD installer compiles for the new kernel version.

sudo reboot

Next, using your workstation, download the latest driver for your video card from AMD's site.

I like to do my work outside of my home directory, so I'm going to setup a directory on the server at /work.

cd /
sudo mkdir work
sudo chown your_username:your_username work/

On your workstation, copy over the AMD driver installer:

scp amdgpu-pro-17.40-492261.tar.xz server:/work

On the server, unpack and install the AMD driver. Make sure to add the --compute to ./amdgpu-pro-install so that you don't pull in X.

cd /work
tar xvf amdgpu-pro-17.40-492261.tar.xz
cd amdgpu-pro-17.40-492261/
./amdgpu-pro-install --compute
# Add yourself to the video group
sudo usermod -a -G video $LOGNAME
# Install the generic opencl headers because amdgpu-pro doesn't include them
# DO NOT install the AMD-APP-SDK, it WILL NOT peacefully coexist with amdgpu-pro
# and AMD does a bad job of pointing this out.
sudo apt-get install opencl-headers
sudo apt-get install ocl-icd-opencl-dev
# Finally, reboot again so that the kernel module gets loaded correctly.
sudo reboot

At this point you should be able to log back in and verify that OpenCL is working correctly:

/opt/amdgpu-pro/bin/clinfo

Install xmr-stak to simultaneously mine Monero on the CPU and the GPU

We are going to be building xmr-stak v2.0.0 from source.

cd /work
# We are going to build xmr-stak from source, so clone the source code repo
git clone https://github.com/fireice-uk/xmr-stak.git
cd xmrstak/
git checkout v2.0.0
# install some build tools.
sudo apt install libmicrohttpd-dev libssl-dev cmake build-essential libhwloc-dev
mkdir build && cd build
# We don't need CUDA to mine on AMD hardware
cmake -DCUDA_ENABLE=OFF ..
make

You can now run xmr-stak and it will walk you through configuration:

cd bin
./xmr-stak

Go ahead and exit xmr-stak with ^C after it is configured. Note: your general configuration will be placed in config.txt, your CPU specific configuration will be placed in cpu.txt, and your GPU specific configuration will be placed in amd.txt. Note, I messed around with the amd.txt config and got ~5% increase in GPU mining. I'm currently getting 612 H/s on a stock RX-470:

  { "index" : 0,
    "intensity" : 1024, "worksize" : 8,
    "affine_to_cpu" : 3
  }

Next let's make a startup script for optimum xmr-stak execution. Place the following file in /work/xmr-stak/build/bin/start.sh:

#!/bin/bash
sysctl -w vm.nr_hugepages=128
cd /work/xmr-stak/build/bin/
nice --20 ./xmr-stak

Make sure that start.sh is executable:

chmod a+x start.sh

However, setting vm.nr_hugepages and running nice with a negative value are privileged operations, so you will need to run the script with sudo. This presents a problem because we want to automatically run it at boot. So, let's edit the sudoers file so that you don't need to enter a password to sudo /work/xmr-stak/build/bin/start.sh:

sudo visudo

Now, add the following lines:

# Don't prompt for password
your_username ALL = (root) NOPASSWD: /work/xmr-stak/build/bin/start.sh

Finally, we can setup an init.d script to start xmr-stak in a screen session after every reboot. Copy the following file to /etc/init.d/xmr-stak.sh:

#!/bin/sh
### BEGIN INIT INFO
# Provides:          xmr-stak
# Required-Start:    $local_fs $network $named $time $syslog
# Required-Stop:     $local_fs $network $named $time $syslog
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Description:       <DESCRIPTION>
### END INIT INFO

start() {
    echo 'Starting service xmr-stak' >&2
    su your_username -c "screen -dmS xmr-stak sudo /work/xmr-stak/build/bin/start.sh" &
    echo 'Service started xmr-stak' >&2
}

stop() {
    echo 'Stopping service xmr-stak' >&2
    su your_username -c "screen -S xmr-stak -X stuff ^C"
    echo 'Service stopped xmr-stak' >&2
}

uninstall() {
    echo "uninstall() "
}

case "$1" in
    start)
      start
      ;;
    stop)
      stop
      ;;
    uninstall)
      uninstall
      ;;
    retart)
      stop
      start
      ;;
    *)
      echo "Usage: $0 {start|stop|restart|uninstall}"
esac

Now make sure that xmr-stak.sh is executable and use update-rc.d to configure it to be executed on boot:

sudo chmod a+x xmr-stak.sh
sudo update-rc.d xmr-stak.sh defaults
sudo update-rc.d xmr-stak.sh enable

One more time, reboot to check your configuration. Now, when you login, you should already see your screen session running:

screen -list
There is a screen on:
    1101.xmr-stak   (12/02/2017 09:22:31 PM)    (Detached)
1 Socket in /var/run/screen/S-your_username.

This is cool for a couple reasons:

  1. After a power outage (if your BIOS is set correctly), your server will reboot and immediately start mining again.
  2. You can setup your firewall to allow you to SSH in and check your server from anywhere in the world. To view your xmr-stak status use screen -x xmr-stak. However, when you are done checking xmr-stak make sure to detach the screen session with ^A+D (CTRL-A followed by D), not ^C, as the ^C will be delivered to xmr-stak and it will terminate.

Final notes

We now have a server running xmr-stak on both the CPU and GPU. On my hardware I get ~215H/s on the CPU and ~610H/s on the (stock, not overclocked) AMD RX-470 GPU. Note, I tried Claymore's Cryptonote miner, but only got ~345H/s. Don't forget to pay your taxes!