Operating System Installation and Configuration

From xx network wiki
Jump to navigation Jump to search
This is a team contributed page

Before installing any xx network software, a Node or Gateway must have the operating system (OS) correctly installed and configured. This process must be done twice, once for each machine.

These instructions assume that you have a working machine that meets to exceeds the Hardware Requirements, has an active internet connection, and an empty storage drive or a drive that can be formatted.

Some Tips for Inexperienced Users

If this is your first time using a command line interface or you do not remember how to use it, the following are some tips to make using the interface a little easier.

  • In this document, anytime code is presented in a black box with a monospaced font, it means that it is command line input or output. Commands prefixed by a $ are commands to enter into your command prompt (do not include $ in the command). Any lines without that prefix are output from the system.
  • The sudo command is often prepended to commands found in these instructions. It enables commands to be run with elevated privileges. When used, the system will ask for your password to continue running the command.
  • Whenever the system asks for a password to continue, no characters will appear when typing, but type in your password, press Enter, and it will work.
  • When typing a command or path, use the Tab key to auto-complete a partially written statement.

Installing the Operating System

The xx network software has been tested only on Ubuntu Server 20.04 and instructions are only provided for that OS. Therefore, support cannot be guaranteed if a different operating system version is used, although no decisions have been made to preclude any operating systems specifically.

If you have direct access to your hardware and can install the operating system yourself, then go to the Local Hardware section. If your machine is hosted and you do not have physical access, then go to the Hosted section.

Local Hardware

If you have physical access to the machine and can install an operating system, then follow the instructions below. If you are using a VPS or hosting service,

It is recommended that your machine be connected to the internet via ethernet cable before installation.

  1. Download the Ubuntu Server install image from the Official Ubuntu website.

    Make sure to select the Server install image, not the Desktop image. The desktop version of Ubuntu can work, but it includes extra programs and processes that are unneeded and take up resources.
    Click on the link "64-bit PC (AMD64) server install image" to download the Ubuntu Server image.
  2. Next, a bootable disk with Linux needs to be created. This can be done by writing it to a DVD or, more commonly, a flash drive. Follow one of the following tutorials on how to do so depending on your current operating system and chosen media.

    The resources linked below are provided by a third-party source. The instructions may be out of date but should generally be correct.
  3. Once your flash drive or DVD is ready, follow the Tutorial on Installing Ubuntu Server.

    1. In step 6, make sure you select the first option Install Ubuntu.

    2. In step 7, make sure to configure your internet connection and get an IP address.

    3. In step 8, ensure that you select Use an Entire Disk.

      Please be sure the full amount of available space is formatted if selecting to use LVM. See Check root Disk Space
    4. In step 12, pick a server name that does not have any personal identifying information and create a strong password.

      Create a strong but memorable password. It is recommended that it is longer than 12 characters. Store this password in a safe and secure location. Never share this password with anyone.
  4. Ensure the machine has turned back on and then log in using the credentials created in the previous step. Sometimes extra text is printed to the console and you will not see the login prompt. The prompt should still be there; just type your username and press Enter to continue.

Check Internet Connection

The rest of the instructions require internet access. Follow these steps to ensure the machine is connected.

  1. Check your current local connection status and local IP address.

    This should result in a similar output to the below. The machine should have a valid local IP address.

  2. Finally, check that the machine has access to the internet by pinging another server. We chose to use xx.network, but you can use any domain.

    If the machine is connected to the internet, the output should look similar to the following example.

    If not all packets were received or packet loss is greater than 0%, then you may have internet connectivity problems.

Hosted

If your machine is hosted, then it will usually be delivered to you with the operating system preinstalled or you can select an OS to install. If you have the option, select Ubuntu 20.04.

Some hosting services deliver the server to you with access to the root account. The root account has broad permissions to modify the Linux environment and should be disabled to avoid destructive changes. These instructions will detail how to add a new non-root user account to be used as the primary account when accessing your machine.

If your user is not a root, then you can skip this section and go to Updating Software and Installing Dependencies.

  1. Open the terminal on your computer.

  2. SSH into your server using the root account.

    Terminal
    The IP address is provided to you by your hosting service.
  3. You will either authenticate using the SSH key you provided to the host or using a password.

  4. Once logged in, create a new user with a username of your choice. The username ubuntu is used as an example.

    Terminal

    It will print output similar to the following.

    Terminal
  5. It will then prompt you for a new password. Select a strong and secure password.

    Terminal
    Create a strong but memorable password. It is recommended that it is longer than 12 characters. Store this password in a safe and secure location. Never share this password with anyone.

    It will then confirm that the password has been updated.

    Terminal
  6. You will then be asked a series of questions. Press Enter after every answer. Answering them is optional. At the end, confirm that the information entered is correct by pressing Y and then Enter.

    Terminal
  7. To allow the new user to perform actions with superuser privileges using sudo, add them to the sudo group.

    Terminal
  8. In the future, use this new account to log in to your server. Connecting to your server via SSH with the root account will be disabled in future steps.

Updating Software and Installing Dependencies

To ensure all the software works correctly, the system needs to be updated. In addition, an additional dependency for the Wrapper Script to function.

  1. Before continuing, check for updates. This will print many lines about what software is being checked.

  2. Once the check is done, install the updates.

  3. Reboot the machine to ensure all updates are installed fully.

  4. Once the machine starts up, log back in.

  5. Install the Python package installer.

  6. Update the package installer.

    The output should look similar to the following.

  7. Install the boto3, pyOpenSSL, substrate-interface, and packaging dependencies. The Wrapper Script uses the first package to read commands and send logs to xx network through AWS, and the second is used to authenticate them. The third, substrate-interface, is used to interact with the Substrate node. packaging is a dependency of substrate-interface.

    The output should look similar to the following.

Configuring Local Network (Port Forwarding)

To ensure that the machine can be accessed from outside the local network, the local network gateway must be configured to allow external access to the machine on ports configured above. Three main pieces of information are needed for this part: (1) the port numbers to forward (the defaults are 11420 for cMix, 22840 for Gateway, 15974 for xx chain), and (optionally) 22 for SSH, (2) the protocol to use (TCP), and (3) the local IP address of the machine, which is retrieved below.

If your machine is hosted for you, then the port may already be open, or you need to configure it with your hosting provider.
  1. Get the local IP address of the machine.

    The local IP address will be printed; it will be in the form of 0.0.0.0. Make sure to make a note of this for the later steps.

    If the machine has multiple network interfaces or an IPv6 address, they will also appear in this list. Ensure that only the correct internal IPv4 address is used.


The following section describes how to configure the networking equipment on your network. Because of the varying equipment configurations, these instructions are generic and may not be accurate for your hardware. Please refer to the manufacturer’s instructions for more detailed and accurate information. Configuration of the network will most likely occur from a different machine on the network.
  1. To access the network gateway, get its IP address.

    This will output the following line, where the first address printed is the router IP address.

  2. Go to this IP address in a browser (on a different machine) and log in using the gateway credentials. These credentials are either set up by the network administrator or are the default credentials located on the gateway or found online.

  3. It is highly recommended to provide your machine a static local IP address or port forwarding may need to be reconfigured if your machine changes local IP addresses. Instructions for doing so are dependent on your network hardware and outside of the scope of these instructions.

  4. Locate the port forwarding options (occasionally called virtual server). These options are sometimes found under the advanced section.

  5. Forward the port for xx chain (15974) and the port for either cMix (11420) or Gateway (22840). For each, create a new entry and enter the IP address found in step 1 or the one set in step 4, set the port to the chosen ports, and select the TCP protocol.

  6. If you plan on using SSH to access your machine remotely from outside your local network, make sure to forward port 22. If you do not know if you need to access your machine outside the local network, skip this step.

    Enabling SSH access from the internet can expose your machine to unwanted access by outside parties. If you forward port 22, then make sure you follow all the security features for SSH outlined in later steps. However, if you do not need outside access to your machine, then it is recommended that you do not forward this port.
  7. Save or apply the changes.

Setting Up UFW

Uncomplicated Firewall (UFW) is the default firewall configuration tool for Ubuntu. The operating system should come with UFW already installed; the following instructions will describe how to configure and enable it.

  1. First, ensure that UFW is disabled so that it can be configured.

    This should result in the following output.

  2. Allow the port that xx chain will use to communicate over TCP. When setting a rule in UFW, you can optionally set a comment so that the rule is labeled when looking at the status of UFW.

    Port 15974 is the default port in the provided xx chain service file. A different port may be used, but it must be configured in xxnetwork-chain.service, which is downloaded in a future step.
  3. Allow the port that cMix will use to communicate over TCP. Only do this for machines running the cMix software.

    Port 11420 is the default port in the provided cMix configuration file. A different port may be used, but it must be configured in cmix.yaml, which is downloaded in a future step.
  4. Allow the port that the Gateway will use to communicate over TCP. Only do this for machines running the Gateway software.

    Port 22840 is the default port in the provided Gateway configuration file. A different port may be used, but it must be configured in gateway.yaml, which is downloaded in a future step.
  5. SSH is an internet protocol that allows you to access your server from your personal computer remotely. It is recommended that you set up SSH to make steps later in the software setup easier. But note that SSH should only be enabled with key authentication and rate-limiting to prevent unwanted parties from accessing your server. Key authentication is set up in the next section Setting Up SSH. If you do not want to use SSH, then skip this step.

    To enable SSH with rate limiting, limit port 22 over TCP. UFW will prevent access if someone attempts to connect six or more times within 30 seconds.

    Enabling SSH access without the proper security can result in unwanted access. Make sure to enable rate-limiting and key authentication.
  6. Finally, enable UFW.

    If you are connected over SSH, you may be prompted to continue, press Y, and then Enter. Note that you may be disconnected when doing so.

    If successful, then it will print the following.

Verify Firewall Configuration

This section describes how to check if the firewall is active and if the ports are correctly configured.

Print the status of UFW.

On the Node machine, the output should match the following.


On the Gateway machine, the output should match the following.


Setting Up SSH

SSH, or Secure Shell Protocol, is a network protocol that allows you to access your machine remotely. It is helpful so that you can set up and configure your machine from your personal computer. It also makes setup easier because you can copy and paste commands into your console instead of having to manually type everything.

It is recommended that you enable SSH access as it will make some steps easier to accomplish later in this manual. But note that SSH opens up a possible attack vector on your Node. If you enable SSH, then it must be rate-limited (described above in Setting Up UFW) and only accept key authentication, as explained below. If you do not want to use SSH, then skip this section.

Rate Limiting

  1. If you have not yet done so, enable rate-limiting on port 22 as described in Setting Up UFW.

  2. Ensure that UFW is correctly configured for SSH by printing its status for port 22.

    This should print all the UFW rules. Make sure that you see the two rules below in your list.

    Make sure that the action is set to LIMIT, not ALLOW
    .

Installing OpenSSH Server

If you already have SSH working and are logged in using a key pair, then skip to Configure SSH Security Options.
  1. Next, install OpenSSH Server.

    OpenSSH may have already been installed during operating system installation. If it has, then OpenSSH Server will not install. Continue with the next step.
  2. Make sure that the SSH server is running.

    This will print the status of the service. It should say active, as shown below.

Generating Key Pair

If you have already generated a SSH key pair, then you can use the same key pair again. Skip to Copying Public Key to Server.

By default, SSH has password authentication enabled, which allows you to connect to your machine using only your username, password, and your machine's IP address. SSH also offers key authentication, which involves generating an RSA key pair and is generally more secure. These instructions detail how to set up an SSH key pair between your machine and your personal machine. Note that these instructions assume that you have an OpenSSH client on your personal computer. If you are on a Linux or macOS machine, SSH is available by default. On Windows, you need to either install the native OpenSSH Client, download a third-party client such as PuTTY, or use WSL.

The instructions supplied below are for Ubuntu 20.04. For other operating systems, the processes should be similar.
  1. First, on your personal computer, open up the terminal and generate a 4096-bit key pair using ssh-keygen

    Terminal
    By default, ssh-keygen creates a 3072-bit key pair. We suggest you use 4096 bits as it is stronger than the default and the same size as the keys used in the xx network.

    This should print the following output.

    Terminal

    Press Enter to save the key pair to the .ssh/ directory in your home directory.

  2. If you already have an SSH key pair with the name id_rsa, you will see the following prompt. Enter n and press Enter. Then, skip the rest of this section and go to Copying Public Key to Server. Otherwise, continue to the next step.

    Terminal
  3. If you do not already have an SSH key pair, then you will be prompted to create a passphrase. You must set a strong passphrase; it will be used every time you use the key when connecting over SSH.

    Terminal
    Create a strong but memorable password. It is recommended that it is longer than 12 characters. Store this password in a safe and secure location. Never share this password with anyone.

    This will result in a similar output to below.

    Terminal

Copying Public Key to Server

Now that you have generated a public/private key pair, the public key needs to be copied to your Node/Gateway machine.

These instructions use ssh-copy-id to copy the public key. ssh-copy-id is available on many operating systems.

  1. First, get the address for the Node or Gateway machine.

    1. If you are on the same local network as your machine, then this will be the machine's local IP address. Get the local IP address of the machine with the following command.

      If the machine has multiple network interfaces or an IPv6 address, they will also appear in this list. Ensure that only the correct internal IPv4 address is used.
    2. If you are not on the same network as your machine, then you will need to get the machine’s public IP address by running the following command.

      ipinfo.io is a third-party service. You can use any valid IP lookup service.
  2. Next, get the username of your account on the machine. This is the username you made when installing the operating system. If you do not remember the username, use whoami to get it.

    This command should be done on the Node or Gateway machine.
  3. Next, get the ECDSA key fingerprint for the machine. The fingerprint is used to verify the Node or Gateway machine in the following steps.

    This should be done on the Node or Gateway machine.

    This will print the ECDSA key fingerprint for your machine. Make a note of it for a later step.

  4. Use ssh-copy-id to copy your public key remotely. Enter in the username and host address found above.

    Terminal
    This is back on your personal computer.
  5. If this is your first time connecting to the machine, you will see the following message. Check that the ECDSA key fingerprint matches the fingerprint found above, type yes, and press Enter. If the fingerprints do not match, then you may be connecting to the wrong system, or a malicious actor is intercepting the connection. In either case, do not attempt to continue.

    Terminal

    The host will then be added to the list of known hosts.

    Terminal
  6. Once connected, ssh-copy-id will look for the public key id_rsa.pub created above.

  7. Terminal
  8. When the public key is found, you will be prompted to enter the password for the user account on the remote machine (this is the password you created when installing the operating system).

    Terminal
  9. ssh-copy-id will then copy the public to the .ssh directory in the user's home folder on the remote machine and print the following message.

    Terminal

Testing SSH Authentication Using Keys

Once the key pair has been generated and the public key copied to the remote machine, the connection needs to be tested.

  1. From the terminal on your personal machine, connect to the Node or Gateway machine.

    Terminal
  2. If you connected without needing to put in your password for the remote user, then key authentication is working.

Configure SSH Security Options

OpenSSH server comes preconfigured to work without modification; however, several options can be configured to harden the SSH server to prevent malicious attacks. You should configure all the settings as described to ensure your machine is secure.

  1. First, create a backup of the configuration file sshd_config so that it can be restored if something goes wrong.

  2. On the Node or Gateway machine, open /etc/ssh/sshd_config in nano or your favorite text editor.

  3. Once the file is open, use the down arrow key to go to #Authentication. Uncomment the line for #LoginGraceTime by deleting the # and set the value to 30. On the next line, uncomment #PermitRootLogin and set to no.

    Setting LoginGraceTime to 30 makes unsuccessful connections close after 20 seconds. Setting PermitRootLogin to no disallows the root user from logging in over SSH.

  4. Next, go to #PasswordAuthentication. Uncomment the line by deleting the # and make sure the value is set to no. On the next line, uncomment #PermitEmptyPasswords and set to no

    Setting PasswordAuthentication to no makes it so that you cannot connect using a password; a key is required. Setting PermitEmptyPasswords to no only allows users to connect with accounts that have passwords set.

  5. Once the change is made, save the file by pressing Ctrl+X, and when prompted to save the buffer, press Y. Finally, when prompted with the file name, press Enter.

  6. To activate the change, restart the ssh service.

  7. To test that the SSH service is functioning correctly, open a new terminal session and log in.

  8. If you can successfully connect using your key, then SSH setup has been successful and you can close any extra terminal windows.

Clock Synchronization (NTP)

Commands received from the Scheduling server are time-stamped and a synchronized clock is important to interpret them properly. To do so, NTP (Network Time Protocol) must be set up and synchronized.

In Ubuntu Server 20.04, this is done through timedatectl. In most installations, it is already running and Node operators only need to check that it is correctly configured. However, the process for other operating systems may be different and it will need to be enabled.

  1. Check if the time synchronization service is running and that the clock is synchronized by entering the following command.

    This will print the following output.

    1. If System clock synchronized is set to yes and NTP service is set to active, then no further action is needed. Skip to the next section Modifying Max Number of Processes and Files. Otherwise, continue to the next step.

    2. If System clock synchronized is set to no and NTP service is set to active, then the service has had insufficient time to synchronize the clock. Skip to the next section Modifying Max Number of Processes and Files but make sure to check that the clock is synchronized before starting the Node or Gateway software. Otherwise, continue to the next step.

      Ensure the time is synchronized before starting the software in the System Services section.
    3. If System clock synchronized is set to no and NTP service is set to inactive, then the service must be manually started in the next step.

  2. Enter in the following command to get a list of time zones.

    This will print a list of time zones. Use the up key and down key to navigate the list and find the time zone of where the machine is located. Once found, make a note of the time zone, and press Q to exit.

  3. Once the correct time zone for the machine is found, use the following command to set it.

  4. Using the date command, ensure that the correct time zone was selected. If the printed time is incorrect, return to step 2 to select a new time zone.

  5. Begin the clock synchronization service by entering the following command.

  6. Ensure that the service is running by calling timedatectl again.

    If NTP service is set to active, then it has been successful. If System clock synchronized is set to yes, then this section is done. If it is set to no, then it may take up to 30 minutes for the clock to synchronize. You can continue to the following section, but make sure to check that the clock is synchronized before starting the Node or Gateway software.

Modifying Max Number of Processes and Files

By default, there is a maximum number of processes and files that can be opened at once. To prevent the Node and Gateway from encountering this limit, it must be removed for all users.

  1. To remove this limit for users, open the limits configuration file using nano or your favorite text editor.

    The following process of using the nano text editor to modify a file is used elsewhere in this document. Refer back here for detailed steps on how to use it.
  2. Once the file is open, use the down arrow key to go to the second to last line above where it says # End of file. Then, add the following four lines above that line.

  3. Once the change is made, save the file by pressing Ctrl+X, and when prompted to save the buffer, press Y. Finally, when prompted with the file name, press Enter.

  4. Once the change has been made, reboot the system.

Configuring TCP Networking Options

The TCP congestion window size limits the maximum amount of data sent out to a network after a time of little operation. The default size needs to be increased to remove a bottleneck in the xx network.

This change is necessary for the xx network because the cMix protocol transmits in short bursts. As a result, the congestion windows contract between transmissions, causing them to reopen on every transmission, significantly slowing down the network in high latency environments.

  1. First, to prevent the congestion windows from shrinking unnecessarily when the connection is idle, disable tcp_slow_start_after_idle.

  2. To make these settings persist across reboots, store them in the sysctl configuration file.

  3. Modify the TCP congestion control algorithm (CCA) to use TCP Bottleneck Bandwidth and RRT (BBR).

    This should output the following.

  4. Modify the default queuing discipline to be fq; this is required to use BBR.

    This should output the following.

  5. Apply these two options to sysctl.conf so that they persist on reboot.

  6. Optional: If you want to verify that the correct values were added to /etc/sysctl.conf, then print it to the console.

    This will print the entire file to the terminal. The last three lines should match the following.

GPU Drivers

These steps are only required for Nodes with a GPU. If setting up a Gateway, skip this step.

The cMix software requires an Nvidia RTX graphic processor and the installation of its drivers.

  1. Install the Nvidia driver.

  2. The driver relies on a dependency that enables the GUI, which can unnecessarily consume computer resources. Once the installation is complete, disable the GUI using the following command.

    The command should result in the following output.

  3. Reboot the system.

  4. Once the system reboots, log back into the computer.

Verifying the Driver Installation

  1. Check that the system has claimed the device.

    This should result in a similar output to the following.

  2. Next, check the driver and state information.

    This should result in a similar output (version numbers may differ) to the following.

    The name of the GPU and its details should match the GPU installed in your machine.