Gateway Set Up

From xx network wiki
Jump to navigation Jump to search
This is a team contributed page

Below you will find the instructions for setting up the xx Gateway with the Gateway and xx chain software for the first time. These instructions are only for the Gateway. For instructions for setting up a Node, refer to Node Set Up.

The installation instructions assume that your machine meets or exceeds the Gateway Hardware Requirements and that it is configured per the Operating System Installation and Configuration instructions.

Overview

Setting up Gateway and xx chain software is not particularly difficult, but it can take up to 2 hours due to the number of components. The general steps for setting up the xx software are:

  1. Install and configure the operating system
  2. Install and configure the Gateway and xx chain software
  3. Copy TLS credentials from the Node machine
  4. Install and configure the database
  5. Set up and start the services
  6. Backup important files
    Make sure not to skip this critical step!

Before Starting

Before installing the software, make sure that:

  1. Your Gateway machine meets or exceeds Gateway Hardware Requirements
  2. You have followed the Operating System Installation and Configuration instructions

Download Software

  1. Change the working directory to /opt/.

    $ cd /opt/

  2. Download the tarball for Gateway using curl with the following URL. The newest versions can also be found on the xx network website.

    It will show the download progress similar to below.

      % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100  155k    0  155k    0     0   262k      0 --:--:-- --:--:-- --:--:--  262k

  3. Optional: To verify that the integrity of the downloaded tarball, you can verify that its SHA256 checksum matches the checksum provided on the xx network website. Run the sha256sum to generate the checksum.

    $ sha256sum gateway.tar.gz
    If this command produces an error similar to No such file or directory, then the file name is incorrect or the file did not download correctly in the previous step. Try downloading the file again and ensure the URL is correct.

    The checksum and file name are printed, similar to below. Make sure to verify that the checksum matches the checksum on the xx network website.

    If the checksums differ, then try downloading and checking the checksum a two more times. If the checksums still differ, then contact the team.

  4. Next, extract the Gateway tarball to the /opt/ directory. Use tar to extract it with the flags -xf so that the directory structure remains intact. This will create a new /opt/xxnetwork/ directory with the extracted files inside.

    $ sudo tar -xvf gateway.tar.gz
    If this command produces an error similar to not in gzip format, then the file did not download correctly in the previous step. Try downloading the file again and ensure the URL is correct.

    It will print out all the files extracted to the newly created /opt/xxnetwork/ directory. Those files are:

    File Note
    bin/xxnetwork-chain The binary for the xx chain software.
    config/gateway.yaml The configuration file for the Gateway binary.
    cred/network-management.crt The TLS certificate used to authenticate commands sent to the Wrapper Script.
    cred/scheduling-cert.crt The TLS certificate for communicating with the Scheduling server.
    gateway-wrapper.py A management script for the Gateway binary.
    xxnetwork-chain.service systemd service file for the xx chain binary.
    xxnetwork-gateway.service systemd service file for the Gateway Wrapper Script.

  5. Next, make the binaries executable.

    $ sudo chmod +x /opt/xxnetwork/bin/*

Copy Identity Information

The Gateway TLS credential (a certificate and private key pair) and the Node's TLS certificate retrieved in the following section and Gateway's IDF generated on initial launch are extremely important to back up and store securely. Loss of these files will make it impossible to reconnect your Gateway to the network. If following the default configuration, then these files are named gateway-cert.crt, gateway-key.key, cmix-cert.crt, and gateway-IDF.json. Refer to the Lost Certificate and Important Files Policy for more information.

Each Node and Gateway is identified by its ID (contained in an ID File or IDF) and TLS certificate (a public certificate and private key pair). TLS credentials for the Node and Gateway are critical to its identity in the network.

The two TLS certificates are generated on the Node machine. The Gateway credentials (gateway-cert.crt and gateway-key.key) and the cMix public certificate (cmix-cert.crt) need to be copied to the Gateway machine.

The instructions below assume that you have SSH enabled and configured on your Gateway machine. If you have not, follow the instructions for Setting Up SSH.

  1. If you have not already, Generate the Node's and Gateway's TLS certificates on the Node machine and transfer the files to your personal computer.

  2. First, get the address for the Gateway machine.

    1. If you are on the same local network as the Gateway machine, then this will be the Gateway's local IP address. Get the local IP address of the Gateway with the following command.

      $ hostname -I
[Gateway's internal IP address]
      If the machine has multiple network interfaces or an IPv6 address, they will also appear in this list. Ensure that only the correct internal IPv4 address is used.

    2. If you are not on the same network as the Gateway, then you will need to get the machine’s public IP address by running the following command.

      $ curl -w "\n" ipinfo.io/ip
[Gateway's public IP address]
      ipinfo.io is a third-party service. You can use any valid IP lookup service.

  3. Next, you will need the username for the Gateway machine. If you do not remember your username, use whoami.

    $ whoami
[your username on the Gateway machine]

  4. Using SCP, you will remotely connect to the Gateway machine and copy the three required files gateway-cert.crt, gateway-key.key, and cmix-cert.crt.

    Terminal
    $ scp cred/gateway-cert.crt cred/gateway-key.key cred/cmix-cert.crt [Gateway username]@[Gateway public IP]:/opt/xxnetwork/cred/
    This should be back on your personal computer that has the cred/ directory copied to it.

  5. The three files should download to the cred/ directory.

    Terminal
    gateway-cert.crt                   100% 2053    60.3KB/s   00:00
gateway-key.key                    100% 3276    93.0KB/s   00:00
cmix-cert.crt                      100% 2053    69.3KB/s   00:00

  6. Check that all the correct files are in the cred/ directory using ls to list the files.

    $ ls -1 /opt/xxnetwork/cred/
cmix-cert.crt
gateway-cert.crt
gateway-key.key
network-management.crt
scheduling-cert.crt
    This should be run on the Gateway machine.

  7. Delete the Gateway's private key gateway-key.key from the Node machine.

    Ensure that the Gateway key was copied correctly to the Gateway or securely backed up before deleting it off the Node machine.
    $ rm /opt/xxnetwork/cred/gateway-key.key
    This should be done on the Node machine.

Configuring Gateway and xx chain

Gateway Configuration

Configuring the Gateway software requires modification of the YAML file retrieved in Download Software. These instructions go over the basic configuration to get cMix working in the default state. Refer to the Gateway README file to learn more.

The default gateway.yaml file only requires setting the public IP address of its Node if no changes have been made to the default file names and directory locations. However, there are other optional flags for non-standard configurations. Database information is set in a later step.

  1. Get the Node’s public IP address by running the following command on the Node machine.

    $ curl -w "\n" ipinfo.io/ip
[your IP address]
    Straight Arrow 1.svg
    Make note of this IP address for the next steps.
    This should be run on the Node machine.

    ipinfo.io is a third-party service. You can use any valid IP lookup service.

  2. The Gateway configuration file must be modified to include the Node's IP address. To do this, open gateway.yaml in nano or your favorite text editor.

    $ sudo nano /opt/xxnetwork/config/gateway.yaml

  3. Once the file is open, use the down arrow key ↓ to scroll down to the line that says cmixAddress and replace the placeholder with the one found in step 1. Make sure to include the chosen port for cMix (the default is 11420).

      GNU nano 4.8

    gateway.yaml
    Modified  
    
# The IP address of the machine running cMix that the Gateway communicates with.
# Expects an IPv4 address with a port. (Required)
cmixAddress: "[Node IP address]:11420
    Curve Arrow 2.svg
    Enter the Node IP address from the previous step and the chosen port.
    It should be in the format of 0.0.0.0:00000.    "

^G Get Help
^X Exit
^O Write Out
^R Read File
^W Where Is
^\ Replace
^K Cut Text
^U Uncut Text
^J Justify
^T To Spell
^C Cur Pos
^_ Go To Line

  4. For non-default setups and experienced users, the following options can be used.

    Note that all addresses must be IPv4; IPv6 is not currently supported.

    1. By default, the Gateway runs on port 22840; however, this can be modified using the port flag.

    2. The internal listening address of the Gateway defaults to 0.0.0.0. However, a different internal address can be set via listeningAddress. This expects an IPv4 address without a port; it uses the port from the port flag.

    3. The public IP of the Gateway, as reported to the network, is determined by pinging an external service. To override this or to avoid this external communication, the public IP and port can be manually set via overridePublicIP. If no port is set, then the port from the port flag is used. This expects an IPv4 address only.

    4. If you placed the TLS certificate and key files in a different location than the default one in the Generate TLS Credentials, then modify the location via keypath, certPath, cmixCertPath, and schedulingCertPath.

  5. Once the change is made, save the file by pressing Ctrl+X and when prompted to save the buffer, press Y. Finally, when prompted with the file name, press ↵ Enter.

Service File Configuration

The service files contain the parameters and flags to use when running the Gateway and xx chain software. For default setups, these files come correctly preconfigured and only the username needs to be modified. For additional information on the flags, refer to Management Tools.

  1. The following steps require the machine’s username. For most users, this will be the username created during the installation of the operating system. If you do not remember your username, use whoami.

    $ whoami
[your username]
    Straight Arrow 1.svg
    Make note of your username for the following steps.

  2. Open xxnetwork-gateway.service with nano or your favorite text editor.

    $ sudo nano /opt/xxnetwork/xxnetwork-gateway.service

  3. Under the [Service] heading, after the User= field, modify the username to the one found in step 1.

      GNU nano 4.8

    xxnetwork-gateway.service
    Modified  
    
[Service]
User=ubuntu
    Straight Arrow 1.svg
    Change this to your user
    account username.    
Type=simple

^G Get Help
^X Exit
^O Write Out
^R Read File
^W Where Is
^\ Replace
^K Cut Text
^U Uncut Text
^J Justify
^T To Spell
^C Cur Pos
^_ Go To Line

  4. Once the change is made, save the file by pressing Ctrl+X and when prompted to save the buffer, press Y. Finally, when prompted with the file name, press ↵ Enter.

  5. Open xxnetwork-chain.service with nano or your favorite text editor.

    $ sudo nano /opt/xxnetwork/xxnetwork-chain.service

  6. Under the [Service] heading, after the User= field, modify the username to the one found in step 1.

      GNU nano 4.8

    xxnetwork-chain.service
    Modified  
    
[Service]
User=ubuntu
    Straight Arrow 1.svg
    Change this to your user
    account username.    
Type=simple

^G Get Help
^X Exit
^O Write Out
^R Read File
^W Where Is
^\ Replace
^K Cut Text
^U Uncut Text
^J Justify
^T To Spell
^C Cur Pos
^_ Go To Line

  7. Next, the name used for telemetry of your Gateway needs to be configured.Use the down key ↓ to navigate to the line that starts with ExecStart and, after /opt/xxnetwork/bin/xxnetwork-chain, add the flag --name followed by your desired name for the Gateway. While not required, you may want to make the name unique so you can differentiate your Gateway from other Nodes and Gateways. If you do not set a name, then a random name will automatically be assigned to your Gateway.

      GNU nano 4.8

    xxnetwork-chain.service
    Modified  
    

ExecStart=/bin/bash -c "/opt/xxnetwork/bin/xxnetwork-chain --name [Gateway's name]
    Curve Arrow 1.svg
    Put your selected name for your Gateway.     --telemetry-url 'wss://telemetry.polkadot.io/submit/ 0' --base-path /opt/xxnetwork/db --port 15974 --ws-port 63007  >> /opt/xxnetwork/log/chain.log 2>&1"

^G Get Help
^X Exit
^O Write Out
^R Read File
^W Where Is
^\ Replace
^K Cut Text
^U Uncut Text
^J Justify
^T To Spell
^C Cur Pos
^_ Go To Line

  8. Once the change is made, save the file by pressing Ctrl+X and when prompted to save the buffer, press Y. Finally, when prompted with the file name, press ↵ Enter.

Database Set Up

The Gateway software requires a PostgreSQL database to store key pairs with clients. The following instructions detail how to install PostgreSQL, set up the required database, and Verify the Database.

  1. Install PostgreSQL and its dependencies.

    $ sudo apt install -y postgresql-client postgresql postgresql-contrib

  2. Once the installation is complete, enable the PostgreSQL service.

    $ sudo update-rc.d postgresql enable

  3. Next, start the service.

    $ sudo service postgresql start

  4. Create a database user with the username cmix, which is used to access the database.

    $ sudo -u postgres createuser --createdb --pwprompt cmix

  5. You will be asked to set a password for the cmix user. Create a long and secure password but note that it will only be saved into the Gateway configuration file in the following steps and you will not need to remember it once everything is configured.

    Enter password for new role:
    Enter it again:
    Straight Arrow 1.svg
    Save this password for the next steps.
    Never store this password digitally except when putting it into the Gateway configuration file. Never provide this password to anyone.

  6. Create the required database with the name cmix_gateway.

    $ sudo -u postgres createdb -O cmix cmix_gateway
    The following output: could not change directory to "/current/working/directory": Permission denied can safely be ignored.

  7. Next, modify the Gateway configuration file to use the newly generated password in step 5 above. To do this, open gateway.yaml in nano or your favorite text editor.

    $ sudo nano /opt/xxnetwork/config/gateway.yaml

  8. Under the field dbPassword, enter in the new password. If the database was created with a different username or database name, then update those values too.

      GNU nano 4.8

    gateway.yaml
    Modified  
    
# Database connection information. (Required)
dbName: "cmix_gateway"
dbAddress: "0.0.0.0:5432"
dbUsername: "cmix"
dbPassword: "[password for database]
    Curve Arrow 2.svg
    Enter in the password
    created above.    "

^G Get Help
^X Exit
^O Write Out
^R Read File
^W Where Is
^\ Replace
^K Cut Text
^U Uncut Text
^J Justify
^T To Spell
^C Cur Pos
^_ Go To Line

  9. Once the change is made, save the file by pressing Ctrl+X and when prompted to save the buffer, press Y. Finally, when prompted with the file name, press ↵ Enter.

Verify the Database

The following steps will describe how to ensure the database was created correctly. This section is optional but highly recommended.

  1. Log in with the postgres account.

    $ sudo su postgres

  2. Run the PostgreSQL client server via psql.

    $ psql
psql (10.12 (Ubuntu 10.12-0ubuntu0.18.04.1))
Type "help" for help.
    The command prompt has changed from $ to #.

  3. Once at the postgres prompt, enter in \l to get a list of databases (do not enter the postgres-# part).

    postgres-# \l

    Ensure that the correct databases with the correct owners show up.

                                   List of databases
     Name     |  Owner   | Encoding | Collate |  Ctype  |   Access privileges
--------------+----------+----------+---------+---------+-----------------------
 cmix_gateway | cmix     | UTF8     | C.UTF-8 | C.UTF-8 |
 postgres     | postgres | UTF8     | C.UTF-8 | C.UTF-8 |
 template0    | postgres | UTF8     | C.UTF-8 | C.UTF-8 | =c/postgres          +
              |          |          |         |         | postgres=CTc/postgres
 template1    | postgres | UTF8     | C.UTF-8 | C.UTF-8 | =c/postgres          +
              |          |          |         |         | postgres=CTc/postgres
(4 rows)

  4. Enter \q to exit the command line.

    postgres-# \q

  5. Enter exit to return to your prompt.

    $ exit

System Services

This section describes how to install the service files for Gateway and xx chain.

The following steps will start the Gateway and xx chain software and will have it join the xx network. If you do not want to do so currently, then do not perform the following steps until you are ready.

  1. Before starting services, ensure that the clock is synchronized by entering the following command.

    $ timedatectl status

    This should result in the following output.

                   Local time: Thu 2021-07-08 17:03:01 PDT
           Universal time: Fri 2021-07-09 00:03:01 UTC
                 RTC time: Fri 2021-07-09 00:03:02
                Time zone: America/Los_Angeles (PDT, -0700)
System clock synchronized: yes
              NTP service: active
          RTC in local TZ: no

  2. If System clock synchronized is set to yes and NTP service is set to active, then proceed to the next step. Otherwise, refer back to the Clock Synchronization (NTP) section.

  3. Next, soft link the systemd service file(s) to the systemd directory /etc/systemd/system/.

    For Gateway:

    $ sudo ln -s /opt/xxnetwork/xxnetwork-gateway.service /etc/systemd/system

    For xx chain:

    $ sudo ln -s /opt/xxnetwork/xxnetwork-chain.service /etc/systemd/system

  4. Reload the systemd service files.

    $ sudo systemctl daemon-reexec

  5. To ensure that your user has the correct permissions, run the following command. Make sure to replace both instances of [user] with your username.

    $ sudo chown [user]:[user] -Rv /opt/xxnetwork
    You can use whoami to get your username.

  6. Next, enable the service(s) for the appropriate binary.

    For Gateway:

    $ sudo systemctl enable xxnetwork-gateway.service

    This should result in the following output.

    Created symlink /etc/systemd/system/multi-user.target.wants/xxnetwork-gateway.service → /opt/xxnetwork/xxnetwork-gateway.service.

    For xx chain:

    $ sudo systemctl enable xxnetwork-chain.service

    This should result in the following output.

    Created symlink /etc/systemd/system/multi-user.target.wants/xxnetwork-chain.service → /opt/xxnetwork/xxnetwork-chain.service.

  7. Then, start the services.

    For Gateway:

    $ sudo systemctl start xxnetwork-gateway.service

    For xx chain:

    $ sudo systemctl start xxnetwork-chain.service

Verify the Service has Started

  1. First, check the status of the service to ensure it is running.

    For Gateway:

    $ sudo systemctl status xxnetwork-gateway.service

    This should result in the following output. Press Q to exit the output.

    ● xxnetwork-gateway.service - Job that starts the gateway Wrapper service
     Loaded: loaded (/etc/systemd/system/xxnetwork-gateway.service; enabled; vendor preset: enabled)
     Active: active (running) since Fri 2021-07-09 19:42:17 UTC; 15min ago
   Main PID: 8867 (python3)
      Tasks: 3 (limit: 9513)
     Memory: 94.7M
     CGroup: /system.slice/xxnetwork-gateway.service
             ├─8867 python3 /opt/xxnetwork/gateway-wrapper.py --wrapper-log /opt/xxnetwork/log/gateway-wrapper.log --cmd-dir /opt/xxnetwork/log/gateway-cmd --gateway --binary-path /opt/xxne>
             ├─8879 python3 /opt/xxnetwork/gateway-wrapper.py --wrapper-log /opt/xxnetwork/log/gateway-wrapper.log --cmd-dir /opt/xxnetwork/log/gateway-cmd --gateway --binary-path /opt/xxne>
             └─8880 python3 /opt/xxnetwork/gateway-wrapper.py --wrapper-log /opt/xxnetwork/log/gateway-wrapper.log --cmd-dir /opt/xxnetwork/log/gateway-cmd --gateway --binary-path /opt/xxne>

Jul 09 19:42:17 emira systemd[1]: Started Job that starts the gateway Wrapper service.

    For xx chain:

    $ sudo systemctl status xxnetwork-chain.service

    This should result in the following output. Press Q to exit the output.

    ● xxnetwork-chain.service - Job that starts the xx network chain binary
     Loaded: loaded (/etc/systemd/system/xxnetwork-chain.service; enabled; vendor preset: enabled)
     Active: active (running) since Fri 2021-07-09 19:53:03 UTC; 4min 29s ago
   Main PID: 9491 (bash)
      Tasks: 27 (limit: 9513)
     Memory: 238.4M
     CGroup: /system.slice/xxnetwork-chain.service
             ├─9491 /bin/bash -c /opt/xxnetwork/bin/xxnetwork-chain --name [Gateway's name] --telemetry-url 'wss://telemetry.polkadot.io/submit/ 0' --base-path /opt/xxnetwork/db --port 15974 --ws-port 63007 >> /opt/xx>
             └─9492 /opt/xxnetwork/bin/xxnetwork-chain --name [Gateway's name] --telemetry-url 'wss://telemetry.polkadot.io/submit/ 0' --base-path /opt/xxnetwork/db --port 15974 --ws-port 63007

Jul 09 19:53:03 emira systemd[1]: Started Job that starts the xx network chain binary.

  2. Next, check that the process is running. If the commands result in the expected output, then the Gateway and xx chain processes are running.

    $ ps -A | grep xxnetwork

    This will print a list of running xx network processes. If xxnetwork-gatew appears on the list, then the Gateway process is running. If xxnetwork-chain appears on the list, then the xx chain process is running.

      11883 ?        00:00:12 xxnetwork-chain
  13624 ?        00:00:18 xxnetwork-gatew
    The PID (the first number printed) and the elapsed time will be different.

Backup Certificates and Important Files

Ambox warning pn.svg ATTENTION Ambox warning pn.svg


Lost Certificates and Identity Cannot Be Recovered

Node operators must backup the TLS credentials for their Node and Gateway and the IDF for the Node. Loss of these files will irreparably disconnect a Node and Gateway from the network.

These files are sensitive as they identify your Node and Gateway uniquely on the network. Therefore, anybody who gains access to these files can appear on the network as your Node or Gateway.

It is extremely important to keep these files secure as any third-party access can compromise your Node and Gateway’s identity.

If you already backed up these files when setting up the Node, then skip to Delete Copied Files.

There are five important files that need to be backed up; Node's TLS credentials and IDF and Gateway's TLS credentials. These files are unrecoverable. Loss of these files will lead to your Node or Gateway being unable to participate in the network. Therefore, every Node Operator must backup these files to a secure location not on the same machine. It is recommended that these files be backed up to multiple secure locations.

Backing Up Files

  1. During the software setup, you should have copied the cred/ directory to your personal computer. Find that directory. It should include the following files.

    File Default Name Default Location on Server
    1. cMix’s public certificate cmix-cert.crt /opt/xxnetwork/cred/
    2. cMix’s private key cmix-key.key /opt/xxnetwork/cred/
    3. cMix’s ID file (IDF) cmix-IDF.json /opt/xxnetwork/cred/
    4. Gateway’s public certificate gateway-cert.crt /opt/xxnetwork/cred/
    5. Gateway’s private key gateway-key.key /opt/xxnetwork/cred/

  2. Move these five files to a safe, secure, and reliable location.

Delete Copied Files

Once the files are backed up, you need to delete the unsecured copies that you downloaded from the Node machine.

  1. Delete the files in the cred directory from your personal machine. You can do so using your computer's file browser or by using the command below.

    Terminal
    $ rm -r cred/
    Run this on your personal computer

    This will then print a list of all removed files.

    Terminal
    removed 'cred/gateway-cert.crt'
removed 'cred/gateway-key.key'
removed 'cred/cmix-cert.crt'
removed 'cred/network-management.crt'
removed 'cred/cmix-key.key'
removed 'cred/cmix-IDF.json'
removed 'cred/scheduling-cert.crt'
removed 'cred/session-keys.json'
removed directory 'cred/'
Retrieved from "https://xxnetwork.wiki/index.php?title=Gateway_Set_Up&oldid=8361"