Update: We’ve moved to Discord. You can join us here.
Our colleague Andreas, who loves Slack and brought it into our company, suggested the other day that we could build a Slack and Neo4j integration to demonstrate how useful a graph database backend would be. And, of course, how much fun.
In the process of making the integration, we came up with some surprisingly useful benefits. More below.
Building Blocks
As it was only midnight we, the amazing Nicole and Michael, decided to put something together. We set up a Neo4j 2.2.3 instance in the cloud. Then, we created a Python app for our Slack-Neo4j server and pushed it to GitHub. The application usesweb.py for the webapp, requests to access the Slack APIs and py2neo to talk to Neo4j.
Next we pushed it to Heroku to make it available publicly so that Slack could connect to it.
You have to provide environment variables to your Neo4j server, your Slack API token and the team token configured with your slash command. Find the details in the project readme.
Slack Slash Command
With all that complete, we could then set up a slash command. For our integration, the ideas came from Andreas:/graph import– import users, channels and membership into Neo4j/graph cypher MATCH ... RETURN– execute read only cypher statement and return the results/graph– provide an overview of the data that’s in the database
POST payload and checking the team token, we then got the first word of the text parameter as “command” to dispatch on.
Getting Data from Slack into Neo4j
For the integration with Neo4j we sent Cypher statements to Neo4j using py2neo’s APIs. Example below:from py2neo import Graph
graph = Graph(os.environ.get('NEO4J_URL'))
graph.cypher.execute("MATCH (u:User)-[:MEMBEROF]->(c:Channel) return u.screenname, c.name")
requests. We then passed the JSON response directly as parameters to a Cypher statement to create the graph structure in Neo4j.
res = requests.get("https://slack.com/api/channels.list?token={}".format(token))
query = """
UNWIND {channels} AS channel
MERGE (c:Channel {id:channel.id}) ON CREATE SET c.name = channel.name
"""
graph.cypher.execute_one(query, res.json())
/graph import channels
slackbot: 115 users uploaded. Only you can see this message
/graph import users
slackbot: 117 channels uploaded Only you can see this message
Graph All the Slack Things
And to show you that it worked, here is a graph of our Slack universe:
And here are some queries that show the most prolific people:
/graph cypher match (u:User)-->() return u, count(*) as memberships order by memberships desc limit 3`
slackbot:
| u | memberships
---+--------------------------------------------------------------+-------------
1 | (n200:User {fullname:"Michael",id:"U02HVJ36",username:"mh"}) | 64
2 | (n151:User {fullname:"Chris",id:"U0KLMP5X",username:"cl"}) | 37
3 | (n210:User {fullname:"Philip",id:"U02HDEF0EX",username:"pr"}) | 37
Recommendations
Finally, the biggest surprise of all: We wanted to recommend new channels to people. We used traditional collaborative filtering for this concept of “channels of your colleagues that are not yet your channels”. But we also filter out prolific users and channels so that they don’t distort the picture./graph cypher
MATCH (c:Channel) with toInt(count(*)*0.618) as channel_cutoff
MATCH (u:User) with toInt(count(*)*0.618) as user_cutoff, channel_cutoff
MATCH (u:User {username:"laeg"})-[:MEMBER_OF]->(c:Channel)
<-[:MEMBER_OF]-(coll:User)-[:MEMBER_OF]->(reco:Channel)
WHERE size((c)<--()) < channel_cutoff
AND size((reco)<--()) < channel_cutoff
AND size((coll)-->()) < user_cutoff
AND NOT (u)-[:MEMBER_OF]->(reco)
RETURN reco.name, count(*) AS freq
ORDER BY freq DESC
LIMIT 5;
slackbot:
| reco.name | freq
--+---------------------+------
1 | feedback | 218
2 | dev-team | 179
3 | sales_marketing | 161
4 | marketing | 142
5 | cypher-the-language | 125
