Neo4j + Docker + CoreOS, a (Gentle) Guided Introduction


Written by Nick Manning, co-founder of Swig, a community for drinking enthusiasts powered by Neo4j, originally posted on his blog.

Neo4j + Docker + CoreOS, a (Gentle) Guided Introduction

I’d like to share with everyone how awesome CoreOS is for leveraging Docker. We use this set up for our search proxy here at Swig. (a community for drink enthusiasts for iOS and Android)

Neo4j is used as our database and as stated in my previous blog post, it works brilliantly as a startup-friendly (read: flexible) database.

(What’s a search proxy? It’s our term for a service that runs alongside our main API that tracks drink searches in the background and scrapes all kinds of metadata for the drinks we don’t yet have in our database.)

In this example, I’ll have a Neo4j database instance running in a Docker container. Now, I used to use Ubuntu for running our Docker containers but soon realized CoreOS made everything easier. CoreOS is a lean and mean flavor of linux that’s especially catered to Docker users offering easy scalability. It’s fast and their site has great documentation.

1. Droplet, Anyone?

I’m going use Digitial Ocean’s $5 droplet for this example. Sign up on their site and create a droplet (instance) with CoreOS as the operating system (screenshot below). I recommend setting up an SSH key ahead of time to use for sign in.



2. Me Like CoreOS

Sign into your droplet with ssh core@<IP of your droplet> (note that I registered my SSH key ahead of time for my digitial ocean account, so there’s no password).

Apart from its built-in Docker and easy clustering features, CoreOS offers something called systemd which makes working with Docker much more easy. More on this later.

3. Installing Neo4j 70 Seconds with Docker

Because Docker comes pre-installed with CoreOS, we will now download a Neo4j Docker image that I created myself.

docker pull seenickcode/neo4j-community

This “pulls” a prebuilt image of an entirely independently runnable virtual instance of linux with Java and the latest Neo4j Community Edition installed. If you’re interested, my Docker recipe is here.

While it downloads..

Interested in learning more about Docker? Here’s a concise overview is here. Or else, if you just want to follow along with this tutorial, feel free to read up on the following basic Docker functions like: ‘images’, ‘ps’, ‘build’ and ‘run’here. Also, James Turnbull wrote a really solid book on Docker.

Honestly, it took me some time to get my head around Docker. Even after learning it, it takes time to get used to leveraging it in a realistic environment. Yet I think the payoff of simplicity, scalability and reliability it gives is worth the investment.

4. “Your Very Own, Cheese Pizz..”, er, Neo4j Instance

Now run..
docker run -d --name neo4j --privileged -p 7474:7474 -p 1337:1337 -v /home/core:/var/lib/neo4j/data seenickcode/neo4j-community  
What this does is it runs the seenickcode/neo4j-community image you downloaded, gives it a casual name neo4j (--name), exposes some ports to the outside world (-p) and finally maps (-p) the Neo4j data directory to your CoreOS home path (/home/core), so you can back it up on your own or load your own graph.db directory.

Now since we exposed Neo4j’s port 7474, we should be able to use the Neo4j Data Browser now at <yourIP>:7474.



Create a node or two with Neo4j: CREATE (n:Thing)

Now since we mapped a Docker volume to /home/core/, we should now see a graph.db directory there, cool.

5. Systemd Makes Docker a Lot Easier to Use

CoreOS’s systemd lets us control which Docker containers start up, in what order as well as any other commands we want pre and post xyz. Also, if a container exits with an error, it can automatically restart that container.

In this example, we’ll specify our own systemd service for the Neo4j container we have. Here’s what it looks like:
[Unit]
Description=Neo4j Community  
//After=example-other-service-of-mine.service
Requires=docker.service

[Service]
TimeoutStartSec=0  
ExecStartPre=-/usr/bin/docker kill neo4j  
ExecStartPre=-/usr/bin/docker rm neo4j  
ExecStart=/usr/bin/docker run -d --name neo4j --privileged -p 7474:7474 -p 1337:1337 -v /home/core:/var/lib/neo4j/data seenickcode/neo4j-community

[Install]
WantedBy=multi-user.target  
This defines a service called neo4j.service and upon starting it, we’ll kill our running neo4j Docker container, remove it, then run it again.

You can create this as /etc/systemd/system/neo4j.service

You’ll then have to enable this using systemctl via sudo systemctl enable neo4j.service

Now ensure Neo4j is stopped (note that rm doesn’t remove the image itself, only the recently run container) via:
docker kill neo4j  
docker rm neo4j  
Then to start our systemd service sudo systemctl start neo4j.service

Just to be sure, run docker ps to ensure Neo4j was actually started.

We can tail our service via journalctl -f -u neo4j.service

6. Final Notes

Definitely check out the technologies Docker and CoreOS offer. I think they represent a sophisticated, forward-thinking approach to server/cluster management and it make sense to invest the time in getting familiar with them.

If you don’t know much about Neo4j, you will certainly one time or another want to get familiar with it in the next year or so. It’s a flexible, sophisticated NOSQL database that really excels in persisting and querying modern data. It’s becoming less and less of a niche “database for only graph data” and more of a solid choice for any NOSQL need.

Questions? Comments? Get in touch with me on Twitter via @seenickcode

Check out our new app, Swig, for iOS and Android. It’s a community for drink enthusiasts that runs on Neo4j!