Better, Faster, and More Scalable Neo4j than ever before

Neo4j 2.2 aims to be our fastest and most scalable release ever. With Neo4j 2.2 our engineering team introduces massive enhancements to the internal architecture resulting in higher performance and scalability. This first milestone (or beta release) pulls all of these new elements together, so that you can “dial it up to 11” with your applications. You can download it here for your testing. Three of the key areas being tackled in this release are:

1. Highly Concurrent Performance

With Neo4j 2.2, we introduce a brand new page cache designed to deliver extreme performance and scalability under highly concurrent workloads. This new page cache helps overcome the limitations imposed by the current IO systems to support larger applications with hundreds of read and/or write IO requirements. The new cache is auto-configured and matches the available memory without the need to tune memory mapped IO settings anymore.

2. Transactional & Batch Write Performance

We have made several enhancements in Neo4j 2.2 to improve both transactional and batch write performance by orders of magnitude under highly concurrent load. Several things are changing to make this happen.
        • First, the 2.2 release improves coordination of commits between Lucene, the graph, and the transaction log, resulting in a much more efficient write channel.
        • Next, the database kernel is enhanced to optimize the flushing of transactions to disk for high number of concurrent write threads. This allows throughput to improve significantly with more write threads since IO costs are spread across transactions. Applications with many small transactions being piped through large numbers (10-100+) of concurrent write threads will experience the greatest improvement.
        • Finally, we have improved and fully integrated the “Superfast Batch Loader”. Introduced in Neo4j 2.1, this utility now supports large scale non-transactional initial loads (of 10M to 10B+ elements) with sustained throughputs around 1M records (node or relationship or property) per second. This seriously fast utility is (unsurprisingly) called neo4j-import, and is accessible from the command line.

3. Cypher Performance

We’re very excited to be releasing the first version of a new Cost-Based Optimizer for Cypher, under development for nearly a year. While Cypher is hands-down the most convenient way to formulate queries, it hasn’t always been as fast as we’d like. Starting with Neo4j 2.2, Cypher will determine the optimal query plan by using statistics about your particular data set. Both the cost-based query planner, and the ability of the database to gather statistics, are new, and we’re very interested in your feedback. Sample queries & data sets are welcome! Despite the strong focus on performance & scalability, we delivered some functional improvements too:

Cypher Profiling

As part of work on the Cypher planner, we extended the profiling output in the neo4j-shell. You can now choose to only EXPLAIN or fully PROFILE a query but just prefixing it with one of the keywords. And you can manually select a query planner with CYPHER 2.2-cost and CYPHER 2.2-rule prefixes.

Neo4j Browser UI

Many small improvements have been made to the UI, including panning, and the ability to kill a running Cypher query. (Query killing is also supported in the Neo4j Shell using CTRL-C.) Please explore these and tell us what you think of them. The Neo4j Browser also handles long running queries more reliably, which is especially important for imports with LOAD CSV.

Basic Authentication

We’ve received requests for a variety of authentication features, and while these are largely planned for the next (Neo4j 2.3) release, we are very pleased to introduce token-based authentication in Neo4j 2.2. This is enabled by default in Neo4j 2.2 M, which means you must either (a) explicitly disable it in conf/neo4j-server.properties (if you have assured security using another means), or (b) change your app to use the authentication token. We default setting in the milestone release is by design to ensure we get your indispensable feedback!

Some things to be aware of

As with all milestones: do not run this in production. This is an early access version to help you prepare for the upcoming production release and provide feedback on the new features. Make sure you refer to the manual for information about new features (especially token-based authentication, neo4j-import, and Cypher optimization and statistics gathering). We are eager to hear your feedback. Please post it to the Neo4j Google Group, or send us a direct email at feedback@neotechnology.com. Enjoy, and please tell us what you discover! Philip Rathle, VP Product on behalf of the Neo4j Team Want to learn more about graph databases? Click below to get your free copy of O’Reilly’s Graph Databases ebook and discover how to use graph technologies for your application today. Download My Ebook

 

Keywords:  


About the Author

Philip Rathle, VP of Products

Philip Rathle Image

Philip Rathle has a passion for building great products that help users solve tomorrow’s challenges. He spent the first decade of his career building information solutions for some of the world’s largest companies: first with Accenture, then with Tanning Technology, one of the world’s top database consultancies of the time, as a solution architect focusing on data warehousing and BI strategy.


9 Comments

Flavio says:

Interesting stuff Philip and congrats on the improvement.
Question: Is true distribution (a distribution with no single write master) in your development plan?
Thanks, cheers.

zazi says:

Congrats! The features of the 2.2 release look very promising. Can’t wait to try them out!

[…] Neo4j 2.2 Milestone 1 Release news  […]

[…] Better, Faster, and More Scalable Neo4j than ever before by Philip Rathle. […]

Very interest and awesome

Pavel says:

Great work guys! Keep it up!

[…] has recently announced the 2.2 Milestone 1 release. Among the exciting features is the improved and fully integrated […]

Ken Williams says:

Regarding the UI improvements – is it possible to try out the new UI and hit a non-new (v1.2.6) backend server?

Dino says:

Any sign of full text indexing? i.e. not having to use brute force regex and using underlying lucene?

Wynand Coetzer says:

This looks very promising. However, we need it very soon for production. Do you have any idea when it will be ready for production deployment?

Thanks

[…] Inc. is taking steps to move its Neo4J graph database into the real-time analytics world with updates that significantly improve read and write performance and the addition of a cost-based optimizer to […]

[…] Inc. is taking steps to move its Neo4J graph database into the real-time analytics world with updates that significantly improve read and write performance and the addition of a cost-based optimizer to […]

[…] and test the latest Neo4j Milestone (blog) and give us […]

[…] Neo4j 2.2 Milestone 1 Release […]

8 Trackbacks

Leave a Reply

Your email address will not be published. Required fields are marked *

Subscribe

Upcoming Event

 

From the CEO

Emil's Blog


Have a Graph Question?

Stackoverflow
Slack
Contact Us

Share your Graph Story?

Email us: content@neotechnology.com


Popular Graph Topics