If you haven’t heard already, the Neo4j 2.3 Milestone 2 release is now available.
(Full disclosure: This milestone release is for development and experimentation only since not all features are in their finalized form. Click here to download the fully stable version of Neo4j 2.2.2.)
As we continue to develop Neo4j 2.3, we want to make Neo4j more scalable and efficient in handling your graph data. That’s why the Neo4j engineering team put a lot of effort into several areas that more efficiently use existing resources.
Not all of these features detailed below are fully operational in the Milestone 2 release, but here’s your first glance at what’s to come.
Highly Scalable Off-Heap Graph Cache
Neo4j 2.3 concludes a 2-year journey that started with Neo4j 2.1, to move the graph cache fully off of the JVM heap. One major facet of the 2.3 release is the replacement of the now unnecessary second-level object cache.
As Neo4j’s implementation (especially Cypher) moves closer to the metal, it creates fewer and fewer objects, putting less strain on the garbage collector. The new page cache introduced in Neo4j 2.2 made it possible to more efficiently map small fragments of store-files into memory and lessened the need for a higher level cache.
With this change, there is much more memory available to map your store files into memory. Also, with the lower strain on the garbage collector, Neo4j and clusters of Neo4j machines run more stable and with fewer long pause times.
The new page cache also provides significant (up to 2-3x) improvements in concurrent read scaling. You should be able to reduce your heap settings and increase the page-cache settings by the same amount.
Please help us test this big change (which allowed us to delete 13,000 lines of code) by running more intensive test-workloads on Neo4j 2.3.M02 and report back how Neo4j behaves for you in terms of GC-pauses, performance and cluster stability.
Mac Installer
’Nuff said.
Cypher Cost-Based Optimizer Improvements
The optimizer is now better at choosing the most efficient query plan. This primarily impacts large or complex read queries, improving performance.
Please note, that you can always force the planner to revert to the previous rule-based planner with the hint: CYPHER planner=rule and remember to check your query plans visually by prefixing slow queries with EXPLAIN or PROFILE.
Compiled Runtime
As part of the memory and runtime efficiency improvements, we are moving the Cypher runtime engine from its current form, which uses Scala and interpreted execution plans, to a new and much more efficient engine that compiles queries down to bytecode. This bytecode is then executed and optimized by the Hotspot JVM.
Currently the compilation phase only works with very simple queries but over time, it will cover a larger scope with further milestone releases. If the bytecode compiler is unable to compile the query, it simply falls back to the existing interpreted runtime.
You’ll notice some new Cypher hints to select the runtime, alongside the existing hints that can be used to specify the planner:
CYPHER planner=cost|rule runtime=compiled|interpreted
MATCH (n) return n
The profile does not yet show what runtime is used, but you can see what runtime it uses by inspecting the code-view (more below) of a PROFILE execution and examining the http-response.
UX Improvements
We also have a number of user experience improvements in this milestone release. One example is that now you can pin guides, especially ones that continuously execute Cypher queries as part of the guide. These include examples like the movie graph (:play movie graph), Northwind database import (:play northwind graph), query templates (:play query template) and history (:history).
Another key UX improvement is code-view. If you ever wondered what the browser executes under the hood, when you execute a query, code-view is for you. It is a new view in the result pane of your actions, indicated by a </> icon.
The code-view shows the http request(s) sent to the server including headers and the responses, including response-headers, so you can reproduce the same with a http command line tool like curl or a browser plugin like postman. Code-view also helps you examine any issues you might encounter while the Neo4j browser communicates with the server backend.
Going forward we also want the code-view feature to show what you would have to write with our most widely used drivers and languages.
We hope you enjoy taking the Neo4j 2.3.0-M02 release for a spin. Please send us your feedback, especially when it comes to performance and stability with the removed object cache. We’re looking forward to the next stable release of Neo4j 2.3 once it’s released.
Have fun testing this milestone,
Cheers, Michael
Download: https://neo4j.com/download-center/ Documentation: https://neo4j.com/docs/milestone/ Cypher Reference Card: https://neo4j.com/docs/milestone/cypher-refcard Feedback please to: feedback@neotechnology.com GitHub: https://github.com/neo4j/neo4j/issues
O’Reilly’s Graph Databases shows you how to apply the native graph model to real-world problems. Click below to get your free copy of the definitive book on graph databases and your introduction to Neo4j.
