# K-Means Clustering

## Glossary

Directed

Directed trait. The algorithm is well-defined on a directed graph.

Directed

Directed trait. The algorithm ignores the direction of the graph.

Directed

Directed trait. The algorithm does not run on a directed graph.

Undirected

Undirected trait. The algorithm is well-defined on an undirected graph.

Undirected

Undirected trait. The algorithm ignores the undirectedness of the graph.

Heterogeneous nodes

Heterogeneous nodes fully supported. The algorithm has the ability to distinguish between nodes of different types.

Heterogeneous nodes

Heterogeneous nodes allowed. The algorithm treats all selected nodes similarly regardless of their label.

Heterogeneous relationships

Heterogeneous relationships fully supported. The algorithm has the ability to distinguish between relationships of different types.

Heterogeneous relationships

Heterogeneous relationships allowed. The algorithm treats all selected relationships similarly regardless of their type.

Weighted relationships

Weighted trait. The algorithm supports a relationship property to be used as weight, specified via the relationshipWeightProperty configuration parameter.

Weighted relationships

Weighted trait. The algorithm treats each relationship as equally important, discarding the value of any relationship weight.

## Introduction

K-Means clustering is an unsupervised learning algorithm that is used to solve clustering problems. It follows a simple procedure of classifying a given data set into a number of clusters, defined by the parameter `k`. The Neo4j GDS Library conducts clustering based on node properties, with a float array node property being passed as input via the `nodeProperty` parameter. Nodes in the graph are then positioned as points in a `d`-dimensional space (where `d` is the length of the array property).

The algorithm then begins by selecting `k` initial cluster centroids, which are `d`-dimensional arrays (see section below for more details). The centroids act as representatives for a cluster.

Then, all nodes in the graph calculate their Euclidean distance from each of the cluster centroids and are assigned to the cluster of minimum distance from them. After these assignments, each cluster takes the mean of all nodes (as points) assigned to it to form its new representative centroid (as a `d`-dimensional array).

The process repeats with the new centroids until results stabilize, i.e., only a few nodes change clusters per iteration or the number of maximum iterations is reached.

Note that the K-Means implementation ignores relationships as it is only focused on node properties.

For more information on this algorithm, see:

## Initial Centroid Sampling

The algorithm starts by picking `k` centroids by randomly sampling from the set of available nodes. There are two different sampling strategies.

Uniform

With uniform sampling, each node has the same probability to be picked as one of the `k` initial centroids. This is the default sampler for K-Means denoted with the `uniform` parameter.

K-Means++

This sampling strategy adapts the well-known K-means++ initialization algorithm[1] for K-Means. The sampling begins by choosing the first centroid uniformly at random. Then, the remaining `k-1` centroids are picked one-by-one based on weighted random sampling. That is, the probability a node is chosen as the next centroid is proportional to its minimum distance from the already picked centroids. Nodes with larger distance hence have higher chance to be picked as a centroid. This sampling strategy tries to spread the initial clusters more evenly so as to obtain a better final clustering. This option can be enabled by choosing `kmeans++` as the initial sampler in the configuration.

It is also possible to explicitly give the list of initial centroids to the algorithm via the `seedCentroids` parameter. In this case, the value of the `initialSampler` parameter is ignored, even if changed in the configuration.

## Considerations

In order for K-Means to work properly, the property arrays for all nodes must have the same number of elements. Also, they should contain exclusively numbers and not contain any NaN values.

## Syntax

K-Means syntax per mode
Run K-Means in stream mode on a named graph.
``````CALL gds.kmeans.stream(
graphName: String,
configuration: Map
)
YIELD
nodeId: Integer,
communityId: Integer,
distanceFromCentroid: Float,
silhouette: Float``````
Table 1. Parameters
Name Type Default Optional Description

graphName

String

`n/a`

no

The name of a graph stored in the catalog.

configuration

Map

`{}`

yes

Configuration for algorithm-specifics and/or graph filtering.

Table 2. Configuration
Name Type Default Optional Description

nodeLabels

List of String

`['*']`

yes

Filter the named graph using the given node labels. Nodes with any of the given labels will be included.

relationshipTypes

List of String

`['*']`

yes

Filter the named graph using the given relationship types. Relationships with any of the given types will be included.

concurrency

Integer

`4`

yes

The number of concurrent threads used for running the algorithm.

jobId

String

`Generated internally`

yes

An ID that can be provided to more easily track the algorithm’s progress.

logProgress

Boolean

`true`

yes

If disabled the progress percentage will not be logged.

nodeProperty

String

`n/a`

no

A node property corresponding to an array of floats used by K-Means to cluster nodes into communities.

k

Integer

`10`

yes

Number of desired clusters.

maxIterations

Integer

`10`

yes

The maximum number of iterations of K-Means to run.

deltaThreshold

Float

`0.05`

yes

Value as a percentage to determine when to stop early. If fewer than 'deltaThreshold * |nodes|' nodes change their cluster , the algorithm stops. Value must be between 0 (exclusive) and 1 (inclusive).

numberOfRestarts

Integer

`1`

yes

Number of times to execute K-Means with different initial centers. The communities returned are those minimizing the average node-center distances.

randomSeed

Integer

`n/a`

yes

The seed value to control the initial centroid assignment.

initialSampler

String

`"uniform"`

yes

The method used to sample the first `k` centroids. "uniform" and "kmeans++", both case-insensitive, are valid inputs.

seedCentroids

List of List of Float

`[]`

yes

Parameter to explicitly give the initial centroids. It cannot be enabled together with a non-default value of the `numberOfRestarts` parameter.

computeSilhouette

Boolean

`false`

yes

If set to true, the silhouette scores are computed once the clustering has been determined. Silhouette is a metric on how well the nodes have been clustered.

Table 3. Results
Name Type Description

nodeId

Integer

Node ID.

communityId

Integer

The community ID.

distanceFromCentroid

Float

Distance of the node from the centroid of its community.

silhouette

Float

Silhouette score of the node.

Run K-Means in stats mode on a named graph.
``````CALL gds.kmeans.stats(
graphName: String,
configuration: Map
)
YIELD
preProcessingMillis: Integer,
computeMillis: Integer,
postProcessingMillis: Integer,
communityDistribution: Map,
centroids: List of List of Float,
averageDistanceToCentroid: Float,
averageSilhouette: Float,
configuration: Map``````
Table 4. Parameters
Name Type Default Optional Description

graphName

String

`n/a`

no

The name of a graph stored in the catalog.

configuration

Map

`{}`

yes

Configuration for algorithm-specifics and/or graph filtering.

Table 5. Configuration
Name Type Default Optional Description

nodeLabels

List of String

`['*']`

yes

Filter the named graph using the given node labels. Nodes with any of the given labels will be included.

relationshipTypes

List of String

`['*']`

yes

Filter the named graph using the given relationship types. Relationships with any of the given types will be included.

concurrency

Integer

`4`

yes

The number of concurrent threads used for running the algorithm.

jobId

String

`Generated internally`

yes

An ID that can be provided to more easily track the algorithm’s progress.

logProgress

Boolean

`true`

yes

If disabled the progress percentage will not be logged.

nodeProperty

String

`n/a`

no

A node property corresponding to an array of floats used by K-Means to cluster nodes into communities.

k

Integer

`10`

yes

Number of desired clusters.

maxIterations

Integer

`10`

yes

The maximum number of iterations of K-Means to run.

deltaThreshold

Float

`0.05`

yes

Value as a percentage to determine when to stop early. If fewer than 'deltaThreshold * |nodes|' nodes change their cluster , the algorithm stops. Value must be between 0 (exclusive) and 1 (inclusive).

numberOfRestarts

Integer

`1`

yes

Number of times to execute K-Means with different initial centers. The communities returned are those minimizing the average node-center distances.

randomSeed

Integer

`n/a`

yes

The seed value to control the initial centroid assignment.

initialSampler

String

`"uniform"`

yes

The method used to sample the first `k` centroids. "uniform" and "kmeans++", both case-insensitive, are valid inputs.

seedCentroids

List of List of Float

`[]`

yes

Parameter to explicitly give the initial centroids. It cannot be enabled together with a non-default value of the `numberOfRestarts` parameter.

computeSilhouette

Boolean

`false`

yes

If set to true, the silhouette scores are computed once the clustering has been determined. Silhouette is a metric on how well the nodes have been clustered.

Table 6. Results
Name Type Description

preProcessingMillis

Integer

Milliseconds for preprocessing the data.

computeMillis

Integer

Milliseconds for running the algorithm.

postProcessingMillis

Integer

Milliseconds for computing percentiles and community count.

communityDistribution

Map

Map containing min, max, mean as well as p1, p5, p10, p25, p50, p75, p90, p95, p99 and p999 percentile values of community size for the last level.

centroids

List of List of Float

List of centroid coordinates. Each item is a list containing the coordinates of one centroid.

averageDistanceToCentroid

Float

Average distance between node and centroid.

averageSilhouette

Float

Average silhouette score over all nodes.

configuration

Map

The configuration used for running the algorithm.

Run K-Means in mutate mode on a named graph.
``````CALL gds.kmeans.mutate(
graphName: String,
configuration: Map
)
YIELD
preProcessingMillis: Integer,
computeMillis: Integer,
mutateMillis: Integer,
postProcessingMillis: Integer,
nodePropertiesWritten: Integer,
communityDistribution: Map,
centroids: List of List of Float,
averageDistanceToCentroid: Float,
averageSilhouette: Float,
configuration: Map``````
Table 7. Parameters
Name Type Default Optional Description

graphName

String

`n/a`

no

The name of a graph stored in the catalog.

configuration

Map

`{}`

yes

Configuration for algorithm-specifics and/or graph filtering.

Table 8. Configuration
Name Type Default Optional Description

mutateProperty

String

`n/a`

no

The node property in the GDS graph to which the cluster is written.

nodeLabels

List of String

`['*']`

yes

Filter the named graph using the given node labels.

relationshipTypes

List of String

`['*']`

yes

Filter the named graph using the given relationship types.

concurrency

Integer

`4`

yes

The number of concurrent threads used for running the algorithm.

jobId

String

`Generated internally`

yes

An ID that can be provided to more easily track the algorithm’s progress.

nodeProperty

String

`n/a`

no

A node property corresponding to an array of floats used by K-Means to cluster nodes into communities.

k

Integer

`10`

yes

Number of desired clusters.

maxIterations

Integer

`10`

yes

The maximum number of iterations of K-Means to run.

deltaThreshold

Float

`0.05`

yes

Value as a percentage to determine when to stop early. If fewer than 'deltaThreshold * |nodes|' nodes change their cluster , the algorithm stops. Value must be between 0 (exclusive) and 1 (inclusive).

numberOfRestarts

Integer

`1`

yes

Number of times to execute K-Means with different initial centers. The communities returned are those minimizing the average node-center distances.

randomSeed

Integer

`n/a`

yes

The seed value to control the initial centroid assignment.

initialSampler

String

`"uniform"`

yes

The method used to sample the first `k` centroids. "uniform" and "kmeans++", both case-insensitive, are valid inputs.

seedCentroids

List of List of Float

`[]`

yes

Parameter to explicitly give the initial centroids. It cannot be enabled together with a non-default value of the `numberOfRestarts` parameter.

computeSilhouette

Boolean

`false`

yes

If set to true, the silhouette scores are computed once the clustering has been determined. Silhouette is a metric on how well the nodes have been clustered.

Table 9. Results
Name Type Description

preProcessingMillis

Integer

Milliseconds for preprocessing the data.

computeMillis

Integer

Milliseconds for running the algorithm.

mutateMillis

Integer

Milliseconds for adding properties to the projected graph.

postProcessingMillis

Integer

Milliseconds for computing percentiles and community count.

nodePropertiesWritten

Integer

Number of properties added to the projected graph.

communityDistribution

Map

Map containing min, max, mean as well as p1, p5, p10, p25, p50, p75, p90, p95, p99 and p999 percentile values of community size for the last level.

centroids

List of List of Float

List of centroid coordinates. Each item is a list containing the coordinates of one centroid.

averageDistanceToCentroid

Float

Average distance between node and centroid.

averageSilhouette

Float

Average silhouette score over all nodes.

configuration

Map

The configuration used for running the algorithm.

Run K-Means in write mode on a named graph.
``````CALL gds.kmeans.write(
graphName: String,
configuration: Map
)
YIELD
preProcessingMillis: Integer,
computeMillis: Integer,
writeMillis: Integer,
postProcessingMillis: Integer,
nodePropertiesWritten: Integer,
communityDistribution: Map,
centroids: List of List of Float,
averageDistanceToCentroid: Float,
averageSilhouette: Float,
configuration: Map``````
Table 10. Parameters
Name Type Default Optional Description

graphName

String

`n/a`

no

The name of a graph stored in the catalog.

configuration

Map

`{}`

yes

Configuration for algorithm-specifics and/or graph filtering.

Table 11. Configuration
Name Type Default Optional Description

nodeLabels

List of String

`['*']`

yes

Filter the named graph using the given node labels. Nodes with any of the given labels will be included.

relationshipTypes

List of String

`['*']`

yes

Filter the named graph using the given relationship types. Relationships with any of the given types will be included.

concurrency

Integer

`4`

yes

The number of concurrent threads used for running the algorithm.

jobId

String

`Generated internally`

yes

An ID that can be provided to more easily track the algorithm’s progress.

logProgress

Boolean

`true`

yes

If disabled the progress percentage will not be logged.

writeConcurrency

Integer

`value of 'concurrency'`

yes

The number of concurrent threads used for writing the result to Neo4j.

writeProperty

String

`n/a`

no

The node property in the Neo4j database to which the cluster is written.

nodeProperty

String

`n/a`

no

A node property corresponding to an array of floats used by K-Means to cluster nodes into communities.

k

Integer

`10`

yes

Number of desired clusters.

maxIterations

Integer

`10`

yes

The maximum number of iterations of K-Means to run.

deltaThreshold

Float

`0.05`

yes

Value as a percentage to determine when to stop early. If fewer than 'deltaThreshold * |nodes|' nodes change their cluster , the algorithm stops. Value must be between 0 (exclusive) and 1 (inclusive).

numberOfRestarts

Integer

`1`

yes

Number of times to execute K-Means with different initial centers. The communities returned are those minimizing the average node-center distances.

randomSeed

Integer

`n/a`

yes

The seed value to control the initial centroid assignment.

initialSampler

String

`"uniform"`

yes

The method used to sample the first `k` centroids. "uniform" and "kmeans++", both case-insensitive, are valid inputs.

seedCentroids

List of List of Float

`[]`

yes

Parameter to explicitly give the initial centroids. It cannot be enabled together with a non-default value of the `numberOfRestarts` parameter.

computeSilhouette

Boolean

`false`

yes

If set to true, the silhouette scores are computed once the clustering has been determined. Silhouette is a metric on how well the nodes have been clustered.

Table 12. Results
Name Type Description

preProcessingMillis

Integer

Milliseconds for preprocessing the data.

computeMillis

Integer

Milliseconds for running the algorithm.

writeMillis

Integer

Milliseconds for adding properties to the Neo4j database.

postProcessingMillis

Integer

Milliseconds for computing percentiles and community count.

nodePropertiesWritten

Integer

Number of properties added to the projected graph.

communityDistribution

Map

Map containing min, max, mean as well as p1, p5, p10, p25, p50, p75, p90, p95, p99 and p999 percentile values of community size for the last level.

centroids

List of List of Float

List of centroid coordinates. Each item is a list containing the coordinates of one centroid.

averageDistanceToCentroid

Float

Average distance between node and centroid.

averageSilhouette

Float

Average silhouette score over all nodes.

configuration

Map

The configuration used for running the algorithm.

## Examples

 All the examples below should be run in an empty database. The examples use Cypher projections as the norm. Native projections will be deprecated in a future release.

In this section we will show examples of running the K-Means algorithm on a concrete graph. The intention is to illustrate what the results look like and to provide a guide in how to make use of the algorithm in a real setting. We will do this on a small cities graph of a handful nodes connected in a particular pattern. The example graph looks like this:

The following Cypher statement will create the example graph in the Neo4j database:
``````CREATE
(:City {name: 'Surbiton', coordinates: [51.39148, -0.29825]}),
(:City {name: 'Liverpool', coordinates: [53.41058, -2.97794]}),
(:City {name: 'Kingston upon Thames', coordinates: [51.41259, -0.2974]}),
(:City {name: 'Sliven', coordinates: [42.68583, 26.32917]}),
(:City {name: 'Solna', coordinates: [59.36004, 18.00086]}),
(:City {name: 'Örkelljunga', coordinates: [56.28338, 13.27773]}),
(:City {name: 'Malmö', coordinates: [55.60587, 13.00073]}),
(:City {name: 'Xánthi', coordinates: [41.13488, 24.888]});``````

This graph is composed of various City nodes, in three global locations - United Kingdom, Sweden and the Balkan region in Europe.

We can now project the graph and store it in the graph catalog. We load the `City` node label with `coordinates` node property.

The following statement will project the graph and store it in the graph catalog.
``````CALL gds.graph.project(
'cities',
{
City: {
properties: 'coordinates'
}
},
'*'
)``````

In the following examples we will demonstrate using the K-Means algorithm on this graph to find communities of cities that are close to each other geographically.

### Memory Estimation

First off, we will estimate the cost of running the algorithm using the `estimate` procedure. This can be done with any execution mode. We will use the `write` mode in this example. Estimating the algorithm is useful to understand the memory impact that running the algorithm on your graph will have. When you later actually run the algorithm in one of the execution modes the system will perform an estimation. If the estimation shows that there is a very high probability of the execution going over its memory limitations, the execution is prohibited. To read more about this, see Automatic estimation and execution blocking.

For more details on `estimate` in general, see Memory Estimation.

The following will estimate the memory requirements for running the algorithm:
``````CALL gds.kmeans.write.estimate('cities', {
writeProperty: 'kmeans',
nodeProperty: 'coordinates'
})
YIELD nodeCount, bytesMin, bytesMax, requiredMemory``````
Table 13. Results
nodeCount bytesMin bytesMax requiredMemory

8

33248

54240

"[32 KiB ... 52 KiB]"

### Stream

In the `stream` execution mode, the algorithm returns the cluster for each node. This allows us to inspect the results directly or post-process them in Cypher without any side effects.

For more details on the `stream` mode in general, see Stream.

The following will run the algorithm and stream results:
``````CALL gds.kmeans.stream('cities', {
nodeProperty: 'coordinates',
k: 3,
randomSeed: 42
})
YIELD nodeId, communityId
RETURN gds.util.asNode(nodeId).name AS name, communityId
ORDER BY communityId, name ASC``````
Table 14. Results
name communityId

"Kingston upon Thames"

0

"Liverpool"

0

"Surbiton"

0

"Sliven"

1

"Xánthi"

1

"Malmö"

2

"Solna"

2

"Örkelljunga"

2

In the example above we can see that the cities are geographically clustered together.

### Stats

In the `stats` execution mode, the algorithm returns a single row containing a summary of the algorithm result. This execution mode does not have any side effects. It can be useful for evaluating algorithm performance by inspecting the `computeMillis` return item. In the examples below we will omit returning the timings. The full signature of the procedure can be found in the syntax section.

For more details on the `stats` mode in general, see Stats.

The following will run the algorithm and returns the result in form of statistical and measurement values
``````CALL gds.kmeans.stats('cities', {
nodeProperty: 'coordinates',
k: 3,
randomSeed: 42
})
YIELD communityDistribution``````
Table 15. Results
communityDistribution

{max=3, mean=2.6666666666666665, min=2, p1=2, p10=2, p25=2, p5=2, p50=3, p75=3, p90=3, p95=3, p99=3, p999=3}

### Mutate

The `mutate` execution mode extends the `stats` mode with an important side effect: updating the named graph with a new node property containing the cluster for that node. The name of the new property is specified using the mandatory configuration parameter `mutateProperty`. The result is a single summary row, similar to `stats`, but with some additional metrics. The `mutate` mode is especially useful when multiple algorithms are used in conjunction.

For more details on the `mutate` mode in general, see Mutate.

The following will run the algorithm and store the results in `cities` graph:
``````CALL gds.kmeans.mutate('cities', {
nodeProperty: 'coordinates',
k: 3,
randomSeed: 42,
mutateProperty: 'kmeans'
})
YIELD communityDistribution``````
Table 16. Results
communityDistribution

{max=3, mean=2.6666666666666665, min=2, p1=2, p10=2, p25=2, p5=2, p50=3, p75=3, p90=3, p95=3, p99=3, p999=3}

In `mutate` mode, only a single row is returned by the procedure. The result is written to the GDS in-memory graph instead of the Neo4j database.

### Write

The `write` execution mode extends the `stats` mode with an important side effect: writing the cluster for each node as a property to the Neo4j database. The name of the new property is specified using the mandatory configuration parameter `writeProperty`. The result is a single summary row, similar to `stats`, but with some additional metrics. The `write` mode enables directly persisting the results to the database.

For more details on the `write` mode in general, see Write.

The following will run the algorithm and write the results back to Neo4j:
``````CALL gds.kmeans.write('cities', {
nodeProperty: 'coordinates',
k: 3,
randomSeed: 42,
writeProperty: 'kmeans'
})
YIELD nodePropertiesWritten``````
Table 17. Results
nodePropertiesWritten

8

In `write` mode, only a single row is returned by the procedure. The result is written to the Neo4j database instead of the GDS in-memory graph.

### Seeding initial centroids

We now see the effect that seeding centroids has on K-Means. We run K-Means with initial seeds the coordinates of New York, Amsterdam, and Rome.

The following will run the algorithm and stream results:
``````CALL gds.kmeans.stream('cities', {
nodeProperty: 'coordinates',
k: 3,
seedCentroids: [[40.712776,-74.005974], [52.370216,4.895168],[41.902782,12.496365]]
})
YIELD nodeId, communityId
RETURN gds.util.asNode(nodeId).name AS name, communityId
ORDER BY communityId, name ASC``````
Table 18. Results
name communityId

"Kingston upon Thames"

1

"Liverpool"

1

"Malmö"

1

"Solna"

1

"Surbiton"

1

"Örkelljunga"

1

"Sliven"

2

"Xánthi"

2

Notice that in this case the cities have been geographically clustered into two clusters: one contains cities in Northern Europe whereas the other contains in Southern Europe. On the other hand, the cluster with New York as the initial centroid was not the closest to any city at the first phase.

1. Arthur, David and Sergei Vassilvitskii. "k-means++: The Advantages of Careful Seeding." ACM-SIAM Symposium on Discrete Algorithms (2007).