Competency Management: a matter of filtering and recommendation engines.

This GraphGist example is built with certain features in mind, making it capable to reproduce and improve features of existing competency management tools for large organizations ^[2]. However, some general features and patterns are kept, making this example also capable of being used to represent online general-purpose recommendation systems. In this model, the prototype is flexible enough to integrate information from both internal repositories of the organization as well as external information collected from online social networks.

See Alistair Jones's arrow tool.

Starting from a focus on the single "ego-network" ^[3] of each Employee in Figure 1, this network visualization shows several interesting data relationships. Related to the company internal data, note that:

The employee’s personal information may be derived from the Employee profile in a professional social network, data based on profiling activities inside the organization, or integrating these two data sources:

The figure above demonstrates the network of activities and roles in the organization graph. Each Role node is characterized by a few properties:

Activity nodes are assigned a complexity (with 1<`complexity`<5), an index which measures how difficult a certain Activity is, taking into account both the technical expertise required, as well as the level of social-interaction with other colleagues, advisors, customers, and more.

Role nodes can be linked to one or more Activity nodes via a RELATED_ACTIVITY link. This connection is enriched by the property level_weight (also in a discrete scale 1→4), indicating that the Role requires a certain level of command of the corresponding RELATED_ACTIVITY, in order to be performed in a satisfactory way by the employee. Manager Role`s may also be connected via an `SUPERVISES_ACTIVITY link: in this case, the employee may be expected to have a good capability in directing the activity, but not necessarily in performing it directly.

Finally, each Activity can be connected to only one Competence Area node, via the IN_AREA link (i.e. in this model no overlap among different competence areas is allowed, but this assumption may be obviously relaxed). Competency Areas shall be understood as high-level classifications of the different `Activitie`s performed by the organization.

Here is the set of commands to generate the model with Cypher.

The whole graph looks like:

In order to rank the best possible candidates, the first approach will be to compile a hiring committee of employees that may work or will work with the candidate within a team. To demonstrate the advantage of collaborative ranking, assume the the team offering the open position (Team 3) currently consists of only one employee.

This illustrates the bias problem outlined in the introduction ^(go to)^: when choosing the ideal candidate, we must integrate the knowledge of the employee already working in the team with the open position (Employee 5). In this case, an intuitive and effective solution is to rely on data relationships like:

(Employee A)-[:Endorses {level:x}]-(Personal_Skill a)-[:HAS]-(Employee B)

where the property level x is a rating of the Personal Skill of Employee B evaluated by Employee A. This kind of Cypher queries provide an understanding of the candidates' skills as perceived by their colleagues.

To make this analysis quantitative, a metric of difference or similarity among the employees will be introduced to understand which employees have similar opinions regarding their colleagues. With this metric, it is possible to query for a ranking of who would best fit the open position in a certain group, even if the people in the group do not know the candidate directly. The approach then closely resembles an user-based recommendation system, one basic collaborative filtering technique. In order to associate a quantitative distance metric, an easy solution is to apply cosine-similarity (this approach is thoroughly explained in this GraphGist by Nicole White). The basic concept is that colleagues of a specific team member, who have evaluated other employees in a similar manner, are likely to be an appropriate option to join the hiring committee. The committee is thus expanded to employees who are not team members, but nevertheless good fits to the team hiring committee.

Cosine-similarity ranking can also be the basis for the recommendation analysis. However, it is important to remember that employees endorse Personal Skills and not other employees in our data model. Therefore, an additional directed relation among Employees A and B ([:RATES {rating: …​}]) must be included, with:

\( rating_{ A \to B }= \frac{ \sum_{ i=1 }^{ N } \textrm{ level }(\textrm{ Personal Skill }(i))} {N} \)

where \(N:= \textrm{# endorsed Personal_Skills of B, by A }\), and the property rating is the average of the endorsements' levels made by A about single Personal Skills of B.

These preliminary calculations provide the ability to calculate the cosine similarities for Employee 5. However, adding a further step can be useful to express a more general query. The query should also be able to handle the case of Team 3 being composed of more than only one member. In this case, the required approach would be to calculate a vector of averaged group ratings for each employee that is not in the group, but who has been evaluated by at least one group member. This averaged score is labeled team_rating and set as a property on the data relationship from the Team to the Employee being evaluated.

The portion of the graph with the [:ENDORSES] and [:RATES] data relationships can be visualized with:

After calculating a group rating, it is now possible to introduce the similarity of Team 3 as a whole with other employees who are not members of Team 3. The similarity works in the same manner as if it were calculated for a single employee. Note that in order to retrieve co-ratings, it is critical to perform a MATCH clause against 2^nd order connections with explicit filters according to the type of data relationships. With a graph database, this is simple since the data relationships are objects themselves.

To account also for direct recommendations from team members, set s.similarity=2 as the similarity of the members of Team 3 with the team itself. Being similarity<1 for all other employees in the graph, prioritizing the team members’ evaluations ahead of others is simple.

Once the similarities among Team 3 and all other employees ^[8] are known, a first recommendation can be made about which employees may be ideal matches for the open position.

For the calculation of this first score, assume that all the colleagues outside of Team 3 are equally copmetent in evaluating the skills of other colleagues. Algorithmically, this means there will be no extra weight or score for those people who have been evaluated directly by Employee 5. To cap the number of ratings to consider, it is possible to limit them in accordance to the similarities of the employees who provided those ratings. Adopting a k-nearest neighbors ^[9] (k-NN) algorithm will allow the query to only pick the evaluations by the k most similar colleagues. How to choose k? A simple choice is to render a small team as competent in evaluating his next member, at least as the average team in the organization. If the team is above the average size, one could instead select all and only the evaluations made by team members. For this example ^[10], k=5, including Employee 5, under the assumption he has evaluated at least some of the candidates.

Once a value for k has been selected, the next step is to query the graph model for the k-NNs employees, and then average their evaluations as a likely estimate of how good a certain employee may perform within the Team with the open position. As mentioned before, evaluations by internal employees of the Team will be considered first.

This query results in the first recommendations for assigning the open position to existing employees. Notice how the recommendation obtained for Employee 1 is even higher than the one obtained for Employee 3, even if no one in Team 3 knows Employee 1 directly. The only person directly known to Team 3 is Employee 3, making this pick the only possible one, without any recommendation system. Using a graph model, now the organization can explore for candidates their entire employee network and eliminate the bias originating from having a small hiring committee.

In this section, the greatest importance is assigned to the network and data relationships composing the data model itself, rather than the properties and characteristics of the single member. This approach does not take into account (yet) the competency area where the open position is available. The advantage of using a graph data model is that even in cases where almost nothing is known about the single employee’s activities and areas of expertise, the method is able to rank according to mutual rating connections.

However, the graph data model in this example can provide even more insight. One can use additional criterions in order to improve the candidate ranking. The most intuitive choice is to introduce a content-based weighting to the recommendations based on the social data relationships of the employees' graph. In fact, the start was a purely "social" collaborative filtering, where the expertise of the evaluating employees is not taken into account. This approach is fine when no information is available about the required or desired skills and experiences for the open position. However, additional features that characterize the ideal candidate and the team with the available opening can now be used to further improve the ranking through a variety of approaches.

Emphasizing the role of competency area nodes instead of employee nodes, it is for example possible to weight the recommendation scores of the candidates, according to the distance in the organization graph of a certain employee from the Competence Area associated with the open position. The weights will be based on the length of the path (Employee)--(Competence Area). Considering that for the open position of Role 6 we know that: (Role 6)-[:RELATED_ACTIVITY]-(Activity 6)-[:IN_AREA]-(Competence Area 2), an improved query for recommendations looks like:

Constrained by shortestPath, it is possible to:

When calculating the weights of the candidate ratings, subtract 1 because in this GraphGist, the shortest path possible for whatever employee towards a Competency Area has precisely length 2: ((Employee)-[:CAN_PERFORM]-(Activity)-[:IN_AREA]-(Competence Area)). Ratings from those employees who minimize this thematic distance are left unaffected. Specifically in this example, the purely collaborative ranking above was further confirmed, even assigning priority to evaluations made by employees more familiar with the competency Area involved in the open positions. Employee 1 is still calculated as the optimal choice, even if now his advantage over other colleagues is smaller.

Another possible refinement strategy relies on the so called "Jaccard similarity coefficient". The cosine similarity used in the first paragraph is indeed derived from this index, which is widely used in Social and Economic Sciences to evaluate the diversity or similarity of two samples. Here it is possible to use the simplest case: in fact, we may refer to the presence/absence of matches with the required features as a binary value, stating if the feature belongs or not to the sample. Another refinement strategy will use Jaccard similarity coefficient. Cosine similarity used earlier in this GraphGist is derived from this index, which is widely used in academics to evaluate the diversity or similarity of two samples. Here, it is used in the simplest case: thematic features are used to build samples, and binary values state if the feature belongs or not to the sample. This building two kinds of sets:

Included in T are all of the required criterions for the evaluation of the candidates, like Personal Skills possessed in a certain Competence Area, Activities performed or related to their own Roles, Degrees held in the Competency Area of interest. These cumulatively characterize the whole set of Employees already within Team 3. The criterions included in T will be matched against those possessed by the i^th Employee, and filtered in set E_i only if a certain path connects the criterion with this Employee.

Once these two sets have been retrieved, one can use the Jaccard formula for the coefficient J as:

\( J = \frac{|T \; \cap \; E_i |}{|T|} \)

(remembering that \(T \cup E_i = T\) by definition ^[12].

Now, all of the elements for this refinement query have been introduced. First set the binary property pool for all those nodes that are worth being included in the candidates’ evaluation. To refine the results, adopt the same criterion as above for this preliminary selection: a maximum of 5 nearest-neighbour employees, those with highest cosine similarity to Team 3 in evaluating other colleagues.

Next set the property t_feature to label those nodes representing the selected team features. For example, one could include Competence Areas linked to the employees in the team via personal skills and activities, Degrees in a certain area(s), and the ability to perform specific Activities.

Now, to calculate the Jaccard coefficient, evaluate how many of the team features are possessed by each of the `Employee`s contributing to the evaluation. This can be done by querying for nodes within the set T:

The final step is to use the Jaccard coefficients as weights: this can be done with a query very similar to the path-based refinement. Here, an additional WHERE clause filters evaluations by employees for whom no Jaccard coefficient can be provided.

Looking at the results, Employee 3 is given a slightly higher ranking now, accounting for criterions such as Degree or Personal Skill related to the same Competence area. Note that this challenges the previous ranking of Employee 1 as the optimal choice.

In this example, shortest-path and Jaccard distances are adopted only as metrics improving the k-NN and cosine similarity recommendations. However, these metrics based on established criterions may also replace recommendations based on collaborative filtering when handling a cold start problem. This occurs when the organization, or the evaluating group, started too recently to provide a sufficient number of evaluations about other colleagues, for different activities. This may prevent a successful adoption of the similarity as outlined above. If the organization keeps an updated and detailed record of its employees’ profiles, though, feature-based similarities could be used for the ranking of the (few) evaluations, and help to solve the problem.