YarcData, a Cray company working with the Pittsburgh Supercomputing Center (PSC) , announced the development of “Sherlock”, a graph analytics appliance.
“Sherlock,” says Nick Nystrom, PSC director of strategic applications, “provides a unique capability for discovering new patterns and relationships in data. It will help to discover how genes work, probe the dynamics of social networks, and detect the sources of breaches in Internet security.” Those diverse challenges, along with many others, he adds, have two important features in common: Their data are naturally expressed as interconnected webs of information called graphs, and data sizes for problems of real-world interest become extremely large. “Until now, graph analytics has largely been impractical for big data,” says Nystrom. This is because, he explains, processing of graph structures requires irregular and unpredictable access to data. On ordinary computers and clusters, nearly all the time is spent waiting for that data to move from memory to processors. Even more challenging, graphs of interest typically cannot be partitioned; their high connectivity prevents dividing them into subgraphs that can be mapped independently onto distributed-memory computers. These factors have precluded large-scale graph analytics, especially for the interactive response times that analysts need to explore data. “YarcData’s uRiKA,” says Nystrom, “overcomes that barrier through groundbreaking innovations in computer hardware and software.” Sherlock enables large-scale, rapid graph analytics through massive multithreading, a shared address space, sophisticated memory optimizations, a productive user environment, and support for heterogeneous applications — all packaged as an enterprise-ready appliance. “Sherlock provides researchers with a uniquely powerful tool for doing complex analytics on big data, expanding the capability to address problems of societal importance,” says Nystrom.
Keywords: Big Data • Cray • Graph Analytics • Graph Appliance • YarcData