Hewlett Packard Labs has developed tools that enable programmers to realize the benefits of Memory-Driven Computing and persistent memory. To simplify the developer experience, we are exposing familiar programming environments like Linux and Portable Operating System Interface APIs with programming languages like C/C++ and Java, making the performance advantages of massive memory on fabrics available to you in a way that lets you be productive quickly.
During the design phase of the prototype, simulations had predicted that the speed of the Memory-Driven Computing architecture would improve current computing by multiple orders of magnitude. Novel programming tools developed to harness massive, shared memory have shown execution speeds improved by up anywhere from 100-8,000 times on a range of workloads in areas such as image search, graph inference and financial modelling.
These results were achieved using Memory-Driven Computing simulations running on existing high-performance HPE systems, such as HPE Integrity Superdome X and HPE ProLiant servers.
Large Scale Graph Inference
Graphical Inference is a remarkable algorithm for graph analytics that abstracts knowledge by combining probabilities and graph representations. It captures useful insights for solving problems like malware detection, genomics analysis, IoT analytics, and online advertisement.
Similarity Search
As part of The Machine program, we built a similarity search framework for high-dimensional objects. High-dimensional search enables important component of many analytic tasks in information retrieval, (e.g. document duplicate detection), machine learning (e.g., nearest neighbor classification) and computer vision (e.g., pose estimation and object recognition).
Managed Data Structures
A software library for persistent memory programming, which enables developers to declare familiar data structures such as lists and maps as persistent and reuse them across programs and languages, easing programming and data sharing at scale to reap the full benefits of Memory-Driven Computing.
Multi-Process Garbage Collector
Our state-of-the-art Multi Process Garbage Collector (MPGC) delivers automatic memory management for applications, enabling multiple applications to share objects in memory, while avoiding memory management errors and memory leaks.
Sparkle
Hewlett Packard Labs is making changes available under the same license as Spark (the Apache 2.0 License), a fast and general cluster computing system for Big Data - and is collaborating with HortonWorks to make these enhancements available in the Spark upstream distribution and through their Enterprise Spark at Scale offering.
Fast optimistic engine for data unification services
A completely new database engine that speeds up applications by taking advantage of a large number of CPU cores and non-volatile memory.
Fault-tolerant programming model for non-volatile memory
Adapts existing multi-threaded code to store and use data directly in persistent memory, provides simple, efficient fault-tolerance in the event of power failures or program crashes.
Performance emulation for non-volatile memory latency and bandwidth
A DRAM-based performance emulation platform that leverages features available in commodity hardware to emulate different latency and bandwidth characteristics of future byte-addressable non-volatile memory technologies.
Allocator Layer for Persistent Shared Memory
Provides a collection of low-level abstraction layers that relief the user from the details of mapping, addressing, and allocating persistent shared memory.
Application-transparent Checkpoint with Persistent Memory
System level application-transparent tool for suspending and resuming Linux applications and Docker containers with persistent memory. It optimizes Checkpoint/Restore In Userspace software tool to support fast checkpoint and restore with persistent memory, such as HPE NVDIMMs.
Shoveller
Shoveller is a scalable, memory-capacity efficient key-value store for very large scale machines (e.g., tens of terabytes or more memory and hundreds of CPU cores).
Non-Volatile Memory Manager (NVMM)
Non-Volatile Memory Manager (NVMM) is a library written in C++ that provides simple abstractions for accessing and allocating Non-Volatile Memory (NVM) from Fabric-Attached Memory.
Radix Tree
Radix Tree is a user-space library written in C++ that implements a radix tree that relies on fabric-attached memory atomics, which are atomics primitives within a cache-incoherent memory-semantic fabric environment.
Write-Ahead-Logging Library for Non-Volatile DIMMs
Write-Ahead-Logging (WAL) is the central component in various software that require atomicity and durability, such as data management systems (DBMS).
Linux for Memory-Driven Computing
Hewlett Packard Labs is adapting the Linux Operating System to support Memory-Driven Computing architectures such as, The Machine project. Modifications to Linux include support for fabric – attached persistent memory, block device abstractions for persistent memory. Also included is an emulator that can be used by developers to explore the new APIs on industry standard machines.
Fabric Attached Memory Emulation
An environment designed to allow users to explore the new architectural paradigm of Memory-Driven Computing. The emulation employs virtual machines performing the role of “nodes”. Explore shared, global memory space and expect it to behave like Memory-Driven Computing. Linux for Fabric-Attached Memory Emulation is also available to provide software for our APIs and allow you to explore Memory-Driven Computing using current hardware.
The Oral History of The Machine-Open Source
HPE, Labs, and Hortonworks collaborate to bring big-memory Spark to the enterprise
TheNextPlatform: First Steps In The Program Model For Persistent Memory
Building The Machine Ecosystem in the Open
Memory-Driven Computing
Learn more about Memory-Driven Computing project and its potential to transform computing from the ground up.
Read The LatestPublished papers
The next chapters of computing are already being written. Read them here.
Get Your Nerd On