Software building blocks for Memory-Driven Computing.

Hewlett Packard Labs has developed tools that enable programmers to realize the benefits of Memory-Driven Computing and persistent memory. To simplify the developer experience, we are exposing familiar programming environments like Linux and Portable Operating System Interface APIs with programming languages like C/C++ and Java, making the performance advantages of massive memory on fabrics available to you in a way that lets you be productive quickly.

During the design phase of the prototype, simulations had predicted that the speed of a Memory-Driven Computing architecture would improve current computing by multiple orders of magnitude. Novel programming tools developed to harness massive, shared memory have shown execution speeds improved by up anywhere from 100-8,000 times on a range of workloads in areas such as image search, graph inference and financial modelling.

These results were achieved using simulations of prototype of The Machine project running on existing high-performance HPE systems, such as HPE Integrity Superdome X and HPE ProLiant servers.

Sign up here

to be the first to get the latest updates on The Machine User Group and learn how you can get started with the Memory-Driven Computing Developer Toolkit.

Large Scale Graph Inference

Graphical Inference is a remarkable algorithm for graph analytics that abstracts knowledge by combining probabilities and graph representations. It captures useful insights for solving problems like malware detection, genomics analysis, IoT analytics, and online advertisement.

View Repo

Similarity Search

As part of The Machine program, we built a similarity search framework for high-dimensional objects. High-dimensional search enables important component of many analytic tasks in information retrieval, (e.g. document duplicate detection), machine learning (e.g., nearest neighbor classification) and computer vision (e.g., pose estimation and object recognition).

View Repo

Managed Data Structures

A software library for persistent memory programming, which enables developers to declare familiar data structures such as lists and maps as persistent and reuse them across programs and languages, easing programming and data sharing at scale to reap the full benefits of Memory-Driven Computing on The Machine and other persistent memory architectures.

View Repo

Multi-Process Garbage Collector

Our state-of-the-art Multi Process Garbage Collector (MPGC) delivers automatic memory management for applications, enabling multiple applications to share objects in memory, while avoiding memory management errors and memory leaks.

View Repo

Sparkle

Hewlett Packard Labs is making changes available under the same license as Spark (the Apache 2.0 License), a fast and general cluster computing system for Big Data - and is collaborating with HortonWorks to make these enhancements available in the Spark upstream distribution and through their Enterprise Spark at Scale offering.

View Repo

Fast optimistic engine for data unification services

A completely new database engine that speeds up applications by taking advantage of a large number of CPU cores and non-volatile memory.

View Repo

Fault-tolerant programming model for non-volatile memory

Adapts existing multi-threaded code to store and use data directly in persistent memory, provides simple, efficient fault-tolerance in the event of power failures or program crashes.

View Repo

Performance emulation for non-volatile memory latency and bandwidth

A DRAM-based performance emulation platform that leverages features available in commodity hardware to emulate different latency and bandwidth characteristics of future byte-addressable non-volatile memory technologies.

View Repo

Allocator Layer for Persistent Shared Memory

Provides a collection of low-level abstraction layers that relief the user from the details of mapping, addressing, and allocating persistent shared memory.

View Repo

Application-transparent Checkpoint with Persistent Memory

System level application-transparent tool for suspending and resuming Linux applications and Docker containers with persistent memory. It optimizes Checkpoint/Restore In Userspace software tool to support fast checkpoint and restore with persistent memory, such as HPE NVDIMMs.

View Repo

Shoveller

Shoveller is a Scalable, memory-capacity efficient key-value store for very large scale machines (e.g., tens of terabytes or more memory and hundreds of CPU cores).

View Repo

Non-Volatile Memory Manager (NVMM)

Non-Volatile Memory Manager (NVMM) is a library written in C++ that provides simple abstractions for accessing and allocating Non-Volatile Memory (NVM) from Fabric-Attached Memory.

View Repo

Radix Tree

Radix Tree is a user-space library written in C++ that implements a radix tree that relies on fabric-attached memory atomics, which are atomics primitives within a cache-incoherent memory-semantic fabric environment.

View Repo

Write-Ahead-Logging Library for Non-Volatile DIMMs

Write-Ahead-Logging (WAL) is the central component in various software that require atomicity and durability, such as data management systems (DBMS).

View Repo

Linux for Memory-Driven Computing

Hewlett Packard Labs is adapting the Linux Operating System to support Memory-Driven Computing architectures such as, The Machine project. Modifications to Linux include support for fabric – attached persistent memory, block device abstractions for persistent memory.. Also included is an emulator that can be used by developers to explore the new APIs on industry standard machines.

View Repo

Fabric Attached Memory Emulation

An environment designed to allow users to explore the new architectural paradigm of The Machine, called Memory-Driven Computing. The emulation employs virtual machines performing the role of “nodes” in The Machine. Explore shared, global memory space and expect it to behave like Memory-Driven Computing on The Machine. Linux for Fabric-Attached Memory Emulation is also available to provide software for The Machine APIs and allow you to explore Memory-Driven Computing using current hardware.

View Repo

Hewlett Packard Enterprise Puts The Machine in the “Open”

Building The Machine Ecosystem in the Open

The Oral History of The Machine-Open Source

HPE, Labs, and Hortonworks collaborate to bring big-memory Spark to the enterprise

TheNextPlatform: First Steps In The Program Model For Persistent Memory

The Machine

Learn more about The Machine project and its potential to transform computing from the ground up.

Read The Latest

Published Papers

The next chapters of computing are already being written. Read them here.

Get Your Nerd On