Alex Shamis

Alex Shamis

Research Software Engineer and PhD Student working on systems and security

Posts by Collection

publications

No compromises: distributed transactions with consistency, availability, and performance

Aleksandar Dragojevic, Dushyanth Narayanan, Ed Nightingale, Matthew Renzelmann, Alex Shamis, Anirudh Badam, Miguel Castro
Published in Symposium on Operating Systems Principles (SOSP), 2015

In this paper, we show that there is no need to compromise in modern data centers. We show that a main memory distributed computing platform called FaRM can provide distributed transactions with strict serializability, high performance, durability, and high availability.

Download here

CCF: A Framework for Building Confidential Verifiable Replicated Services

Mark Russinovich, Edward Ashton, Christine Avanessians, Miguel Castro, Amaury Chamayou, Sylvan Clebsch, Manuel Costa, Cedric Fournet, Matthew Kerner, Sid Krishna, Julien Maffre, Thomas Moscibroda, Kartik Nayak, Olga Ohrimenko, Felix Schuster, Roy Schuster, Alex Shamis, Olga Vrousgou, Christoph M. Wintersteiger
Technical Report, 2019

This paper present CCF, a framework to build premissioned confidential blockchains. CCF provides a simple programming model of a highly-available data store and a universally-verifiable log that implements a ledger abstraction. CCF leverages trust in a consortium of governing members and in a network of replicated hardware-protected execution environments to achieve high throughput, low latency, strong integrity and strong confidentiality for application data and code executing on the ledger.

Download here

snmalloc: A Message Passing Allocator

Paul Liétar, Theodore Butler, Sylvan Clebsch, Sophia Drossopoulou, Juliana Franco, Matthew J. Parkinson, Alex Shamis, Christoph M. Wintersteiger, David Chisnall
Published in 2019 ACM SIGPLAN International Symposium on Memory Management, 2019

This paper presents snmalloc, a new point in the allocator/deallocator design space. Instead of thread-caching, we use lightweight lock-free message-passing to send batches of deallocations to the originating thread.

Download here

Fast General Distributed Transactions with Opacity

Alex Shamis, Matthew Renzelmann, Stanko Novakovic, Georgios Chatzopoulos, Aleksandar Dragojevic, Dushyanth Narayanan, Miguel Castro
Published in ACM SIGMOD International Conference on Management of Data, 2019

This paper extends the design of FaRM — which provides strict serializability only for committed transactions — to provide opacity while maintaining FaRM’s high throughput, low latency, and high availability within a modern data center. It uses timestamp ordering based on real time with clocks synchronized to within tens of microseconds across a cluster, and a failover protocol to ensure correctness across clock master failures.
Best Paper - Honorable Mention

Download here

AMP: Authentication of Media via Provenance

Paul England, Henrique S Malvar, Eric Horvitz, Jack W Stokes, Cédric Fournet, Amaury Chamayou, Sylvan Clebsch, Manuel Costa, Shabnam Erfani, Kevin Kane, Alex Shamis
Technical Report, 2020

Advances in graphics and machine learning algorithms and processes have led to the general availability of easy-to-use tools for modifying and synthesizing media. The proliferation of these tools threatens democracies around the world by enabling wide-spread distribution of false information to billions of individuals via social media platforms. One approach to thwarting the flow of fake media is to detect synthesized or modified media via the use of pattern recognition methods, including statistical classifiers developed via machine learning. While detection may help in the short-term, we believe that it is destined to fail as the quality of the fake media generation continues to improve. Within a short period of time, neither humans nor algorithms will be able to reliably distinguish fake versus real content. Thus, pipelines for assuring the source and integrity of media will be required—and will be increasingly relied upon. We propose AMP, a system that ensures authentication of a media contents source via provenance.

Download here

A1: A Distributed In-Memory Graph Database

Chiranjeeb Buragohain, Knut Magne Risvik, Paul Brett, Miguel Castro, Wonhee Cho, Joshua Cowhig, Nikolas Gloy, Karthik Kalyanaraman, Richendra Khanna, John Pao, Matthew Renzelmann, Alex Shamis, Timothy Tan, Shuheng Zheng
Published in ACM SIGMOD International Conference on Management of Data, 2020

This paper present CCF, a framework to build premissioned confidential blockchains. CCF provides a simple programming model of a highly-available data store and a universally-verifiable log that implements a ledger abstraction. CCF leverages trust in a consortium of governing members and in a network of replicated hardware-protected execution environments to achieve high throughput, low latency, strong integrity and strong confidentiality for application data and code executing on the ledger.

Download here

talks

Personal experiences building Microsoft’s Cosmos and FaRM

Published:

During my time at Microsoft, I have worked on several different distributed systems that the company uses internally. In this talk, I will describe two of them and also discuss the trade-offs and design choices that were made to build and run these systems. I will firstly talk about Microsoft Cosmos which is Microsoft’s internal big data framework that runs dryad-like jobs which are programmed using the SCOPE language. Then, I will describe FaRM which is a main-memory distributed transactional store that exploits new hardware trends such as RDMA and non-volatile memory to achieve performance that is several orders of magnitude faster than systems that offer similar capability but run using TCP. – LSDS - Seminar

FaRM and MVCC

Published:

FaRM is a main memory distributed computing platform that provides distributed transactions with strict serializability, high performance, durability, and high availability. FaRM’s transactional reads however, do not provide opacity. Transactions sometimes operate on inconsistent state before aborting, which can result in program crashes. Programmers using FaRM are required to introduce consistency checks, similar to the checks used in lock-free algorithms to prevent this. This is a major issue as it significantly complicates the programming model, even for the experience programmers that use FaRM to build production systems. To fix this problems, we have changed the FaRM protocols to provide opacity using Multiversion Concurrency Control (MVCC). – UK Systems Workshop

Microsoft Research Faculty Summit 2018 - FaRM

Published:

FaRM is an in-memory transactional object store for random-access latency-sensitive workloads, such as graphs and key-value stores. It provides high throughput and low microsecond latency by using remote direct memory access (RDMA) and novel transactional protocols. In addition to presenting FaRM, we have built A1, a graph store built on top of FaRM, and which is being adopted as the next generation graph platform targeting several scenarios in Bing, Office, and Cortana. – Microsoft Research

FaRM Distributed transactions with consistency, availability, performance, and opacity

Published:

FaRM is an in-memory transactional object store for random-access latency-sensitive workloads, such as graphs and key-value stores. It provides high throughput and low microsecond latency by using remote direct memory access (RDMA) and novel transactional protocols. In this talk I present FaRM, discuss it’s history, and the the direction of the project. Finally, I show how FaRM is being used by Microsoft as a production service.

Fast General Distributed Transactions with Opacity

Published:

Transactions can simplify distributed applications by hiding data distribution, concurrency, and failures from the application developer. Ideally the developer would see the abstraction of a single large machine that runs transactions sequentially and never fails. This requires the transactional subsystem to provide opacity (strict serializability for both committed and aborted transactions), as well as transparent fault tolerance with high availability. As even the best abstractions are unlikely to be used if they perform poorly, the system must also provide high performance.

The Confidential Consortium Framework

Published:

The Confidential Consortium Framework (CCF) is an open-source framework for building permissioned confidential applications. CCF uses replicated hardware-protected trusted execution environments to ensure confidentiality and integrity whilst providing high throughput and low latency. CCF exposes a transactional programming model atop a key-value store, and supports transparent programmable governance.