🔏 Edge#34: Homomorphic Encryption
TheSequence is a convenient way to build and reinforce your knowledge about machine learning and AI
In this issue:
we explain the concept of homomorphic encryption;
we explore Intel nGraph–HE that shows how neural networks can operate on homomorphically encrypted data;
we overview Microsoft SEAL, the HE library every data scientist must know about.
Enjoy the learning!
💡 ML Concept of the Day: What is Homomorphic Encryption?
For the last installment of our series about private machine learning (Edge#30, #31, #32, #33), we are going to focus on one of the technologies that is considered almost ideal for these types of scenarios: homomorphic encryption (HE). Since its inception, homomorphic encryption has been, in some context, the security technique linked to private machine learning. Some of the practical challenges with HE forced private machine learning implementations to rely on simpler alternatives, such as secure multi-party computation (sMPC) or differential privacy. But HE continues to be seen as an ideal method for private machine learning solutions.
HE can be considered one of the greatest breakthroughs in the cryptography space within the last decade. The core ideas of HE are based on a mathematical phenomenon known as homomorphism, which is defined as “a mapping of a mathematical set (such as a group, ring, or vector space) into or onto another set or itself in such a way that the result obtained by applying the operations to elements of the first set is mapped onto the result obtained by applying the corresponding operations to their respective images in the second set”. HE allows specific types of computations to be carried out on ciphertext, which produces an encrypted result that is also in ciphertext.
In the context of machine learning, there are two types of HE algorithms:
Partial Homomorphic Encryption (PHE): Given the encryption of two data primitives a and b: E(a) and E(b), PHE can compute E(a+b) OR E(ab) without knowing a, b or the private key. PHE is the most common homomorphic encryption technique as it's considerably less expensive than full homomorphic encryption models. There are different types of PHE algorithms including Unpadded RSA, ElGamal and Paillier.
Fully Homomorphic Encryption (FHE): Given the encryption of two data primitives a and b: E(a) and E(b), PHE can compute both E(a+b) AND E(ab). FHE was considered impossible until 2009, when IBM researcher Craig Gentry published his doctoral thesis in which he showed a method to construct FHE systems using lattice-based cryptography.
HE methods have been traditionally expensive to implement and, as a result, have been hard to apply in large scale private machine learning scenarios. However, that dynamic is changing rapidly. Regular progress in HE frameworks is making HE a more viable option for the next generation of private machine learning solutions.
🔎 ML Research You Should Know: Intel’s nGraph–HE Shows How Neural Networks can Operate on Homomorphically Encrypted Data
In the paper, nGraph–HE: A Graph Compiler for Deep Learning on Homomorphically Encrypted Data, Intel Research proposes an extension to its deep learning compiler to operate on homomorphically encrypted data.
The objective: Integrate homomorphic encryption (HE) into Intel’s nGraph deep learning compiler.
Why is it so important: Intel’s nGraph is one of the most popular compilers for deep neural networks. Intel’s research showed how nGraph can be extended to operate in HE scenarios.
Diving deeper: The goal of the nGraph–HE (or HE–Transformer) project was to evaluate the feasibility of HE methods in local machines. Most of the time, building machine learning that operates on HE data requires not only neural network knowledge but also deep expertise in encryption techniques. nGraph–HE looks to act as an abstraction layer that allows data scientists to design deep learning models in different frameworks and then deploy these models to HE cryptosystems that operate on encrypted data. This combination hides the HE complexities from the model development process, making the HE integration as simple as adding a couple of lines of code.
nGraph–HE has been piloted using Microsoft Research’s famous Simple Encrypted Arithmetic Library (SEAL) (see the next section), which is considered the premier open-source HE framework in the market. The initial research for the framework worked with any model serialized with nGraph and supported a variety of operations compatible with HE data. The nGraph–HE prototype was natively integrated with TensorFlow via the nGraph Compiler and runtime for TensorFlow as well as other deep learning frameworks via its integration with Open Neural Network Exchange (ONNX) standard.
🤖 ML Technology to Follow: Microsoft’s SEAL is the HE Library Every Data Scientist Must Know About
Why should I know about this: Microsoft Simple Encrypted Arithmetic Library (SEAL) is one of the most complete HE open source frameworks in the market and a favorite of machine learning researchers.
What is it: Microsoft SEAL is an HE encryption library for implementing different HE-schemes. Built in C++, SEAL supports two fundamental types of HE encryption methods:
The BFV scheme allows modular arithmetic to be performed on encrypted integers.
The CKKS scheme allows additions and multiplications on encrypted real or complex numbers, but yields only approximate results.
Both HE schemes have their very specific applications. For instance, in scenarios relying on exact value computations, BFV-schemes tend to be more appropriate. That’s obviously not the case of machine learning models, for which CKKS-schemes are highly more efficient. Considering that homomorphically encrypted data is many times bigger than the original data, SEAL enables data compression methods using libraries such as Zlib.
In addition to the original C++ implementation, SEAL is available in .NET languages and can be compiled for mobile runtimes like Android. Additionally, SEAL has been integrated into deep learning frameworks such as TensorFlow and PySyft.
How can I use it: Microsoft SEAL is open source and available at https://github.com/microsoft/SEAL
🧠 The Quiz
Now, to our regular quiz. After ten quizzes, we will reward the winners. The questions are the following:
What is the difference between partial and full homomorphic encryption (HE)?
What is the main value proposition of Intel’s nGraph–HE?
That was fun! Thank you. See you on Sunday 😉
TheSequence is a summary of groundbreaking ML research papers, engaging explanations of ML concepts, and exploration of new ML frameworks and platforms. TheSequence keeps you up to date with the news, trends, and technology developments in the AI field.
5 minutes of your time, 3 times a week – you will steadily become knowledgeable about everything happening in the AI space.
