Sgx Documentation
Fleeting- Annotations
- Introduction
- Enclave Programming Model
- Data exchange in and out of the enclave
- Enclave Signature Structure
- The attestation process
- Secret Provisioning
- Sealing
- Processor Features
- Programming for Performance
- Defense In-depth Mechanisms
- Power Management
- Use of Large Addresses for 32-bit Enclaves
- Threading Topics
- Protection from Side-Channel Attacks
- Notes linking here
- Bibliography
- Permalink
- External reference: @1ed9f3ea-721f-4e67-965c-a0ff16fc16fe
sgx documentation
Annotations
Introduction
The document claims to tell how to develop robust application enclaves.
It defines the executing parts as
- Untrusted Run-Time System (uRTS) – code that executes outside of the Intel SGX enclave environment
- Trusted Run-Time System (tRTS) – code that executes within an Intel SGX enclave (also called Trusted Computing Base in the document)
- Edge Routines – functions that may run outside the enclave (untrusted edge routines) or inside the enclave (trusted edge routines)
- 3rd Party Libraries – for the purpose of this document, this is any library that has been tailored to work inside the Intel SGX enclave environment.
- ECall “Enclave Call” a call made into an interface function within the enclave
- OCall “Out Call” a call made from within the enclave to the outside application
- The context Thread Context for a thread running inside the enclave.
- Thread Control Structure (TCS)
- Thread Data/Thread Local Storage – data within the enclave and specific to the thread
- State Save Area (SSA) – a data buffer which holds register state when an enclave must exit due to an interrupt or exception
- Stack – a stack located within the enclave
But it also defines or implicitly references :
- The SGX Application is the whole program
- The Enclave Trusted Computing Base (TCB), or simply enclave is the trusted part of the SGX Application. It is the same as the tRTS.
- Untrusted Component, or Enclave Application is the part of the SGX Application that is not trusted.
- Enclave Page Cache (EPC), the memory location of the enclave
Enclave Programming Model
An enclave “can be developed using standard tools and development environments”, but “enclave writers must also be aware of the differences in how Intel SGX software is designed, developed and debugged to create robust enclave”
Also, “an enclave is a monolithic software entity that reduces the Trusted Computing Base (TCB) for an application to a trusted runtime system, ISV code and 3rd party trusted libraries”.
Enclave File Format
It is “a regular DLL, Dynamic Library, or Shared Object file typically contains code and data sections corresponding to the functions and/or methods as well as the variables and/or objects implemented in the shared library”
“The code in an enclave is no different than code that exists as part of a regular application”.
“enclave code is loaded in a special way such that once the enclave has been initialized, privileged code and the rest of the untrusted application cannot directly read data that resides in the protected environment”
From the point of view of general programming, the enclave programming looks indeed like normal programming.
It is stressed out that the main “it may be possible to move the majority of application code into an enclave, in most cases this would not be desirable”.
One should make sure to run a code as minimal as possible into the enclave to reduce risks and take much time thinking of the boundaries between the trusted and the untrusted part of the application.
Data exchange in and out of the enclave
One must make extra care of the data going in and out of the enclave, meaning in the ECall and the OCall.
For instance, “pointers to C data structures are reasonable, but pointers to C++ objects are not” and “the enclave must ensure the data cannot be modified after it is checked”
An enclave must be designed in such a way that it prevents leaking side-channel information that would allow an attacker, who is looking at the untrusted functions called from an enclave, to gain insight into enclave secrets
[1]
In particular, “Nested ECalls (ECalls during OCalls)” must not make any assumption whether they will be called back or not.
Also, “Third Party Libraries” must follow the same practice.
Enclave Signature Structure
For the enclave to be useful, it must prove it is whatever it claims it is.
Establishing trust in software involved three main activities: Measurement, Attestation and Sealing
- Measurement: As an enclave is instantiated in a trusted environment, an accurate and protected recording of its identity is taken.
- Attestation: Demonstrating to other entities that a particular envir- onment is instantiated in the correct manner.
- Sealing: Enabling data belonging to the trusted environment to be bound to it so that it can be restored only when the trusted environment is restored.
The “enclaves include a self-signed certificate from the enclave author, also known as the Enclave Signature Structure (SIGSTRUCT)”. This structure “contains information that allows the Intel® SGX architecture to detect whether any portion of the enclave file has been tampered with”. Also, it “identifies the enclave author”.
Enclave Measurement […] a single 256-bit hash that identifies the code and initial data to be placed inside the enclave, the expected order and position in which they are to be placed, and the security properties of those pages. A change in any of these variables will result in a different measurement. When the enclave code/data pages are placed inside the EPC, the CPU calculates the enclave measurement and stores this value in the MRENCLAVE register. Then the CPU compares the content of MRENCLAVE against the enclave measurement value in SIGSTRUCT. Only if they match with each other, the CPU will allow the enclave to be initialized.
[1]
“After an enclave is successfully initialized, the CPU records a hash of the enclave author’s public key in the MRSIGNER register”.
This means that not only the processors ensures that the MRENCLAVE is a value unique to the enclave source code and way of loading, but also the valid key that signed the enclave is accessible at MRSIGNER.
Also, the enclave should provide a “Security Version Number of the Enclave (ISVSVN) – The enclave author assigns a Security Version Number (SVN) to each version of an enclave. The SVN reflects the level of the security property of the enclave, and should monotonically increase with improvements of the security property. After an enclave is successfully initialized, the CPU records the SVN, which can be used during attestation. Different versions of an enclave with the same security property should be assigned with the same SVN. For example, a new version of an enclave with non security related bug fixes should have the same SVN as the older version”.
To do all that “The SDK includes a tool for signing enclaves, called sgx_sign, that takes an enclave file and adds the enclave signature structure as required by the Intel® SGX architecture. This tool supports single-step test signing using a test sign- ing private key configured on the local system, and two-step release signing that involves a signing facility/platform, where the release signing private key is protected. sgx_sign can also generate whitelisting materials from a signed enclave file.”
It is stressed out that “the ISV must maintain the development environment free from malware and other potential threads at all times. If the development platform is ever com- promised, you cannot continue using the Intel SGX support software since it could be used to compromise the integrity of the enclaves built on that platform.”
The attestation process
“Attestation is the process of demonstrating that a piece of software has been established on a platform. In the case of Intel SGX, it is the mechanism by which a third entity establishes that a software entity is running on an Intel SGX enabled platform protected within an enclave prior to provisioning that software with secrets and protected data.”
It describes local and remote attestation processes. For I am more interested in remote attestation, I will only focus on that.
Remote (Inter-Platform) Attestation
In order to assert the validity of an enclave, intel provides a separate enclave, called the quoting enclave. It “verifies the reports that have been created to its MRENCLAVE measurement value and then converts and signs them using a device specific asymmetric key, the Intel EPID key. The output of this process is called a quote.”
The device specific asymmetric key (EPID key), is “a group signature scheme, which allows platforms to cryptographically sign objects while at the same time preserving the signer’s privacy.”
That way, the application can generate an EPID signed report with the enclave signature structure (containing the Author public key hash and the enclave hash). This report can then be used to validate the claim that the enclave is really what is pretends to be, provided we trust the EPID mechanism and the robustness of the enclave report creation.
Then an example of remote attestation process is given.
Basically, it looks like a classical key sharing scheme, where :
- the enclave provides
- the enclave report, signed with EPID allowing the enclave user to authenticate the enclave,
- the public key of a temporary key pair whose private part will never get out of the enclave
- the user then sends to the enclave, encrypted with the temporary key, a session symmetric key, to be used in further communication
To authenticate the user of the enclave, it is suggested to put a certificate root inside the code of the enclave, so that the enclave can first authenticate the user prior to communicating anything with them. The enclave user could sign the session symmetric key, so that the enclave would be able to make sure the symmetric key is actually coming from this user.
An important note is that “two running instances of an enclave cannot be distinguished by the automatically generated data in their REPORT’s alone”. A nonce is generally put in the enclave ECall, to allow separating communications with two running instances.
Secret Provisioning
When dealing with a secret, “the enclave may choose to store the secret in non volatile storage using the sealing techniques”.
Indeed, “enclave secrets may be safely stored outside the enclave boundary after such secrets are properly sealed.”
But beware that “data provisioned to a debug enclave is not secret”.
In order to avoid leaking the secret, it is mandatory to “use the memset_s() function to clear any variable that contained secret data”, to avoid data padding during allocation from containing the secret. Also, “the use of this function guarantees that the compiler will not optimize away the write to memory intended by this function call and thus ensuring the secret data is cleared. Using memset_s() is especially important when secret data is stored in a dynamically allocated buffer. After such a buffer is freed it could be reallocated and its previous contents, if they are not erased, may be leaked outside the enclave.”
Also, the document advises “that secrets are cleared after use” to diminish the risk of secret leakage.
Sealing
“A mechanism is in place which allows enclave software to retrieve a key unique to that enclave. This key can only be generated by that enclave on that particular platform”. This allows the enclave to store the secret outside the enclave.
There are two “Software Sealing Policies”: “Seal to the Current Enclave (Enclave Measurement)” or “Seal to the Enclave Author”.
“Enclave writers should be aware that even though two running instances of the same enclave can be distinguished at the time they attest, there is no Intel® SGX mechanism to prevent one enclave instance from having access to the sealed data of another enclave when both enclaves use the EGETKEY instruction.”
“In order to provide SEAL key separation between enclave instances, including those executing in a virtualized environments (e.g. datacenter or cloud), Intel recommends that enclave writers use the KEYID field of the KEYREQUEST structure passed into the EGETKEY instruction to pass an enclave instance specific nonce. This will provide a different key even when the same enclave is in a different virtual machine.”
“Migrating a user’s data from one platform to the next now requires careful planning”, so that the data is not leaked out.
- the enclaves must attest one another using the attestation mechanism
- they share a key to temporarily encrypt the data during the migration
- then the old enclave must
- decrypt the data,
- encrypt it using the temporary key
- send it to the other enclave
- the other enclave then
- decrypt, using the temporary key
- encrypt, using the seal key
“We refer to these encrypt and decrypt operations as sealing and unsealing, respectively as the data is cryptographically sealed to the enclave and platform”.
Processor Features
As indicated earlier, it looks like any program can be run inside an enclave. Yet, the document indicate a few exceptions.
The following is a list of hardware instructions which are illegal within an enclave and will generate a #UD fault if executed:
- Instructions which may VMEXIT if executed inside an enclave. Since it is not permissible for the VMM to update the enclave, they are not allowed. CPUID, GETSEC, RDPMC, RDTSC, RDTSCP, SGDT, SIDT, SLDT, STR, VMCALL, VMFUNC.
- I/O instructions cannot be executed inside an enclave. These instructions could cause faults which cannot be handled by software. IN, INS/INSB/INSW/INSD, OUT, OUTS/OUTSB/OUTSW/OUTSD.
- Instructions that may require a change in privilege levels. Far call, Far jump, Far ret, INT n/INTO, IRET, LDS/LES/LFS/LGS/LSS, MOV to DS/ES/SS/FS/GS, POP DS/ES/SS/FS/GS, SYSCALL, SYSENTER.
Functions which gather host system attributes, perform I/O, or require a higher privilege level should be performed outside an enclave. The CPUID instruction is also illegal inside the enclave. Thus software that retrieves CPUID information must do so outside the enclave.
It is indicated that “the ISV must write a custom exception handler to catch the #UD fault” to allow the correct behavior in case such exceptions occur, yet it must be remembered that “OCalls are not permitted in exception handlers”.
Programming for Performance
This part reminds us that “SGX architecture does not come for free in terms of impact of the performance on your application”.
Creation
Among other things, “the time to create an enclave scales directly with the size of the enclave, because each additional 4KB page that an enclave uses results in cryptographic processing that occurs for the EADD instruction and cryptographic processing for the 16 EEXTEND instructions to measure that 4KB page”.
The document suggests to “consider the following approaches to reduce this impact”:
- “Reduce the size of your enclave.”
- Make use of the “Enclave Dynamic Memory Management (EDMM)”
- or hide the enclave loading time by having the application perform processing that occupies the user’s attention
Transitions
Transition occurs when the code path goes in the enclave and outside of the enclave, meaning ECall and OCall are transitions.
Transitions to an enclave and from an enclave resemble a context switch in many ways. Parameters are marshalled from the untrusted part of the application to the trusted part, and return values are un-marshalled. In the trusted part, parameters from the untrusted part are un-marshalled and return values for the untrusted part are marshalled.
This is important, because “if an application is passing large parameters between the two parts of an application, a noticeable overhead can be experienced”.
To palliate this issue, it is suggested to “pass a pointer to the data structure into the enclave instead of the actual data”.
Yet, “there is a security risk because the Edger8r tool does not verify the pointer before passing it to the enclave”.
In the end, this is not an easy task, because “you as the developer must ensure that you are not exposing secrets in untrusted memory. You must also implement your own pointer verification if you choose this method.”
Memory access
Another aspect increasing processor time is cache misses.
“If your application is experiencing overhead associated with a high number of cache misses,” then “Reduce the size of your enclave’s data. Inspect your data to ensure that only critical elements are in the enclave. Less data means less encryption/decryption and less data structure checking by the Intel® SGX memory control/protection mechanism.”
Also, “Excessive Writing of Pages” may reduce performance a lot.
Defense In-depth Mechanisms
In this part, the document suggests a few tricks to increase the security of the enclave.
“Enclave writers should set the compiler options such that by default enclaves are built with standard defense in-depth mechanisms available on a given platform.”
It is reminded that “the commonly used compiler options only provide protection when the buffer meets certain criteria”, then the developer must ensure those criteria are met in the wanted situations.
For example, “–fstack-protector does not provide protection when the size of the buffer in stack is below certain threshold to avoid significant performance penalty”.
It is recommended to “apply more strict checking options”, like “–fstack-protector-all, –fstack-protector-strong, and –fstack-protector-explicit”.
Also, code relocation inducing the need to make the code writable, it is advised “for the code to contain no relocations”.
It is eventually advised to beware the “Unsafe C++11 Attributes”, like “noreturn, in particular”, that “may cause a potential security risk”.
Power Management
“SGX architecture does not provide a way of directly messaging the power down event into the enclave.”
Particularly, “the operating system does not guarantee that the enclave will be given enough time to seal all its internal state.”
“Enclaves that wish to preserve state across power transition events must periodically seal enclave state data outside the enclave (on disk or the cloud). On re-instantiation of the application, the enclave is rebuilt from scratch and the enclave must retrieve its protected state (from disk or the cloud) inside the enclave.”
To help reduce performances, the document advises that “the enclave writer should design an application enclave that keeps as little state information as possible inside the enclave, so the application can survive a power transition event smoothly.”
Use of Large Addresses for 32-bit Enclaves
“When an enclave writer develops a 32-bit enclave”, “the enclave may be loaded into a large address […] or it may receive a pointer from a large address range”, not fitting into the 32-bit integer representation.
Then, “the enclave should be designed to cope with these scenarios and fail smartly.”
More generally for any 32-bits applications, “all 32-bit enclaves should expect that they can be loaded above the 2 GB limit or receive a pointer that references memory above this limit”.
Threading Topics
This chapter tackles thread binding and the implication regarding the communication between trusted and untrusted context. There are two thread binding policies that the developer may choose. I am not aware of thread mechanism, therefore I do not understand this part.
What I understand though, is that “thread synchronization objects, mutexes in particular, cannot be used safely across root enclave calls regardless the Thread Binding Policy”.
Protection from Side-Channel Attacks
“Intel SGX does not provide explicit protection from side-channel attacks. It is the enclave developer’s responsibility to address side-channel attack concerns.”