This is the third post of a series on providing visibility of TLS 1.3 traffic in the intranet. An index to the series and related materials can be found in the TLS Traffic Visibility page.
Update. This post has been updated to say that, in the ST variant, the messages that convey the traffic secrets also convey the two-byte designation of the cipher suite that specifies the AEAD algorithm to be used with the keys derived from the secrets, and that the messages include the connection ID of the client-server connection as the AEAD associated data. The middlebox needs to be told what algorithm to use to decrypt early data if the early data is rejected by the server.
TLS 1.3 has created a problem for enterprises by discontinuing all key exchange methods that use static key pairs. In the first post of this series I described a solution to this problem that preserves forward secrecy, based on the establishment of an ephemeral shared secret between the TLS server and a visibility middlebox. In the second post I provided full details of the solution for the (EC)DHE-only key exchange mode of TLS 1.3. In this post I show how the solution can be extended to handle the PSK-only and PSK + (EC)DHE key exchange modes and the 0-RTT feature of TLS 1.3 by providing the PSK to the middlebox. In this post I also introduce a variant of the solution that handles the PSK modes without the middlebox having to know the PSK and provides different benefits. Both variants can be used in all three key exchange modes of TLS 1.3.
Two Variants of the Visibility Solution
In the visibility solution for the (EC)DHE-only mode as described in the two previous posts, the server and the middlebox establish a visibility shared secret and use it to independently derive the TLS private key. The middlebox can then compute on its own the TLS shared secret, which is the only secret input to the key schedule in the (EC)DHE-only mode, and derive the handshake and application traffic secrets. But in the PSK modes the PSK is an input to the key schedule.
A straightforward way of extending the visibility solution to the PSK modes is to let the middlebox know the PSK. If the PSK is provisioned to the client and the server, it is provisioned to the middlebox as well. If the PSK is derived from the Resumption Master Secret of an earlier connection and a ticket nonce, it is derived by the middlebox after decrypting the NewSessionTicket and retained for future use.
But as I was working out the details of this straightforward extension I realized that there is a way of providing visibility, applicable to all three key exchange modes of TLS 1.3, that does not require giving the PSK to the middlebox in the PSK modes. Instead of using the visibility shared secret to derive the TLS private key, it can be used to derive an encryption key that the server can use to send the traffic secrets to the middlebox with confidentiality protection.
There are thus two variants of the visibility solution:
- Secret Derivation, which I will call the SD variant or the SD method, where the middlebox derives the traffic secrets on its own from the visibility shared secret; and
- Secret Transmission, which I will call the ST variant or the ST method, where the server transmits the traffic secrets to the middlebox encrypted under keys derived from the visibility shared secret.
Both variants preserve the forward secrecy provided by TLS 1.3 in (EC)DHE-only and PSK + (EC)DHE mode. A PSK by itself does not provide forward secrecy. However, in the session resumption case, PSK-only mode can be viewed as providing a conditional form of forward secrecy if the PSK is deleted after all resumptions of the session that use the PSK end. Both variants of the visibility solution preserve this conditional form of forward secrecy.
Besides both preserving forward secrecy when applicable, the two variants provide different variant-specific benefits in different key exchange modes.
- In the PSK modes, the ST method provides the security benefit that the PSK does not need to be stored in the middlebox. In the (EC)DHE modes ST provides the performance benefit for the middlebox of not having to compute the TLS shared secret. Better performance may allow the middlebox to provide traffic visibility for more servers.
- In all three modes, the SD method provides the performance benefit for the server of not increasing the latency of the TLS handshake if the visibility shared secret is precomputed; I will discuss precomputation in another post. In PSK-only mode, SD provides the performance benefit for the middlebox of not having to perform the visibility key exchange and compute the visibility shared secret.
Secret Derivation in (EC)DHE-only Mode
The SD variant in (EC)DHE-only mode was described in detail in the the previous post.
Secret Derivation in PSK + (EC)DHE Mode
Figure 1 shows the steps of the SD method in PSK + (EC)DHE mode, in the case where the client sends and the server accepts 0-RTT data. As in the figure of the previous post, which showed the SD method in the (EC)DHE-only mode, the middlebox observes the client-server traffic on the wire and communicates with the server over a long-standing TCP connection, without encryption but with message integrity protection provided by HMAC signatures computed with a provisioned key.
At Step 1 the client sends a ClientHello message, which is received by the server and observed on the wire by the middlebox. (Here and elsewhere, observation by the middlebox of messages exchanged between the client and the server is indicated by a small circle.) The middlebox forwards the message to a monitoring facility.
The ClientHello message includes, among other things: a list of cipher suites offered by the client, each specifying an AEAD algorithm and a hash function; a key_share extension listing groups offered by the client and corresponding public keys; a pre_shared_key_exchange_modes extension specifying the PSK modes supported by the client; a pre_shared_key extension offering PSKs; and an early_data extension announcing that 0-RTT data will be sent.
Each PSK is associated with a cipher suite and thus with an AEAD algorithm and a hash function. The client may offer multiple cipher suites and multiple PSKs to the server, but it must use the first listed PSK and its associated cipher suite to derive the traffic keys for the protection of the early data. The server will select that PSK and cipher suite in the ServerHello message if it accepts the early data, as assumed in the figure.
At Step 2 the client, the middlebox and the server independently use the Key Schedule to derive the Early Secret from the first PSK offered by the client, and the client_early_traffic_secret from the Early Secret and the transcript hash of the ClientHello message. The derivations are performed using HKDF parameterized by the hash function associated with the first PSK offered by the client. The middlebox can perform the derivations because it has its own copy of the PSK, knows its associated hash function, and has obtained the ClientHello message from the wire. The three parties further derive the early traffic keys from the client_early_traffic_secret as specified in Section 7.3 of the RFC, using again the hash function associated with the PSK as the HKDF parameter. The early traffic keys consist of a client_write_key used as the AEAD key and a client_write_iv that is x-ored with a padded record sequence number to produce the AEAD nonce.
At Step 3 the client sends the early data protected under the early traffic keys, which the middlebox observes on the wire, using the AEAD algorithm associated with the PSK. As in Figure 4 of the RFC, parentheses indicate protection under the early traffic keys.
At Step 4 the middlebox and the server decrypt the early data. The middlebox forwards the early data to the monitoring facility for inspection.
Steps 5-10 are like Steps 2-7 of the figure of the previous post. The server selects a group for the visibility key exchange and a group for the TLS key exchange, the latter among those offered by the client in its key_share extension. The server tells the middlebox which groups it has selected. The middlebox and the server generate visibility key pairs and use them to establish the visibility shared secret. Then the middlebox and the server independently derive the server’s TLS private key from the visibility shared secret, and the server further computes its TLS public key from private key.
At Step 11 the server sends the ServerHello message, which the middlebox observes on the wire. The middlebox forwards the message to the monitoring facility for inspection.
The ServerHello message includes: a key_share extension specifying the TLS group selected by the server and conveying the server’s TLS public key; a pre_shared_key extension selecting the first PSK offered by the client; and a cipher_suite selection specifying the cipher suite associated with that PSK.
At Step 12 the client, the middlebox and the server independently compute the TLS shared secret. The middlebox performs the computation using the private key generated at Step 10 and the public key that the client’s key_share extension observed at Step 1 associates with the group selected by the server. At Step 13 the client, the middlebox and the server derive the Handshake Secret from the PSK and the TLS shared secret as specified in the Key Schedule, using the hash function associated with the PSK as the HKDF parameter. Then they derive the client_handshake_traffic_secret and the server_handshake_traffic_secret from the Handshake Secret and the transcript hash of the ClientHello and ServerHello messages as specified in the schedule, and the handshake traffic keys from the traffic secrets as specified in Section 7.3 of the RFC, always using the hash function associated with the PSK as the HKDF parameter. The middlebox can perform the derivations because it has its own copy of the PSK, knows its associated hash function, and has observed the ClientHello and ServerHello messages on the wire.This time the traffic keys include a client_write_key and a client_write_iv for handshake messages sent by the client and a server_write_key and server_write_iv for handshake messages sent by the server. As explained above in connection with the early traffic keys, the write keys are used as AEAD keys and the write IVs are used to generate per-record AEAD nonces.
At Step 14 the server sends two messages protected under the handshake traffic keys, using the AEAD algorithm associated with the PSK. As in Figure 4 of the RFC, curly brackets denote protection under the handshake traffic keys. The EncryptedExtensions message includes an early_data extension indicating that the server is accepting the early data. The Finished message includes an HMAC signature where the hash function associated with the PSK is used as the HMAC parameter. The signature is computed using a finished_key derived from the server_handshake_traffic_secret, which is itself derived from the PSK. The Finished message thus demonstrates knowledge of the PSK and authenticates the server.
At Step 15 the client and middlebox decrypt the protected handshake messages using the server_write_key and server_write_iv that they independently derived at Step 13. The middlebox forwards the decrypted handshake messages to the monitoring facility.
At Step 16 the client, the middlebox and the server independently compute the Master Secret from the PSK and the TLS shared secret and derive the initial versions of the application traffic secrets, client_application_traffic_secret_0 and server_application_traffic_secret_0 from the Master Secret and the transcript hash of the handshake messages up to and including the Finished message of the server, as specified in the Key Schedule. (The Master Secret is derived from the same secrets as the Handshake Secret, so the distinction between a Handshake Secret and a Master Secret adds unnecessary computation to the key schedule.) Then they derive the application traffic keys from the application traffic secrets as specified in Section 7.3 of the RFC. The application traffic keys, like the handshake traffic keys, include a client_write_key, a client_write_id, a server_write_key and a server_write_iv.
At Step 17 the server optionally sends application data protected under the server_write_key and server_write_iv derived at Step 16, using the AEAD algorithm associated with the PSK. As in Figure 4 of the RFC, the square brackets denote protection under the application traffic keys and the asterisk indicates that sending data immediately after the ServerHello is optional. At that stage in the handshake the client has not authenticated yet, but is not able to decrypt the data without knowing the PSK.
At Step 18 the client and middlebox decrypt the optional application data using the server_write_key and server_write_iv that they independently derived at Step 16. The middlebox forwards the decrypted application data to the monitoring facility for inspection.
At Step 19 the client sends the EndOfEarlyData message, which is the last message encrypted under the early traffic keys. Before Step 19 the client has been sending early data while the server was sending the ServerHello message, its protected handshake messages, and the optional application data.
At Step 20 the middlebox and the server decrypt the EndOfEarlyData message and the middlebox forwards it to the monitoring facility.
At Step 21 the client sends its Finished message protected under the client_write_key and the client_write_iv derived at Step 13, using the AEAD algorithm associated with the PSK.
At Step 22 the middlebox and the server decrypt the Finished message and the middlebox forwards it to the monitoring facility.
At Steps 23 and 24 the handshake is over and client and server exchange application data protected under the application traffic keys, using the AEAD algorithm associated with the PSK. The middlebox decrypts the data using the keys that it independently derived at Step 16, and forwards it to the monitoring facility.
Secret Derivation in PSK-only Mode
Figure 2 shows the steps of the SD method in the PSK-only mode, in the case where the client sends and the server accepts 0-RTT data.
In PSK-only mode there it no TLS shared secret, and the traffic secrets depend on no secret other than the PSK. In the SD variant the middlebox has the PSK and can therefore derive the traffic secrets by itself, without establishing a visibility shared secret with the server or interacting with the server in any way other than observing the TLS traffic on the wire.
The Key Schedule is the same as in PSK + (EC)DHE mode, except that the second of the two secret inputs to the schedule, which is the TLS shared secret in PSK + (EC)DHE mode and is written “(EC)DHE” in the diagram of page 92 of the RFC, is replaced with a string of zero bytes of length equal to the length in bytes of the hash function associated with the PSK. Thus the SD method in PSK-only mode shown in Figure 2 is derived from the SD method in PSK + (EC)DHE mode of Figure 1 by removing Steps 5-10 and 12, and specifying in Steps 13 and 16 that the handshake secret and the master secret are computed using a string of zero bytes of the above-stated length in lieu of the second input secret.
Secret Transmission in PSK + (EC)DHE Mode
In the ST variant, the server sends the traffic secrets to the middlebox with confidentiality protection, so that the middlebox does not have to derive them and hence does not need the PSK in the PSK modes. It sends the handshake traffic secrets and the application traffic secrets in all cases, and also sends the early traffic secret if the client announces that it will send 0-RTT data by including the early_data extension in the ClientHello message.
Notice that the server must send the early traffic secret when the client sends early data even if the server intends to reject the early data, so that the middlebox can decrypt the data and forward it to the monitoring facility. Rejected early data is a sign of unexpected client behavior, and is therefore of interest to the monitoring facility as an indication of a possible attack.
Early data is encrypted using the AEAD algorithm specified by a cipher suite associated with the first PSK offered by the client, which I will call the early cipher suite. Presumably this is the first cipher suite offered by the client, but RFC 8446 does not require this. And if the server rejects the early data, it may specify a different late cipher suite in the ServerHello message. So the middlebox cannot be sure of what cipher suite is protecting rejected early data by observing the handshake. In the ST variant, the server clears this uncertainty by sending the two-byte code of the early cipher suite along with the early traffic secret. For consistency and to simplify the implementation of the middlebox, the server also includes the two-byte code of the late cipher suite in the messages that convey the handshake traffic secrets and the application traffic secrets.
Notice also that the server must send the application traffic secrets and let the middlebox derive the traffic keys from the traffic secrets instead of sending the traffic keys, because the application traffic secrets established during the handshake are only the first versions of those secrets, client_application_secret_0 and server_application_secret_0, and the middlebox may need the secrets to derive subsequent versions of the secrets and the traffic keys if the client and/or the server perform a key update as described in Sections 4.6.3 and 7.2 of the RFC. For consistency, the server also sends the handshake and early traffic secrets instead of the handshake and early traffic keys in Figure 3.
Confidentiality protection for the transmission of the traffic secrets from the server to the middlebox can be provided by any method agreed upon between the parties. Since the parties must have code for implementing TLS 1.3 cipher suites and symmetric cryptography is not computationally expensive, it would make sense to use to that purpose the most secure TLS 1.3 cipher suite supported by both parties. In the description of Figure 3 I will assume that this is the case and I will refer to that cipher suite as the visibility cipher suite. The traffic secrets are then encrypted using the AEAD algorithm specified by the visibility cipher suite, using the same visibility key for all the secrets and a separate visibility nonce for each transmission. The visibility keys and nonces are derived from the visibility shared secret using HKDF parameterized by the hash function specified by the visibility cipher suite. The same connection ID that identifies the client-server TCP connection in the authenticated but not encrypted messages used by the server and the middlebox to agree on the visibility shared secret is sent along with the encrypted traffic secrets as the AEAD associated data.
Figure 3 shows the ST variant when used in the PSK + (EC)DHE mode in the case where the client sends and the server accepts 0-RTT data.
The steps of the ST method shown in Figure 3 differ from those of the SD method in the same PSK + (EC)DHE mode shown in Figure 1 as follows:
- Steps 8, 17 and 23 are new in Figure 3. The angle brackets denote protection under the visibility key and a visibility nonce using the AEAD algorithm specified by the visibility cipher suite. At Step 8 the server derives the visibility key and the visibility nonce for the early traffic secret and sends the secret protected under key and the nonce. At Step 17 the server derives the visibility nonce for the handshake traffic secrets and sends the secrets protected under the newly derived nonce and the key derived at Step 8. At Step 23 the server derives the visibility nonce for the initial application traffic secrets and sends the secrets protected under that nonce and the key derived at Step 8. The derivations are performed using HKDF, with the hash function specified by the visibility cipher suite as the Hash parameter, the visibility shared secret as the IKM argument of HKDF-Extract, and four different values of the info argument of HKDF-Expand, one for the visibility key, and three for the visibility nonces. The middlebox identifies each message using a plaintext header and derives the key and nonce to be used for decryption using the same info values as the server. Agreement on the info values may be based on ad-hoc configuration of the machines or, preferably, on a visibility interoperability standard.
- Steps 5-9 of Figure 1, which perform the visibility key exchange, have been moved before the step where the client sends the early data in Figure 3, where they are numbered 2-6, so that the visibility shared secret can be used the derive the visibility key and nonce for transmission of the early traffic secret.
- Steps 5 and 6 or Figure 1 differ from the corresponding Steps 2 and 3 of Figure 3 in that there is no selection of the TLS group and the visibility group in Step 2 and no transmission of the choices in Step 3. In the ST variant the middlebox does not need to know the TLS group selected by the server, and the visibility group can be agreed upon once and for all between the middlebox and the server because the visibility shared secret is not an input to handshake computations and there is no need to adjust the choice of visibility group as discussed in the Security Strength section of the previous post.
- The computation of traffic secrets and keys at each of Steps 2, 13 and 16 in Figure 1 has been split into four steps in Figure 3, reflecting the fact that the middlebox does not compute traffic secrets in the ST variant. Step 2 of Figure 1, where the client, the middlebox and the server independently compute the early traffic secret and keys, becomes Steps 7-10 of Figure 3, where client and server compute the secret at Step 7, the server encrypts the secret and sends it to the middlebox at Step 8, the middlebox decrypts the secret at Step 9, and the client, the middlebox and the server derive the keys from the secret at Step 10. Step 13 of Figure 1 similarly becomes Steps 16-19 of Figure 3, and Step 16 of Figure 1 similarly becomes Steps 22-25 of Figure 3.
- While at Step 12 of Figure 1 the client, the middlebox and the server compute the TLS share secret, only the client and the server compute it at the corresponding Step 15 of Figure 3.
Other steps of Figure 3 are the same as corresponding states of Figure 1.
Secret Transmission in PSK-only Mode
The ST variant is the same in PSK-only mode as in PSK + (EC)DHE except that no TLS shared secret is computed by the client and the server and the second secret input to the key schedule is replaced with a string of zero bytes. These are just changes between the two key exchange modes of TLS 1.3, unrelated to visibility. Since the middlebox is provided with traffic secrets and keys by the server, it does not know which of the two PSK modes is being used.
Secret Transmission in (EC)DHE-only Mode
Although the ST variant is primarily motivated by the benefit of not requiring the middlebox to be provided with the PSK in the PSK modes, it can also be used in (EC)DHE-only mode as shown below in Figure 4. In that mode it provides a performance benefit for the middlebox by not requiring it to compute the TLS shared secret as the SD variant does.
ST in (EC)DHE-only mode as shown in Figure 4 differs from ST in PSK + (EC)DHE mode as shown in Figure 3 as follows:
- There is no early data in (EC)DHE-only mode, so Steps 7-12 and 28-29 of Figure 3 go away.
- In (EC)DHE-only mode the Key Schedule takes only one secret input, the PSK input being replaced with a string of zero bytes. Therefore at Steps 10 and 16 of Figure 4 the handshake and application secrets only depend on the TLS share secret. This does not concern the middlebox, however, since in the ST variant it obtains those secrets from the server.
- At Step 14 of Figure 4 the server authenticates with a certificate and may request a certificate from the client, and at Step 22 the client may authenticate with a certificate. The middlebox decrypts the all encrypted handshake messages sent by the server at Step 14 and the client at Step 22, including any Certificate and CertificateVerify messages, and forwards the decrypted messages to the monitoring facility.
- At Step 24 the client and the server independently compute the Resumption Master Secret as specified in the TLS Key Schedule. At Step 25 the server sends the secret to the middlebox protected in the same manner as the traffic secrets under the visibility key and a visibility nonce using the AEAD algorithm specified by the visibility cipher suite. At Step 26 the middlebox decrypts the Resumption Master Secret, making it possible to derive PSKs from any NewSessionTicket messages that it may observe on the wire in the course of the TLS connection.
Previous post in this series: Extending the TLS 1.3 Visibility Solution to Include PSK and 0-RTT.