30

Consider an application using OpenSSL which has a bug. A packet capture of the full SSL session is available, as well as a core dump and debugging symbols for the application and libraries. A RSA private key is also available, but since a DHE cipher suite is in use, this cannot be used to decrypt the packet capture using Wireshark.

Thomas suggests in this post that it is possible to extract keys from RAM. How could this be done for OpenSSL? Assume that the address of the SSL data structure is known and TLS 1.0 is in use.

Lekensteyn
  • 5,958
  • 5
  • 38
  • 62

2 Answers2

29

Note: as of OpenSSL 1.1.1 (unreleased), it will be possible to set a callback function that receives the key log lines. See the SSL_CTX_set_keylog_callback(3) manual for details. This can be injected as usual using a debugger or a LD_PRELOAD hook. Read on if you are stuck with an older OpenSSL version.

For a walkthrough of the LD_PRELOAD approach for Apache on Debian Stretch, see my post to Extracting openssl pre-master secret from apache2. Technical details follow below.


If you have just gdb access to the live process or a core dump, you could read data from data structures. It is also possible to use an interposing library.

In the following text, the basic idea of key extraction using GDB is described, then an automated script is given to perform the capture.

Using GDB (basic idea)

Based on this Stackoverflow post, I was able to construct a function that could print a line suitable for Wireshark's key logfile. This is especially useful while analyzing core dumps. In GDB, execute:

python
def read_as_hex(name, size):
    addr = gdb.parse_and_eval(name).address
    data = gdb.selected_inferior().read_memory(addr, size)
    return ''.join('%02X' % ord(x) for x in data)

def pm(ssl='s'):
    mk = read_as_hex('%s->session->master_key' % ssl, 48)
    cr = read_as_hex('%s->s3->client_random' % ssl, 32)
    print('CLIENT_RANDOM %s %s' % (cr, mk))
end

Then later on, after you step upwards in the stack until you get a SSL structure, invoke the python pm() command. Example:

(gdb) bt
#0  0x00007fba7d3623bd in read () at ../sysdeps/unix/syscall-template.S:81
#1  0x00007fba7b40572b in read (__nbytes=5, __buf=0x7fba5006cbc3, __fd=<optimized out>) at /usr/include/x86_64-linux-gnu/bits/unistd.h:44
#2  sock_read (b=0x7fba60191600, out=0x7fba5006cbc3 "\027\003\001\001\220T", outl=5) at bss_sock.c:142
#3  0x00007fba7b40374b in BIO_read (b=0x7fba60191600, out=0x7fba5006cbc3, outl=5) at bio_lib.c:212
#4  0x00007fba7b721a34 in ssl3_read_n (s=0x7fba60010a60, n=5, max=5, extend=<optimized out>) at s3_pkt.c:240
#5  0x00007fba7b722bf5 in ssl3_get_record (s=0x7fba60010a60) at s3_pkt.c:507
#6  ssl3_read_bytes (s=0x7fba60010a60, type=23, buf=0x7fba5c024e00 "Z", len=16384, peek=0) at s3_pkt.c:1011
#7  0x00007fba7b720054 in ssl3_read_internal (s=0x7fba60010a60, buf=0x7fba5c024e00, len=16384, peek=0) at s3_lib.c:4247
...
(gdb) frame
#4  0x00007fba7b721a34 in ssl3_read_n (s=0x7fba60010a60, n=5, max=5, extend=<optimized out>) at s3_pkt.c:240
240     in s3_pkt.c
(gdb) python pm()
CLIENT_RANDOM 9E7EFAC51DBFFF84FCB9...81796EBEA5B15E75FF71EBE 6ED2EA80181...

Note: do not forget to install OpenSSL with debugging symbols! On Debian derivatives it would be named something like libssl1.0.0-dbg, Fedora/RHEL call it openssl-debuginfo, etc.

Using GDB (improved, automated approach)

The basic idea which is described above works for small, manual tests. For bulk extraction of keys (from a SSL server for example), it would be nicer to automate extraction of these keys.

This is done by this Python script for GDB: https://git.lekensteyn.nl/peter/wireshark-notes/tree/src/sslkeylog.py (see its headers for installation and usage instructions). It basically works like this:

  • Install breakpoints on several functions where new pre-master keys can arise.
  • Wait for the function to finish and write these keys (if not known before) to a file (which follows the SSLKEYLOGFILE format from NSS).

Paired with Wireshark, you perform a live capture from a remote server by running these commands:

# Start logging SSL keys to file premaster.txt. Be careful *not* to
# press Ctrl-C in gdb, these are passed to the application. Use
# kill -TERM $PID_OF_GDB (or -9 instead of -TERM if that did not work).
(server) SSLKEYLOGFILE=premaster.txt gdb -batch -ex skl-batch -p `pidof nginx`
# Read SSL keys from the remote server, flushing after each written line
(local) ssh user@host stdbuf -oL tailf premaster.txt > premaster.txt
# Capture from the remote side and immediately pass the pcap to Wireshark
(local) ssh user@host 'tcpdump -w - -U "tcp port 443"' |
    wireshark -k -i - -o ssl.keylog_file:premaster.txt

Using LD_PRELOAD

SSL/TLS can only negotiate keys at the SSL handshake steps. By interposing the library interfaces of OpenSSL (libssl.so) that performs said actions you will be able to read the pre-master key.

For clients, you need to interpose SSL_connect. For servers you need to interpose SSL_do_handshake or SSL_accept (depending on the application). To support renegotiation, you will also have to intercept SSL_read and SSL_write.

Once these functions are intercepted using a LD_PRELOAD library, you can use dlsym(RTLD_NEXT, "SSL_...") to lookup the "real" symbol from the SSL library. Call this function, extract the keys and pass the return value.

An implementation of this functionality is available at https://git.lekensteyn.nl/peter/wireshark-notes/tree/src/sslkeylog.c.

Note that different OpenSSL versions (1.0.2, 1.1.0, 1.1.1) are all incompatible with each other. If you have multiple OpenSSL versions installed and need to build an older version, you might have to override the header and library paths:

make -B CFLAGS='-I/usr/include/openssl-1.0 -DOPENSSL_SONAME=\"libssl.so.1.0.0\"'
Lekensteyn
  • 5,958
  • 5
  • 38
  • 62
  • 2
    Here's some explanation on LD_PRELOAD in case it's helpful to anyone reading this post http://stackoverflow.com/questions/426230/what-is-the-ld-preload-trick – sa289 Oct 22 '15 at 22:03
  • 1
    A warning if you are going to use the gdb trick: it currently kills a program with a SIGTRAP signal if you kill *gdb* at the wrong moment (i.e. when gdb is not executing a breakpoint). If you know an automated way to detach gdb and continue the program normally, please let me know. – Lekensteyn Oct 22 '15 at 22:12
  • 1
    @Lekensteyn typo in `invoke the python mk() command.`: `mk()` should be `pm()`. Thanks for a great answer :-). – pevik Aug 23 '16 at 18:10
  • @pevik Thanks, fixed! Next time you can also propose an edit :-) – Lekensteyn Aug 23 '16 at 21:08
  • Thanks for the post. Do you have an idea where to look for Openssl debug symbols in Debian Stretch? When looking to older versions, it suggests there should be package libssl-1.1.0-dbg, but I can't find it or any similar package. – v6ak Jun 23 '17 at 20:31
  • 1
    @v6ak Debian Stretch moved tot a separate repo for dbgsym packages. Don't know official docs, but here is some info: https://wiki.debian.org/AutomaticDebugPackages – Lekensteyn Jun 24 '17 at 12:17
  • @Lekensteyn Thank you. I've added the repo and then I can install the libssl-1.1-dbgsym package. – v6ak Jun 24 '17 at 16:06
  • Just one more note: I was able to log it for apps using Openssl 1.0, but not for apps using Openssl 1.1. Even one non-public app compiled against both versions depends on what version I compile it against. – v6ak Jun 26 '17 at 11:51
  • I have tried the GDB automated approach with both OpenSSL 1.0 and OpenSSL 1.1, even within the same app compiled against both versions, and it has worked only under openssl 1.0. With OpenSSL 1.1, it has logged some data, clientrandom seems to be correct (have verified just briefly), but Wireshark is not able to decrypt the communication. Maybe the script logs something incorrectly or maybe Wireshark does not like something in the communication. – v6ak Jun 26 '17 at 12:08
  • @v6ak Have you tried the LD_PRELOAD approach linked in my wireshark-notes repo? In OpenSSL 1.1 you need to use a special API – Lekensteyn Jun 26 '17 at 12:14
  • @Lekensteyn Thanks for noting, maybe it would be worth adding to the post. I'll probably try the LD_PRELOAD approach, but rather for performance reasons, as it seems gdb makes the app much much sloooower. – v6ak Jun 26 '17 at 16:22
4

The master secret is in SSL->session->master_key.

Alternatively, you can get the session struct as follows:

SSL_SESSION ss = SSL_get_session(SSL);

A stated above, there is a master_key field in the SSL_SESSION struct.

Or you can print the session details (including the master_secret) using SSL_SESSION_print() or SSL_SESSION_print_fp().

I do not think the pre_master_secret can be retrieved once the master_secret has been computed.

Erwan Legrand
  • 401
  • 2
  • 13
  • 1
    Yup, it was easier to find than expected. Since only a coredump is available, I executed `p/x s->session->master_key` to get a master key (it is a fixed `char[48]` and TLS had only a 48-byte master secret). Now let's see if I can link it to the ClientRandom value from the same data structure. – Lekensteyn Jan 27 '15 at 12:02