SHA-512 != SHA-512

Top

Is it possible to turn a crypt(3) SHA-2 password hash into an OpenLDAP one? Slightly surprisingly, the answer is no, this is not feasible.

It's fairly obvious that you can't translate, for example, an MD5-hash of a password into a SHA-256 hash, without knowing the password.

However you would think that you'd be able to translate a SHA-512 hash (according to crypt(3)) into a SHA-512 hash (according to OpenLDAP): surely it's just a matter of identifying the salt and hash in one format, and reordering them into the format the other expects. No? What?!

The short reason: although these two hashes are both labelled ‘SHA-512’, and they do use the same underlying cryptographic primitive, they use that primitive in rather different ways, and so should be regarded as, in effect, different hashing functions. Thus you can't turn a crypt(3) SHA-512 hash into an OpenLDAP one for essentially the same reason you can't turn a SHA-512 hash into an MD5 one.

A few details

In OpenLDAP, the format of the userPassword attribute, though not quite documented, is relatively straightforward: it's the fixed-length hash of the password followed by the salt (see eg contrib/slapd-modules/passwd/sha2 in the OpenLDAP sources). This is also evident in notes such as this OpenLDAP FAQ-o-matic article which shows how to generate a {SSHA} hash from password + salt. The OpenLDAP security documentation discusses the (S)MD5 and (S)SHA schemes, but doesn't explicitly commit itself on the byte-for-byte format of the stored hashes.

For systems which use crypt(3), however, you'll see ‘password hashes’ of the form

$id$salt$hash

where $5$ indicates SHA-256, $6$ indicates SHA-512, and so on. The hash element of that is the base64 encoded form of some function of key, salt, and (in some variants) number of encryption rounds, but the actual function is (and here's where it all goes wrong) not documented.

That really means that the $id$salt$hash contents is an implementation detail of the libcrypt library.

So we can look at the glibc sources for crypt. There, we discover (in crypt_util.c) that the base64 alphabet is indeed odd, using "./0-9A-Za-b" rather than the more usual (and PHC-specified) "A-Za-z0-9+/" (with occasional variants in the last two characters). Not helpful, but we could cope if this were the only difference.

However, the sha256-crypt.c function in that library (looking only at the SHA-256 hash for the moment) takes the key and salt and embarks on a long-winded procedure which boils down to re-applying the underlying hash function some large number of times. The point of this is to make the overall hashing function computationally expensive, so that it's expensive to brute-force the key from the hash.

What that means is that although the crypt(3) $5$ and OpenLDAP ‘SHA-256’ hashes are on the face of it using the same cryptographic primitive, they are actually using it in sufficiently different ways that they amount to different hash functions so that, as with the point at the top, you can't translate one to the other without knowing the password. A similar point can be made about $6$/SHA-512 passwords.

This is a bit annoying.

Parenthetical note: all is not lost

As it happens, it is possible to convert the $1$/SMD5 ‘hashes’ between crypt(3) format and OpenLDAP:

$1$salt$hash

corresponds to

'{SMD5}' + hash + salt

where + is simple string concatenation. But we arguably shouldn't be using MD5 hashes any more, so this doesn't much help.

Note also that you can port your crypt(3) hashes to OpenLDAP unchanged, using

{CRYPT}$1$salt$hash

...if your libc is compatible (which it typically will be, if OpenLDAP is running on Linux, but typically won't be, otherwise), but OpenLDAP won't generate new passwords this way, so this is a one-way street.

Why is this done?

Just in case it's not obvious...

You hash passwords, server-side, so that if (or indeed when) your password database is stolen, the thieves have a hard time working out what your users' passwords are. They want to know those passwords so that they can log in to your service as those users (presuming you don't know the passwords are stolen, or haven't changed the passwords), or anticipate that the users will have reused those passwords on other sites.

You use hashes in the password database so that the thieves have at least some work to do to get the passwords back. Depending one what that hashing function is, the thieves have more or less work to do. Re-applying MD5 or a SHA hash function multiple times (as crypt(3) does) increases the difficulty; using a different algorithm, such as bcrypt, increases it substantially further.

There's a good discussion of the issues on Security.stackexchange, and a detailed discussion on crackstation.

Norman, 2019 April 27