Password checking with CommonCrypto

I previously described a system for storing and checking credentials on Mac OS and iOS based on using many rounds of a hashing function to generate a key from the password. Time has moved on, and Apple has extended the CommonCrypto library to provide a simple, standard and supported way of doing this. If this is still a problem you need to solve, you should look at doing it this way instead of following the earlier post.

We’re going to use a key-stretching function called PBKDF2 to make an encryption key of a standard length. This key probably will be much longer than the user’s password. However, it can’t possibly be any more random: it must be deterministically derived from that password so we’re stuck with the same amount of randomness that this password contains. A good key-stretching function should “smooth out” the randomness from the initial password, so that you can’t guess anything about the password given the function’s output.

We’re then going to use this key to calculate an HMAC of some known data. An HMAC is a bit like a digital signature, in that it depends both on the key used and the input data. We’ll store the HMAC but not the password and not the key derived from the password. Whenever the password is needed in the future, it must be provided by a user and cannot be derived from any of the data in the app.

The idea, then, is that when you set a new password, the app calculates this key, uses that to calculate an HMAC and stores the HMAC. When you try to use the app, you present a password, from which the app generates a key and an HMAC of the same data. If this HMAC matches the one that was previously calculated, the same keys were used, which (hopefully) means that the same password was supplied.

The faster you scream, the slower we go.

One of the problems that a key-stretching function must address is that computers are really, really fast. Normally computers being really, really fast is a benefit, but the faster it is to compute all of the above stuff the more guesses an attacker can try at the password in some amount of time. We therefore want to make this function slow enough that brute force attacks are limited, but not so slow that people get frustrated with the app’s performance.

PBKDF2 has a tuneable parameter – the number of rounds of a hashing function it uses internally to stretch the key. With CommonCrypto you can ask for a number of rounds that will result in the function taking (approximately) a certain amount of time to work on a password of a certain length.

Requirements for the rounds parameter

  • The number of rounds used when checking a password should be the same as the number used when generating the stored HMAC, otherwise the keys generated won’t match.
  • The above means that you need to choose a single value to use across app installs and hardware – unless you’re happy with losing access to all user data when a customer upgrades their iPad.
  • You will want to revise this parameter upwards as faster hardware becomes available. This conflicts with the first requirement: you’ll need a fallback mechanism to try the same password with different “versions” of your rounds parameter so that you can upgrade users’ credentials with your app.

With those in mind, you can construct an algorithm to choose a tuning parameter based on this call:

        const uint32_t oneSecond = 1000;
         rounds = CCCalibratePBKDF(kCCPBKDF2,
                                   predictedPasswordLength,
                                   predictedSaltLength,
                                   kCCPRFHmacAlgSHA256,
                                   kCCKeySizeAES128,
                                   oneSecond);

You can probably know what length of salt you’ll use: salt should be a block of random data that you supply, that is used as additional input to the key-stretching function. If two different users supply the same password, the salt stops the function from generating the same output. You probably can’t know what length of password users will use, but you can guess based on experience, data and knowledge of any password strength rules incorporated into your app.

Notice that you should derive this rounds property on the target hardware. The number of rounds that take a second to run through on your brand new iMac will take significantly longer on your customer’s iPhone 3GS.

Generating the HMAC data

Both storing and checking passwords use the same internal function:

- (NSData *)authenticationDataForPassword: (NSString *)password salt: (NSData *)salt rounds: (uint) rounds
{
    const NSString *plainData = @"Fuzzy Aliens";
    uint8_t key[kCCKeySizeAES128] = {0};
    int keyDerivationResult = CCKeyDerivationPBKDF(kCCPBKDF2,
                                                   [password UTF8String],
                                                   [password lengthOfBytesUsingEncoding: NSUTF8StringEncoding],
                                                   [salt bytes],
                                                   [salt length],
                                                   kCCPRFHmacAlgSHA256,
                                                   rounds,
                                                   key,
                                                   kCCKeySizeAES128);
    if (keyDerivationResult == kCCParamError) {
        //you shouldn't get here with the parameters as above
        return nil;
    }
    uint8_t hmac[CC_SHA256_DIGEST_LENGTH] = {0};
    CCHmac(kCCHmacAlgSHA256,
           key,
           kCCKeySizeAES128,
           [plainData UTF8String],
           [plainData lengthOfBytesUsingEncoding: NSUTF8StringEncoding],
           hmac);
    NSData *hmacData = [NSData dataWithBytes: hmac length: CC_SHA256_DIGEST_LENGTH];
    return hmacData;
}

Storing credentials for a new password simply involves generating a salt, computing the authentication data then writing it somewhere:

- (void)setPassword: (NSString *)password
{
    //generate a random salt…

    //do any checking (on complexity, or whether the passwords entered in two fields match)…

    NSData *hmacData = [self authenticationDataForPassword: password salt: salt rounds: [self roundsForKeyDerivation]];
    //store the HMAC and the salt, perhaps by concatenating them and putting them in the keychain…

}

Then testing the credentials involves applying the generation function to the password guess, and comparing the result with what you previously stored:

- (BOOL)checkPassword: (NSString *)password
{
    //recover the HMAC and the salt from wherever you stored them…

    NSData *guessedHmac = [self authenticationDataForPassword: password salt: salt rounds: [self roundsForKeyDerivation]];
    return [guessedHmac isEqualToData: hmacData];
}

Conclusion

If you can avoid storing a password in your app, even in the keychain, you should; there’s then a much reduced chance that the password can be recovered from the app by an attacker. Deriving a key from the password using PBKDF2 then testing whether you can use that key to obtain a known cryptographic result obviates the need to store the password itself. Mac OS X and iOS provide an easy way to use PBKDF2 in the CommonCrypto library.

The key derived from the password could even be used to protect the content in the app, so none of the documents are available without the password being presented. Doing this offers additional confidentiality over simply using the password for access control. Building a useful protocol around this key requires key wrapping, the subject of a future post.

Storing and testing credentials: Cocoa Touch Edition

This article introduces the concept of key stretching, using code examples to explain the ideas. For code you can use in an app that more closely resembles current practice, see Password checking with CommonCrypto.

There’s been quite the media circus regarding the possibility that Sony was storing authentication credentials for its PlayStation Network credentials in plain text. I was even quoted in a UK national daily paper regarding the subject. But none of this helps you: how should you deal with user passwords?

The best solution is also the easiest: if you can avoid it, don’t store the passwords yourself. On the Mac, you can use the OpenDirectory framework to authenticate both local users and users with accounts on the network (where the Mac is configured to talk to a networked directory service). This is fully covered in Chapter 2 of Professional Cocoa Application Security.

On the iPhone, you’re not so lucky. And maybe on the Mac there’s a reason you can’t use the local account: your app needs to manage its own password. The important point is that you never need to see that password—you need to know that the same password was presented in order to know (or at least have a good idea) that the same user is at the touchscreen, but that’s not the same as seeing the password itself.

That means that we don’t even need to use encryption where we can protect the password and recover it when we must check the password. Instead we can use a cryptographic one-way hash function to store data derived from the password: we can never get the password back, but we can always generate the same hash value when we see the same password.

Shut up Graham. Show me the code.

Here it is. This code is provided under the terms of the WTFPL, and comes without any warranty to the extent permitted by applicable law.

The first thing you’ll need to do is generate a salt. This is a random string of bytes that is combined with the password to hash: the point here is that if two users on the same system have the same password, the fact that the salt is different means that they still have different hashes. So you can’t do any statistical analysis on the hashes to work out what some of the passwords are. Otherwise, you could take your knowledge that, say, 10% of people use “password” as their password, and look for the hash that appears 10% of the time.

It also protects the password against a rainbow tables attack by removing the one-one mapping between a password and its hash value. This mitigation is actually more important in the real world than the one above, which is easier to explain :-).

This function uses Randomization Services, so remember to link Security.framework in your app’s link libraries build phase.

NSString *FZARandomSalt(void) {
    uint8_t bytes[16] = {0};
    int status = SecRandomCopyBytes(kSecRandomDefault, 16, bytes);
    if (status == -1) {
        NSLog(@"Error using randomization services: %s", strerror(errno));
        return nil;
    }
    NSString *salt = [NSString stringWithFormat: @"%2x%2x%2x%2x%2x%2x%2x%2x%2x%2x%2x%2x%2x%2x%2x%2x",
                      bytes[0],  bytes[1],  bytes[2],  bytes[3],
                      bytes[4],  bytes[5],  bytes[6],  bytes[7],
                      bytes[8],  bytes[9],  bytes[10], bytes[11],
                      bytes[12], bytes[13], bytes[14], bytes[15]];
    return salt;
}

Now you pass this string, and the password, to the next function, which actually calculates the hash. In fact, it runs through the hashing function 5,000 times. That slows things down a little—on an A4-equipped iPad it takes nearly 0.088s to compute the hash—but it also slows down brute-force attacks.

NSData *FZAHashPassword(NSString *password, NSString *salt) {
    NSCParameterAssert([salt length] >= 32);
    uint8_t hashBuffer[64] = {0};
    NSString *saltedPassword = [[salt substringToIndex: 32] stringByAppendingString: password];
    const char *passwordBytes = [saltedPassword cStringUsingEncoding: NSUTF8StringEncoding];
    NSUInteger length = [saltedPassword lengthOfBytesUsingEncoding: NSUTF8StringEncoding];
    CC_SHA512(passwordBytes, length, hashBuffer);
    for (NSInteger i = 0; i < 4999; i++) {
        CC_SHA512(hashBuffer, 64, hashBuffer);
    }
    return [NSData dataWithBytes: hashBuffer length: 64];
}

Where do I go now?

You now have two pieces of information: a random salt, like edbfe42b3da2995a159c16c0a7184211, and a hash of the password, like 855fec563d91576db0e66d8745a3a9cb71dbe40d7cb2615a82b1c87958dd2e8e56db02860739422b976f182a7055dd223a3037dd3dcc5e1ca28aaaf0bade8a08. Store both of these on the machine where the password will be tested. In principle there isn’t too much worry about this data being leaked, because it’s super-hard to get the password out of it, but it’s still best practice to restrict access as much as you can so that attackers have to brute-force passwords on your terms.

When you come to verify the user’s password, pass the string presented by the user and the stored salt to FZAHashPassword(). You should get the same hash out that you previously calculated, if the same password was presented.

Anything else?

Yes. The weakest part of this solution is no longer the password storage: it’s the password itself. The salt+hash shown above is actually for the password “password” (try it yourself), and no amount of software is going to change the fact that that’s a questionable choice of password…well, software that finally does away with password authentication will, but that’s a different argument.

If you want to limit a user’s ability to choose a simple password, you have to do this at password registration and change time. Just look at the (plain-text) password the user has given you and decide whether you want to allow its use.

Why passwords aren’t always the right answer.

I realised something yesterday. I don’t know my master password.

Users of Mac OS X can use FileVault, a data protection feature that replaces the user’s home folder with an encrypted disk image. Encrypted disk images are protected by AES-128 or AES-256 encryption, but to get at the private key you need to supply one of two pieces of information. The first is the user’s login password, and the second is a private key for a recovery certificate. That private key is stored in a dedicated keychain, which is itself protected by….the master password. More information on the mechanism is available both in Professional Cocoa Application Security and Enterprise Mac.

Anyway, so this password is very useful – any FileVault-enabled home folder can be opened by the holder of the master password. Even if the user has forgotten his login password, has left the company or is being awkward, you can get at the encrypted content. It’s also hardly ever used. In fact, I’ve never used my own master password since I set it – and as a consequence have forgotten it.

There are a few different ways for users to recall passwords – by recital, by muscle memory or by revision. So when you enter the password, you either remember what the characters in the password are, where your hands need to be to type it or you look at the piece of paper/keychain where you wrote it down. Discounting the revision option (the keychain is off the menu, because if you forget your login password you can’t decrypt your login keychain in order to view the recorded password), the only ways to reinforce a password in your memory are to use it. And you never use the FileVault master password.

I submit that as a rarely-used authentication step, the choice of a password to protect FileVault recovery is a particularly bad one. Of course you don’t want attackers able to use the recovery mechanism, but you do want that when you really need to recover your encrypted data, the OS doesn’t keep you out, too.

Why do we annoy our users?

I assume that, with my audience being mainly Mac users, you are not familiar with Microsoft Security Assessment Tool, or MSAT. It’s basically a free tool for CIOs, CSOs and the like to perform security analyses. It presents two questionnaires, the first asking questions about your company’s IT infrastructure (“do you offer wireless access?”), the second asking about the company’s current security posture (“do you use WPA encryption?”). The end result is a report comparing the company’s risk exposure to the countermeasures in place, highlighting areas of weakness or overinvestment. The MSAT app itself isn’t too annoying.

Mostly. One bit is. Some of the questions are accompanied by information about the relevant threats, and industry practices that can help mitigate the appropriate threats. Information such as this:

In order to reduce the ability to 'brute-force' the credentials for privileged accounts, the passwords for such accounts should be changed regularly.

So, how does changing a password reduce the likelihood of a brute-force attack succeeding? Well, let’s think about it. The attacker has to choose a potential password to test. Obviously the attacker does not know your password a priori, or the attack wouldn’t be brute-force; so the guess is independent of your password. You don’t know what the attacker has, hasn’t, or will next test—all you know is that the attacker will exhaust all possible guesses given enough time. So your password is independent of the guess distribution.

Your password, and the attacker’s guess at your password, are independent. The probability that the attacker’s next guess is correct is the same even if you change your password first. Password expiration policies cannot possibly mitigate brute-force attacks.

So why do we enforce password expiration policies? Actually, that’s a very good question. Let’s say an attacker does gain your password.

OK, "an attacker does gain your password."

The window of opportunity to exploit this condition depends on the time for which the password is valid, right? Wrong: as soon as the attacker gains the password, he can install a back door, create another account or take other steps to ensure continued access. Changing the password post facto will defeat an attacker who isn’t thinking straight, but ultimately a more comprehensive response should be initiated.

So password expiration policies annoy our users, and don’t help anyone.