Category Archives: Programming

PAE patch updated for Windows 8.1

This patch allows you to use more than 3/4GB of RAM on an x86 Windows system. Works on Windows Vista SP2, Windows 7 SP0, Windows 7 SP1, Windows 8 and Windows 8.1. Instructions and source code included.

Download: PatchPae2.zip (84099 downloads)

Before using this patch, make sure you have fully removed any other “RAM patches” you may have used. This patch does NOT enable test signing mode and does NOT add any watermarks.

Note: I do not offer any support for this. If this did not work for you, either:

  • You cannot follow instructions correctly, or
  • You cannot use more than 4GB of physical memory on 32-bit Windows due to hardware/software conflicts. See the comments on this page for more information.

PAE patch updated for Windows 8

Note: An updated version for Windows 8.1 is available.

This patch allows you to use more than 3/4GB of RAM on an x86 Windows system. Works on Vista, 7, 8, has been tested on Windows Vista SP2, Windows 7 SP0, Windows 7 SP1 and Windows 8 SP0. Instructions and source code included.

Download: PatchPae2.zip (84099 downloads)

Before using this patch, make sure you have fully removed any other “RAM patches” you may have used. This patch does NOT enable test signing mode and does NOT add any watermarks.

Note: I do not offer any support for this. If this did not work for you, either:

  • You cannot follow instructions correctly, or
  • You cannot use more than 4GB of physical memory on 32-bit Windows due to hardware/software conflicts. See the comments on this page for more information.

How to backup files in C++ using the volume shadow copy service (VSS)

If you’re writing a backup program, you need to be able to read and access files that are in use by other programs. The proper way to do this is to use the Volume Shadow Copy Service. The MSDN pages cover a lot of material that is unnecessary for a simple backup program, so here are some simple steps to get you started.

Before you start: important notes

  • This article assumes that you are using Windows Vista or later. For Windows XP, you may need to make some modifications to this code.
  • You cannot use VSS from a 32-bit program running under a 64-bit version of Windows. On 64-bit Windows, you need to compile a 64-bit version of your program.
  • Usually, your program needs to run under an administrator account. If UAC is enabled, your program needs to be elevated.

Step 1. Initialize backup components

You need to have access to these two functions: CreateVssBackupComponents, VssFreeSnapshotProperties. If you want, you can link to VssApi.lib. Here I will be using GetProcAddress instead. Start with some basic imports and definitions:

#include <vss.h>
#include <vswriter.h>
#include <vsbackup.h>

typedef HRESULT (STDAPICALLTYPE *_CreateVssBackupComponentsInternal)(
    __out IVssBackupComponents **ppBackup
    );

typedef void (APIENTRY *_VssFreeSnapshotPropertiesInternal)(
    __in VSS_SNAPSHOT_PROP *pProp
    );

static _CreateVssBackupComponentsInternal CreateVssBackupComponentsInternal_I;
static _VssFreeSnapshotPropertiesInternal VssFreeSnapshotPropertiesInternal_I;

Note that the functions have “Internal” at the end. Now get the functions and create your IVssBackupComponents object:

HRESULT result;
HMODULE vssapiBase;
IVssBackupComponents *backupComponents;

vssapiBase = LoadLibrary(L"vssapi.dll");

if (vssapiBase)
{
    CreateVssBackupComponentsInternal_I = (_CreateVssBackupComponentsInternal)GetProcAddress(vssapiBase, "CreateVssBackupComponentsInternal");
    VssFreeSnapshotPropertiesInternal_I = (_VssFreeSnapshotPropertiesInternal)GetProcAddress(vssapiBase, "VssFreeSnapshotPropertiesInternal");
}

if (!CreateVssBackupComponentsInternal_I || !VssFreeSnapshotPropertiesInternal_I)
    abort(); // Handle error

result = CreateVssBackupComponentsInternal_I(&backupComponents);

if (!SUCCEEDED(result))
    abort(); // Handle error

Step 2. Connect to VSS

Nothing interesting here.

VSS_ID snapshotSetId;

result = backupComponents->InitializeForBackup();

if (!SUCCEEDED(result))
    abort(); // If you don't have admin privileges or your program is running under WOW64, it will fail here

result = backupComponents->SetBackupState(FALSE, FALSE, VSS_BT_INCREMENTAL);

if (!SUCCEEDED(result))
    abort(); // Handle error

result = backupComponents->SetContext(VSS_CTX_FILE_SHARE_BACKUP);

if (!SUCCEEDED(result))
    abort(); // Handle error

return backupComponents->StartSnapshotSet(&snapshotSetId);

Step 3. Add the volumes

Now you need to add the volumes that you’re interested in by calling AddToSnapshotSet. This will give you a snapshot ID that you need to save. Make sure your volume name has a trailing backslash. In this article we’ll assume that you only have one volume that you’re interested in, but you can add as many as you want.

VSS_ID snapshotId;

result = backupComponents->AddToSnapshotSet(L"D:\\", GUID_NULL, &snapshotId);

if (!SUCCEEDED(result))
    abort(); // Handle error

Step 4. Perform the snapshot

Once you have added your volumes, you need to perform the snapshot using DoSnapshotSet:

IVssAsync *async;

result = backupComponents->DoSnapshotSet(&async);

if (!SUCCEEDED(result))
    abort(); // Handle error

result = async->Wait();
async->Release();

if (!SUCCEEDED(result))
    abort(); // Handle error

Step 5. Use the snapshot(s)

Your snapshot(s) are now ready. For each of the volumes, call GetSnapshotProperties to get a VSS_SNAPSHOT_PROP structure. Use the m_pwszSnapshotDeviceObject field to get the device name (e.g. “\Device\HarddiskVolumeShadowCopy1″) for your snapshot. If you created a snapshot for D:\ and you want to access D:\somefile.txt, open \Device\HarddiskVolumeShadowCopy1\somefile.txt.

VSS_SNAPSHOT_PROP prop;

result = backupComponents->GetSnapshotProperties(snapshotId, &prop);

if (!SUCCEEDED(result))
    abort(); // Handle error

// Use prop.m_pwszSnapshotDeviceObject to access your files.

VssFreeSnapshotPropertiesInternal_I(&prop);

Step 6. Cleaning up

You just need one line:

backupComponents->Release();

For more information:

Please leave a comment if you have any questions.

NiceVS makes Visual Studio 2012 usable again

OK, Visual Studio 2012′s new user interface isn’t that bad. But you need to get some serious work done, and you don’t want to be forced to hover your cursor over each monochrome icon to get a tooltip explaining what it’s for. NOR DO YOU WANT THE MAIN MENUS SHOUTING AT YOU ALL THE TIME. That’s why you need to get NiceVS, which restores the old colorful icons and sensible menu labels.

Choose TOOLS > Extensions and Updates…, click Online, and search for NiceVS. Once installed, you will see the following (hilarious) screen:

NiceVS screenshot

Click Yes, and Visual Studio will be back to normal:

NiceVS 2


Links:

EnumWindows no longer finds Metro/Modern UI windows: a workaround

In the final release of Windows 8, the EnumWindows function no longer lists Metro/Modern UI windows:

Note For Windows 8 and later, EnumWindows enumerates only top-level windows of desktop apps.

This change was made some time around the consumer preview release. I’m not sure why Microsoft did this, but there is a simple workaround. Instead of calling EnumWindows, call this function:

VOID EnumWindowsWithMetro(
    __in WNDENUMPROC lpEnumFunc,
    __in LPARAM lParam
    )
{
    HWND childWindow = NULL;
    ULONG i = 0;

    while (i < 1000 && (childWindow = FindWindowEx(NULL, childWindow, NULL, NULL)))
    {
        if (!lpEnumFunc(childWindow, lParam))
            return;

        i++;
    }
}

The reason this works is that FindWindowEx, unlike EnumWindows, does not ignore Metro windows. We just call FindWindowEx in a loop to keep retrieving the next window in the list until there are no more. Note the i < 1000 condition, which is there to prevent infinite loops (highly unlikely but possible, to my knowledge).

ObQueryTypeInfo and NtQueryObject buffer overrun in Windows 8

**** Update: **** Microsoft is now aware of this bug.

Here’s some output from WinDbg on Windows 8 while I was debugging a driver:

Spot the difference

These two UNICODE_STRINGs are from the OBJECT_TYPE structures of the Section and TmTx (transaction) object types. Can you spot the difference between these two strings? The Section string’s MaximumLength includes two extra bytes for a null terminator, while the TmTx string’s MaximumLength doesn’t.

Now take a look at the code for ObQueryTypeInfo, which is called from NtQueryObject when you pass in the ObjectTypeInformation information class. Look out for the buffer overrun at the end. Also notice that the string “TmTx” happens to be 8 bytes long (when using WCHARs).

#define ALIGN_UP(Length, Type) (((Length) + sizeof(Type) - 1) & ~(sizeof(Type) - 1))

NTSTATUS ObQueryTypeInfo(
__in POBJECT_TYPE ObjectType,
__out_bcount(Length) POBJECT_TYPE_INFORMATION ObjectTypeInfo,
__in ULONG Length,
__out PULONG ReturnLength
)
{
    NTSTATUS status;

    __try
    {
        *ReturnLength += sizeof(OBJECT_TYPE_INFORMATION) + ALIGN_UP(ObjectType->Name.MaximumLength, ULONG_PTR);

        if (Length < *ReturnLength)
        {
            status = STATUS_INFO_LENGTH_MISMATCH;
        }
        else
        {
            ObjectTypeInfo->TotalNumberOfObjects = ObjectType->TotalNumberOfObjects;
            // ...
            ObjectTypeInfo->DefaultNonPagedPoolCharge = ObjectType->TypeInfo.DefaultNonPagedPoolCharge;

            ObjectTypeInfo->TypeName.Buffer = (PWSTR)(ObjectTypeInfo + 1);
            ObjectTypeInfo->TypeName.Length = ObjectType->Name.Length;
            ObjectTypeInfo->TypeName.MaximumLength = ObjectType->Name.MaximumLength;

            memcpy(ObjectTypeInfo + 1, ObjectType->Name.Buffer, ObjectType->Name.Length);
            ((PWSTR)(ObjectTypeInfo + 1))[ObjectType->Name.Length / sizeof(WCHAR)] = 0; // **** Oops! ****

            status = STATUS_SUCCESS;
        }
    }
    __except (EXCEPTION_EXECUTE_HANDLER)
    {
        status = GetExceptionCode();
    }

    return status;
}

The solution is pretty simple. Until Microsoft fixes this bug, if you’re calling NtQueryObject with some ObjectInformationLength, make sure you have ObjectInformationLength + 2 bytes allocated in the buffer that you are passing in.

Introducing WJ’s Backup

Recently I became frustrated with Cobian Backup. It was the only free software I could find that:

  1. supported incremental backups,
  2. supported Volume Shadow Copy,
  3. and didn’t install a bunch of extra, useless startup entries and services.

However, two things sucked:

  1. Incremental backups only seemed to work properly when it used the “archive” attribute. This meant that it couldn’t recover from interrupted backups properly.
  2. It didn’t provide any way of restoring files, so you had to go through the different .7z files and find the file you want to restore.

So, I decided to create my own backup program: “WJ’s Backup”. The possessive in the title is not to indicate some obsessive sense of ownership, but to emphasize the fact that the program was not developed with any other user’s perspective in mind.

Unlike most other backup programs I could find, WJ’s Backup maintains a database with file/directory metadata for each revision. It keeps file versions in a SVN-like way, but without diffs for file content (each new version is stored in full). 7-Zip compression is mandatory.

Project page at SourceForge

PAE patch updated for Windows 7 SP1

Note: An updated version for Windows 8 is available.

This patch allows you to use more than 3/4GB of RAM on an x86 Windows system. Works on Vista and 7, has been tested on Windows Vista SP2, Windows 7 SP0 and Windows 7 SP1. Instructions and source code included.

Download: PatchPae.zip (105281 downloads)

Note: I do not offer any support for this. If this did not work for you, either:

  • You cannot follow instructions correctly, or
  • You cannot use more than 4GB of physical memory on 32-bit Windows due to hardware/software conflicts.

To remove the patch:

  1. Run msconfig, click Boot, highlight the entry named “Windows 7 (PAE Patched)”, and click Delete.
  2. Delete the files ntkrnlpx.exe and winloadp.exe from C:\Windows\system32.

The NT "reserve object"

Windows 7 introduced two new object types: UserApcReserve and IoCompletionReserve. What do these object types have in common? They’re both created using NtAllocateReserveObject. If we look inside this system call we can see that the third argument is an index into two arrays, PspMemoryReserveObjectSizes and PspMemoryReserveObjectTypes. Notice that PspInitPhase0 creates a set number (currently two) of object types, drawing the names from another array: PspMemoryReserveObjectNames. Here’s my reversed definition of NtAllocateReserveObject:

#define USER_APC_RESERVE_TYPE 0
#define IO_COMPLETION_RESERVE_TYPE 1

NTSYSCALLAPI
NTSTATUS
NTAPI
NtAllocateReserveObject(
    __out PHANDLE MemoryReserveHandle,
    __in_opt POBJECT_ATTRIBUTES ObjectAttributes,
    __in ULONG Type
    );

Two new system calls take advantage of the UserApcReserve object and IoCompletionReserve object: NtQueueApcThreadEx and NtSetIoCompletionEx, respectively. In NtQueueApcThreadEx we can see that if a user APC reserve handle/object is supplied, the function uses the space allocated for that object to store the APC, instead of allocating from the pool. Similarly NtSetIoCompletionEx uses the I/O completion reserve object’s already allocated space to store the I/O completion mini-packet, instead of allocating from the pool. It is now clear what the purpose of these reserve objects are: to allow processes to reserve memory before performing certain system calls in order to avoid out-of-memory problems occurring at bad spots (or critical code). Here’s my reversed definitions for the two system calls:

NTSYSCALLAPI
NTSTATUS
NTAPI
NtQueueApcThreadEx(
    __in HANDLE ThreadHandle,
    __in_opt HANDLE UserApcReserveHandle,
    __in PPS_APC_ROUTINE ApcRoutine,
    __in_opt PVOID ApcArgument1,
    __in_opt PVOID ApcArgument2,
    __in_opt PVOID ApcArgument3
    );

NTSYSCALLAPI
NTSTATUS
NTAPI
NtSetIoCompletionEx(
    __in HANDLE IoCompletionHandle,
    __in HANDLE IoCompletionReserveHandle,
    __in PVOID KeyContext,
    __in_opt PVOID ApcContext,
    __in NTSTATUS IoStatus,
    __in ULONG_PTR IoStatusInformation
    );

Note that NtQueueApcThread now calls NtQueueApcThreadEx, which is why the reserve object is optional.