Windows hole discovered after 17 years

Microsoft isn't having an easy time of it these days. In addition to the unpatched hole in Internet Explorer, a now published hole in Windows allows users with restricted access to escalate their privileges to system level – and this is believed to be possible on all 32-bit versions of Windows from Windows NT 3.1 up to, and including Windows 7. While the vulnerability is likely to affect home users in only a minor way, the administrators of corporate networks will probably have their hands full this week.

The problem is caused by flaws in the Virtual DOS Machine (VDM) introduced in 1993 to support 16-bit applications (real mode applications for 8086). VDM is based on the Virtual 8086 Mode (VM86) in 80386 processors and, among other things, intercepts hardware routines such as BIOS calls. Google security team member Tavis Ormandy has found several vulnerabilities in this implementation that allow an unprivileged 16-bit program to manipulate the kernel stack of each process via a number of tricks. This potentially enables attackers to execute code at system privilege level.

The workaround requires users to start the group policy editor and enable the "Prevent access to 16-bit applications" option in the Computer Configuration\Administrative Templates\Windows Components\Application Compatibility section. When tested with these settings by the heise Security team, the exploit no longer functioned. The settings reportedly don't cause any major compatibility problems for most users while no 16-bit applications are being used.

Update - The above option is only available through the group policy editor on Windows 2003 systems. Some versions of Windows do not include a group policy editor. As an alternative, users can also create a registry key under \HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows\AppCompat with a D-Word value of VDMDissallowed = 1. Under Windows XP, to prevent the system from being vulnerable to the exploit, users can place the following text:
Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SOFTWARE\Policies\Microsoft\Windows\AppCompat]
"VDMDisallowed"=dword:00000001
into a file called vdmdisallow.reg and double click the file. Windows will then automatically import the key (admin rights are required to perform this action).