Windows Metafile Vulnerability - The Vulnerability

The Vulnerability

According to assessments by F-Secure, the vulnerability is an inherent defect in the design of WMF files, because the underlying architecture of such files is from a previous era, and includes features which allow actual code to be executed whenever a WMF file opens. The original purpose of this was mainly to handle the cancellation of print jobs during spooling.

According to Secunia, "The vulnerability is caused due to an error in the handling of Windows Metafile files ('.wmf') containing specially crafted SETABORTPROC 'Escape' records. Such records allow arbitrary user-defined function to be executed when the rendering of a WMF file fails." According to the Windows 3.1 SDK documentation, the SETABORTPROC escape was obsoleted and replaced by the function of the same name in Windows 3.1, long before the WMF vulnerability was discovered. However the obsoleted escape code was retained for compatibility with 16 bit programs written for (or at least backwards compatible with) Windows 3.0. This change happened at approximately the same time as Microsoft was creating the 32 bit reimplementation of GDI for Windows NT, and it is likely that the vulnerability occurred during this effort.

The 'Escape' mechanism in question allows applications (not metafiles) to access output device features not yet abstracted by GDI, such as hardware accelerated Bézier curves, encapsulated postscript support, etc. This is done by passing an opcode, a size and a pointer to some data to the call, which will usually just pass it on to the driver. Because most Escape calls produce actual graphics, the general escape mechanism is allowed in metafiles with little thought originally given to the possibility of using it for things like SETABORTPROC, modern non-vulnerable metafile interpreters now checks the opcode against a blacklist or whitelist, while keeping the full set of opcodes available to regular code that calls the GDI escape functions directly (because such code is already running in the same way as the code it could make GDI call, there is no security risk in that case).

It is worth noting that 16 bit Windows (except the rarely used Real mode of Windows 3.0) was immune to the vulnerability because the pointer specified in the metafile can only point to data within the metafile, and 16 bit Windows always had a full no-execute-data enforcement mandated by Intel's design of the 80286 protected mode architecture. Windows NT for CPU architectures other than 32 bit x86 (such as MIPS, PowerPC, Alpha, Itanium and x86_64) had similar immunity because those architectures had the no-execute functionality missing from older x86 processors.

The vulnerability is CVE-2005-4560 in the Common Vulnerabilities and Exposures database, US-CERT reference VU#181038 and Microsoft Knowledge Base Article 912840. It was first observed in the wild by researchers at Sunbelt Software on December 28, 2005 and announced publicly by the company's president Alex Eckelberry.