Lomin Security

Data Execution Prevention

myra | 2008-03-24 14:29

Data Execution Prevention (DEP) is a useful tool to help protect your computer – but it is easily by-passable. There are a few important configuration changes that need to be made to fully use DEP to help insure no one abuses your computer system. These changes are not obvious, intuitive, or readily available from Microsoft. Hopefully this article will help people understand the importance of DEP.

What does DEP Do?

DEP prevents the execution of software in data memory locations.

Malicious programs on the Internet attack errors created by software developers. Typically these attacks create buffer overflows. The attacker uses the overflow situation to place software into space reserved for computer variables. For example, when you log into Windows you type in your user name and password. The user name and password you type in are stored in variables that Windows uses to authorize your access to the system. A malicious user would attack the system by placing a program in those variables. The malicious program inside the variable space would execute when the buffer overflow was triggered. DEP prevents that from happening.

DEP marks segments of memory reserved for variables. This includes stack and heap space. Windows checks the memory for these markers whenever it executes software. If Windows encounters a marker it does not allow execution of the software. This is terrific because DEP stops a lot of malicious software from working.

Windows DEP functionality also supports hardware acceleration/support from Intel and AMD. This means that the protection actually comes at a lower level than the operating system. Windows automagically detects this hardware compatibility and enables it when the system loads.

How do I Enable DEP?

Do not follow Microsoft’s directions. Microsoft in the tradition of keeping things compatible makes DEP easily bypassed. The kernel by default is set to “OptIn”. This means: “protect some of my programs, but let hackers exclude vulnerable programs”. It is typical that convenience should win out against security.

Microsoft misleads the end-user with a DEP user interface that says “Turn on DEP for all programs and services except those I select”. With an empty list one would think that means turn DEP on for everything. Not so.

Microsoft allows programs to by-pass DEP when the kernel is in OptIn mode. Hackers only need to follow the proscribed methods within the Windows Application Compatibility Toolkit (http://support.microsoft.com/kb/294895).

The toolkit has a “DisableNX compatibility fix (sic)” that may be used to exclude programs from DEP. Insanely infuriating, these programmatically excluded programs do not show up on the “exclude vulnerable program” list.

As an example, try installing Yahoo’s Messenger program (versions 7.1-8.1 – 8.1 md5sum 84bd243fb8363ab7cefa19d86ab20767). The installation program by design actually executes software in variable space. DEP is immediately activated when Windows is properly configured. However, when the system is in OptIn Mode the installation program executes normally. The DEP infraction is not reported in the Windows Event Log, nor is the program listed in the Windows exception list (msgr8us.exe or its subsequent temporary files).

The only way to truly protect your system is to use a kernel parameter set within boot options.

How do I Really Enable DEP

Windows XP/Windows 2003
Right click on my computer and select properties. Click the “Advanced” tab and press the “Settings…” button under Startup and Recovery.

Figure 1

Click the “Edit” button under System Startup. That will open up notepad and allow you to edit the boot options for your Windows kernel (you may also find this as a hidden file in the system root directory, typically c:\boot.ini).

Figure 2

Modify the noexecute parameter for your kernel to AlwaysOn. Windows by default has the parameter set to “Optout” . If the noexecute parameter is not there then add it to the kernel’s command line.

Finally, reboot so DEP is always enabled.


Windows Vista
Windows Vista uses a program called bcdedit to handle boot options; this entirely different than Windows XP and Windows 2003. There are no text files to modify or update. More details about bcdedit may be found here: BCDedit

Start a DOS prompt with system administrative privileges. Do this by going to the start menu and then accessories. Right click on “Command Prompt” and select “Run as administrator” from the context menu.

Figure 3

At the command prompt type:

bcdedit /set nx AlwaysOn

Figure 4

If the command worked you will receive the message “The operation completed successfully.” Verify that it really did work by executing the following command

bcdedit /enum

This will display the current boot settings for your computer. See Figure 6 for an example. Notice the last line in the picture that says “nx AlwaysOn”. That means that you have enabled DEP on your computer.

Figure 5

Finally, reboot your computer so the kernel starts with DEP enabled. Verify that DEP is enabled by executing the following command:

wmic OS Get DataExecution_Prevention_SupportPolicy

This will display a number that indicates the status of DEP. The possible numbers are:
0 =AlwaysOff – DEP is disabled for all processes
1 =AlwaysOn – DEP is enabled for all processes
2 =OptIn – DEP is enabled for Windows system components and services (this is the default setting)
3 =OptOut – DEP is enabled for all processes except for those Administrators identify

Figure 6

Conclusion


DEP prevents malicious software from working and harming your computer, but Microsoft makes it easy to by-pass. Set DEP to AlwaysOn to insure that DEP always protects your computer. Without this setting there is risk that you could be hacked with Microsoft’s own technology.

References:

http://technet.microsoft.com/en-us/library/bb490630.aspx
http://windowsitpro.com/article/articleid/97982/q-how-do-i-use-bcdedit-to-set-the-data-execution-prevention-dep-mode.html












 


© Copyright Lomin LLC 2005-2007.