Setting a Debug Breakpoint in a Win32 API in Visual Studio

Recently, I was stress-testing an application of mine and noticed that the handle count kept growing. After a quick check in Process Explorer I saw that the handle was for a registry key. I was curious as to where the key was being opened in the first place but, it wasn’t in my code so I couldn’t just set a breakpoint on a line in my source. Wouldn’t it be nice if you could set a breakpoint on a function in Visual Studio but that’s not in your code?

You can! If you have the NT symbols loaded for your system, then you can set a breakpoint on any public function. In my particular case, I set a breakpoint on RegOpenKey and RegOpenKeyEx for both ANSI and UNICODE versions of the function.

{,,Advapi32.dll}_RegOpenKeyA@12
{,,Advapi32.dll}_RegOpenKeyW@12
{,,Advapi32.dll}_RegOpenKeyExA@20
{,,Advapi32.dll}_RegOpenKeyExW@20

Note that you have to use the decorated name of the function. For Win32 API functions, you have to put an underscore before the function name and remember that many functions are macro redirected to ANSI or UNICODE versions with an A or W extension. Since most functions use the __stdcall calling convention the @<number> is generally 4x the number of arguments. So for RegOpenKey for example, it’s redirected to RegOpenKeyA or RegOpenKeyW and has 3 arguments, so it’s decorated name is _RegOpenKeyA@12 and _RegOpenKeyW@12.

Similarly, you can set breakpoints in third-party libraries for public functions which assumes that you have a PDB for it.

UPDATE 2011-11-13: You only have to use the decorated name of the function for 32-bit applications. For 64-bit applications, the names are undecorated, like this:

{,,Advapi32.dll}RegOpenKeyA
{,,Advapi32.dll}RegOpenKeyW
{,,Advapi32.dll}RegOpenKeyExA
{,,Advapi32.dll}RegOpenKeyExW

Related: Can I Add a Breakpoint on CreateProcess in VS

Comments

3 responses to “Setting a Debug Breakpoint in a Win32 API in Visual Studio”

  1. Justin Dearing Avatar

    Thanks for writing this! However, I think you have the decoration thing backwards. I am debugging Visual Studio 2012 with another copy of Visual Studio 2012. Both are 32 bit processes (I’ve triple checked because I was so sure). The breakpoint would not be set unless I used the decorated name. Also, wikipedia claims the underscore is used for 32 bit processes.

    1. ZachB Avatar

      Glad you found it useful! What you are saying agrees with what I have above – you need decorated names for 32-bit applications and un-decorated names for 64-bit applications. The application that is being debugged is the one that matters.

Leave a Reply

Your email address will not be published. Required fields are marked *

*

This site uses Akismet to reduce spam. Learn how your comment data is processed.