Building, Testing and Debugging Visual Studio C++ project in Visual Studio Code

Notice

Starting from 2021/08/20 (as the result of this PR) Visual Studio Code documentation has been updated to include information about how to run Visual Studio Code outside of Developers Command Prompt.However, if you are interested in a story of how this happened or you need to run and debug Visual Studio tests using vstest.console.exe right from Visual Studio Code then, I invite you to continue reading the post.

Introduction


I started to code in C/Win32 API in the university and I continue to do this almost every day. I am not a professional C/C++ developer nor Windows internals specialist, however I like to write in C. Why I am telling this? Because it won’t be a surprise, all my C/C++ projects were always written in Visual Studio with all it’s goodies like IntelliSense, projects support and extensions like Visual Assist.

However, in a last few years working with .NET Core I’ve found myself doing almost all my coding in Visual Studio Code and to be honest I get used to its simplicity and blazing speed. So for me it was kind of logical and full of sense step to move my C/Win32 API development from Visual Studio to Visual Studio Code, and having previously a slick transition experience in .NET, I was expecting somewhat similar but it turned out to be not that simple, especially if you don’t want to cut off Visual Studio completely.

In this post I would like to share how you can configure Visual Studio Code to build, test and debug Visual Studio C++ project.

While everything described in this post was focused on local environment it is good to know that all of these instructions are also valid in Remote-SSH scenarios.

Author’s note

Understanding the problem


Visual Studio Code documentation for C/C++ development with Microsoft Visual C++ compiler have two requires:

  • … to run Visual Studio Code from within Developer Command Prompt for Visual Studio;
  • … to build apps using cl.exe directly.

#1, at first, doesn’t seem to be a huge deal but if you plan to go with Remote-SSH development scenario (and for me it was a primary case) it becomes a real obstacle because you are no longer of direct control of how you launch Visual Studio Code.

#2 isn’t what we need, because using cl.exe instead of msbuild.exe means replication everything and, in future, loosing an ability to build the project using Visual Studio.

Besides mentioned above, there is one more thing (#3) – there is no documentation on how to run and debug unit-tests written in Microsoft Unit Test Framework for C++, which, is problematic if you didn’t use third-party unit-test frameworks.

For me, the last one was of particular importance because I didn’t want to rewrite all my unit-tests, so I came up with a plan to solve all three of them.

Step #1: Configuring “task shell”


As I mentioned previously, to ensure we have all necessary environment variables set, to develop with Microsoft Visual C++ compiler, we have to start Visual Studio Code from within Developer Command Prompt for Visual Studio. However, if after going through the documentation, you ask yourself – what we need these environment variables for, you will find out that we don’t actually need them for Visual Studio Code itself but instead we need them for running cl.exe i.e., we need them for running tasks.

Visual Studio Code supports a way to customise how tasks are run by allowing us to configure a “shell” used to run them.

Here is how it is done (in tasks.json):

{
  "windows": {
    "options": {
      "shell": {
        "executable": "",
        "args": []
      },
      "cwd": ""
    }
  }
}

You can find a detailed description of these properties in official tasks.json documentation however in a favour of brevity I will slightly cover them here without going into much details:

  1. executable property contains a path to an executable to run as shell when a task is triggered.
  2. args property contains an array of argument to pass to the executable. These arguments are expected to configure the shell, so it will be ready to accept arguments from the task, which also will include an “executable” and “args”.
  3. cwd property contains a path to a directory to set as shell’s current directory.

To understand what we should put into these properties we need to understand what Developer Command Prompt for Visual Studio is. The simplest way is to look at the properties. On my machine properties of Developer Command Prompt for Visual Studio are:

Picture 1. Properties of Developer Command Prompt for Visual Studio.

It turns out Developers Command Prompt for Visual Studio is a shortcut to cmd.exe which runs a VsDevCmd.bat file, which, in turn, does environment setup. So, in general if we can configure the shell in the same way, then everything should start working without a need to start Visual Studio Code from Developers command prompt for Visual Studio.

Let’s сopy “Start in” shortcut property into cwd, set executable to cmd.exe, split command line from “Target” shortcut property into args and then add an extra && argument to the end (I will explain this one in a moment).

Here is how our shell configuration should looks like:

{
  "windows": {
    "options": {
      "shell": {
        "executable": "cmd.exe",
        "args": [
          "/C",
          "VsDevCmd.bat",
          "&&"
        ]
      },
      "cwd": "C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\Community\\Common7\\Tools\\"
    }
  }
}

Please note, depending on the installation, location to “Microsoft Visual Studio” can vary.

Author’s note

Now, let’s see what our configuration does:

  • executable is what Visual Studio Code should run.
  • args is what Visual Studio Code should passes to executable.
  • cwd is where Visual Studio Code should run executable i.e., set process current directory to.

All of the args passed to cmd.exe have a following meaning:

  • /C – instructs cmd.exe to run a command and terminate.
  • VsDevCmd.bat – is interpreted as the command to run.
  • && – instructs cmd.exe to interpret following arguments as a new command that should we executed if current command (VsDevCmd.bat) was executed successfully. This is required because when you run a task, Visual Studio Code concatenates shell command with task command i.e., cmd.exe /C VsDevCmd.bat && msbuild Solution.sln. Without &&, cmd.exe will carry out execution of VsDevCmd.bat with msbuild Solution.sln as parameters.

You can read more about cmd.exe syntax here.

Author’s note

Now when we have a shell, we need a task to run. Let’s start from the build one.

Step #2: Configuring “build task”


This and later steps assume you are running Visual Studio Code from within Development Command Prompt for Visual Studio or have configured shell as described in a “Step 1: Shell”.

Author’s note

When building Visual C++ project (.vcxproj) in Visual Studio, it is build using msbuild.exe because all information required to build the project is within .vcxproj file itself. Visual Studio, on the other hand, administers the process (ex. configures loggers to integrate with errors window) and provides input arguments (ex. Debug or Release configuration).

If you have multiple project united into a solution (.sln), the principle schema remains the same because for quite a long time .sln is a native format for msbuild.exe.

Author’s note

For us, the above information mean we need to use msbuild.exe not cl.exe to build a solution. Here is how it looks in tasks.json:

"tasks": [
  {
    "type": "shell",
    "label": "build",
    "command": "msbuild.exe",
    "args": [
      "${workspaceFolder}/Solution.sln",
      "/p:Configuration=${input:build-configuration}",
    ],
    "problemMatcher": [
      "$msCompile"
    ],
    "group": "build"
  }
],
"inputs": [
  {
    "type": "pickString",
    "id": "build-configuration",
    "description": "Select build configuration",
    "options": [
      "Debug",
      "Release",
    ],
    "default": "Debug"
  }
]

When invoked, the task automatically prompts for a configuration to build (Debug or Release) and invokes msbuild.exe with /p:Configuration property set to selected configuration. Task is straightforward and if invoked, successfully builds all project from Solution.sln.

You can find more about tasks.json and inputs in particular in official Visual Studio Code documentation.

Author’s note

The task can be customised future to support more build properties or user input but for the purpose of establishing the platform it is more than enough.

Let’s continue and configure tests.

Step #3: Configuring “test task”


Visual Studio comes with a support for writing C++ unit-test with help Microsoft Unit Testing Framework for C++. However, in contrary to Visual Studio, Visual Studio Code has no support for Microsoft Unit Testing Framework for C++.

There is a Test Explorer UI extension which has a rich ecosystem of Test Adapters for many C/C++ testing frameworks. However, currently there is no Test Adapter for Microsoft Unit Testing Framework for C++.

Author’s note

Without any kind of Test Adapter the only option we have is to run tests manually. Tests written in Microsoft Unit Test Framework for C++ can be executed by Visual Studio Test Platform using vstest.console.exe application.

The application has simple and well documented command line interface, which in minimum requires a path to a .dll with tests.

Here is an example of tasks.json:

Please note, for brevity it doesn’t include configuration from previous steps.

Author’s note
"tasks": [
  {
    "type": "shell",
    "label": "tests",
    "command": "vstest.console",
    "args": [
      "${workspaceFolder}/${input:test-configuration}/Solution.Test.dll"
    ],
    "problemMatcher": [
      "$msCompile"
    ],
    "group": "test"
  }
],
"inputs": [
  {
    "type": "pickString",
    "id": "test-configuration",
    "description": "Select test configuration",
    "options": [
      "Debug",
      "Release",
    ],
    "default": "Debug"
  }
]

When invoked, the task automatically prompts for a configuration to test (Debug or Release) and then launches vstest.console.exe, passing a path to *.Test.dll.

The task can be extended to provide more arguments to vstest.console.exe i.e., a filter to run a particular set of tests. However, for the purpose of establishing the platform it is more than enough.

Let’s move forward.

Step #4: Configuring “debug a test”


Having “build” and “test” tasks alongside with Visual Studio Code C++ Extension, which provides rich IntelliSense and C/C++ debugging support, might be enough to switch from Visual Studio to Visual Studio Code.

However, for me personally, and I think, for many projects there is one but critical thing missed – an ability to debug a test. It might sound strange because just a few word before I mentioned C/C++ debugging support provided by C++ extension but don’t be fooled by words – debugging a test is a bit different from debugging our application because it is about debugging a separate application, usually, something like test runner or test host. In our case it will be something from Visual Studio Test Platform.

Let’s start from setting up requirements:

  1. We need an ability to hit a breakpoint inside a test and to do that we need to attach a debugger to a correct process.
  2. We need to ensure, tests aren’t started before we attached a debugger.

Now, when we have requirements it is time to understand what to debug.

According to the official Visual Studio Test Platform documentation all tests are loaded into a specific, managed process called “Test Execution Host” which is implemented in testhost.x86.exe or testhost.exe.

Here is a related quotes from the documentation with more details:

This architecture has four major components:
1. Test Runner is the command line entrypoint to test platform (vstest.console).
2. Test Execution Host is an architecture and framework specific process that actually loads the test container and executes tests.
3. Data Collector Host process hosts various test execution data listeners.
4. IDE/Editor process is used by developer for Edit/Build the application and trigger test runs.

A test host is specific to the language runtime, version and platform architecture of the test containers. E.g. testhost.x86.exe is the test host for a test container (assembly) targeting .NET Framework 4.6, x86 architecture; where as testhost.exe is spawned for .NET Framework 4.6, x64 architecture. The test host for .NET Core is platform agnostic and is launched on the required dotnet.exe (x86 or x64).

However the test host is agnostic of test framework. It has no knowledge of what a Test Case or Test Suite is. It delegates the responsibility of discovering and executing the tests to Test Adapters. An adapter understands the intricacies of test framework. E.g. the NUnit adapter understands tests authored in NUnit framework, it can discover and execute them.

VSTest architectural documentation

The quote above also reveals one more important piece of information about the framework. The vstest.console.exe we use to run tests is a Test Runner which is responsible for spawning Test Execution Host.

Now when we know where to attach debugger we have to make sure, Test Execution Host won’t execute any of test code until we attach the debugger, otherwise our debugging will look more like “draw” that software development.

Luckily, the official documentation also has information about how to debug platform components:

For test host process (testhost.exe, or dotnet exec testhost.dll depending on target framework) use the following (VSTEST_HOST_DEBUG=1, note Author) environment variable. Execution will halt until debugger is attached.

VSTest diagnostics documentation

So, all we need is a separate “debug-test” task, which will set VSTEST_HOST_DEBUG environment variable as well as starting test execution:

"tasks": [
  {
    "type": "shell",
    "label": "debug-test",
    "command": "vstest.console",
    "args": [
      "${workspaceFolder}/${input:test-configuration}/Solution.Test.dll"
    ],
    "options": {
      "env": {
        "VSTEST_HOST_DEBUG": "1"
      }
    },
    "problemMatcher": [
      "$msCompile"
    ],
  },
"inputs": [
  {
    "type": "pickString",
    "id": "test-configuration",
    "description": "Select test configuration",
    "options": [
      "Debug",
      "Release",
    ],
    "default": "Debug"
  }
]

We also need a default (provided by C++ extension) launch configuration to attach to process:

"configurations": [
  {
    "name": "Attach C++",
    "type": "cppvsdbg",
    "request": "attach",
    "processId": "${command:pickProcess}"
  }
]

Having both of them in place it is time to test the flow.

When started the task prints the following output:

Microsoft (R) Test Execution Command Line Tool Version 16.9.1
Copyright (c) Microsoft Corporation. All rights reserved.

Starting test execution, please wait…
A total of 1 test files matched the specified pattern.
Host debugging is enabled. Please attach debugger to testhost process to continue.
Process Id: 4800, Name: testhost

Here we can see multiple things:

  1. Environment variable worked and testhost.exe is paused.
  2. vstest.console.exe automatically prints process id and name, so we know exactly to what process to attach.

Attaching a debugger to testhost/4800 leads us to … nowhere. Because nothing happens. The process remains paused.

To understand why it is happening it is important to remind ourself that vstest.console.exe and testhost.exe are in fact .NET applications and we attached native C++ debugger to .NET application. If you are a long time Visual Studio user as I am, this might be a bit frustrating because there were never such a problem. The thing here is that Visual Studio supports, what is called a mixed-mode debugging (an ability to load multiple debuggers into one process) but Visual Studio Code doesn’t.

You may wonder why not attach two debuggers manually: one for managed code (to resume the process) and one for native code (to hit the breakpoint)? Unfortunately, this isn’t possible – Windows doesn’t allow multiple invasive debuggers to be attached to the same process.

So, at this moment we end up being able to pause the process of interest but being unable to resume it and hit a breakpoint.

However, this isn’t the end of the story.

When I came up with such an unfortunate results I was close to drop the whole idea. But then, I caught myself spinning the same though again and again:

All I need is to pause the tests until I attach native debugger.

Author’s thoughs

Then I realised, there is already a solution – I can reuse the same approach as Visual Studio Test Platform team implemented for debugging platform components – pass an environment variable and wait for debugger using a code in tests .dll itself!

Here is a code (I put it into stdafx.h):

/* This is done to ensure a possibility of debugging vstest.console
 *    tests from Visual Studio Code.
 * 
 * When VSCODE_WAIT_FOR_DEBUGGER_TO_ATTACH is set to any value 
 * the code will enter into infinite loop until debugger
 * is attached. 
 *
 */
TEST_MODULE_INITIALIZE(MODULE_INITIALIZE)
{
  CONST DWORD SleepTime = 10;
  if (GetEnvironmentVariable(
        L"VSCODE_WAIT_FOR_DEBUGGER_TO_ATTACH", NULL, 0) != 0)
  {
    DWORD PID = GetCurrentProcessId();

    LPWSTR Message = GetFormattedMessage(
      L"Visual Studio Code native debugging is activated. \
        Waiting for debugger to connect... (PID: %1!d!)", 
      PID);

    Logger::WriteMessage(Message);

    DWORD Elapsed = 0, Timeout = 30 * 1000;
    while (!IsDebuggerPresent()) 
    {
      Sleep(SleepTime);

      if (Elapsed > Timeout)
      {
        Assert::Fail(
          L"ERRO: Visual Studio Code native debugger \
            didn't connect in time, failing.");
      }
      Elapsed += SleepTime;
    }
  }
}

Here is what it does:

  1. According to the official documentation, TEST_MODULE_INITIALIZE macro defines the method that runs when a module is loaded. There can be one TEST_MODULE_INITIALIZE method per test module.
  2. Inside this module we verify if VSCODE_WAIT_FOR_DEBUGGER_TO_ATTACH environment variable is defined. If it is, we write a message (with PID of testhost.exe) to let user know we have detected the intent and waiting for debugger to attach.
  3. We wait for a debugger in a loop. We check if process is being debugged by using IsDebuggerPresent API.

By default I have implemented a 30 seconds timeout and if it no debugger is attached in 30 seconds I use Assert.Fail to fail test module execution.

Now, if we modify “debug-test” task to use our environment variable:

...
"options": {
  "env": {
    "VSCODE_WAIT_FOR_DEBUGGER_TO_ATTACH": "1"
  }
},
...

… and run the task, we should receive the following output:

Microsoft (R) Test Execution Command Line Tool Version 16.9.1
Copyright (c) Microsoft Corporation.  All rights reserved.

Starting test execution, please wait...
A total of 1 test files matched the specified pattern.
Visual Studio Code native debugging is activated. Waiting for debugger to connect... (PID: 2656)

Now, if we attach to the process, the process resumes and hits a breakpoint!

Bonus


While everything described in the post allows us to build, test and debug a C/C++ application written in Visual Studio the experience isn’t the same, especially in “debugging a test” part:

  • We have to manually launch a specific task.
  • We have to manually start debugging and attach to a process, which number we have to find in task output.
  • Current configuration runs all tests in a test project, which is awful.

These three make debugging experience annoying. So, let’s do something about it.

Bonus #1: Running a single test


We start from a critical one – run exactly one test instead of running the whole test suite. vstest.console.exe supports test filters by accepting --TestCaseFilter parameter, so all we need to run a single test is to pass it’s name as a filter.

To make this process dynamic (because I don’t think editing tasks.json each time we need to debug a test is a good experience) we can use Visual Studio Code inputs functionality and define a promptString input:

"tasks": [
  {
    "type": "shell",
    "label": "debug-test",
    "command": "vstest.console",
    "args": [
      "${workspaceFolder}/${input:test-configuration}/Solution.Test.dll",
      "--TestCaseFilter:${input:test-selector}"
    ],
    "options": {
      "env": {
        "VSCODE_WAIT_FOR_DEBUGGER_TO_ATTACH": "1"
      }
    },
    "problemMatcher": [
      "$msCompile"
    ],
  },
"inputs": [
  {
    "type": "pickString",
    "id": "test-configuration",
    "description": "Select test configuration",
    "options": [
      "Debug",
      "Release",
    ],
    "default": "Debug"
  },
  {
    "type": "promptString",
    "id": "test-selector",
    "description": "Write test to debug"
  }
]

With this update when we start “debug-task” we get a friendly prompt for a test to debug:

Picture 2. Prompt for a test to debug.

This is nice but we can do more.

Bonus #2: Automatically run “debug-test” when launching a debug configuration


Next, logical step is to link debug configuration and “debug-test” task. Before going jumping into it is important to know the limitations, which is in our case is an inability to automatically pass name of the test name to “debug-test” task and automatically selecting corresponding testhost.exe process based on output from the task.

So here, we will focus on automatically running “debug-test” when launching debug configuration. The problem here comes from the fact that “debug-test” task doesn’t complete until debugger is attached, so using preLaunchTask property of debug configuration won’t work without modifying a task first:

{
  "name": "Attach C++ XX",
  "type": "cppvsdbg",
  "request": "attach",
  "processId": "${command:pickProcess}",
  "preLaunchTask": "debug-test"
}

But, thanks to Visual Studio Code developers there is a mechanism to notify Visual Studio Code about task completion without waiting for task process to exit – background task.

The background task allows us to start a process and by analysing its output notify Visual Studio Code about successful initialisation or errors, so it can continue to execute other tasks or abort the execution. All output from the background task is processed in the background by Visual Studio Code using a problemMatcher configuration:

"tasks": [
  {
    "type": "shell",
    "label": "debug-test",
    "command": "vstest.console",
    "args": [
      "${workspaceFolder}/${input:test-configuration}/Solution.Test.dll",
      "--TestCaseFilter:${input:test-selector}"
    ],
    "options": {
      "env": {
        "VSCODE_WAIT_FOR_DEBUGGER_TO_ATTACH": "1"
      }
    },
    "isBackground": true,
    "problemMatcher": {
      "owner": "external",
      "fileLocation": "relative",
      "pattern": {
        "regexp": "^ERRO:(.+)$",
        "message": 1
      },
      "background": {
        "activeOnStart": true,
        "beginsPattern": "Starting test execution, please wait...",
        "endsPattern": "Visual Studio Code native debugging is activated. Waiting for debugger to connect..."
      }
    }
  }
]

Here is what we added to “debug-test” task:

  1. isBackground – a property indicating that this task is a background task.
  2. problemMatcher.pattern – a regex pattern to detect errors. If you take a look at the code we introduced into TEST_MODULE_INITIALIZE method you can notice "ERRO:" prefix in error messages.
  3. background – a most interesting section. Here we configure whether our task is considered started from start (controlled by activeOnStart property and confirmed by beginsPattern) and considered completed by matching output with endsPattern which is set to match the message our implementation prints when waiting for native debugger to attach.

These modifications allow us to start the “debug-test” task and when everything is in place continue with debug configuration. However, if you start a debug you won’t see a task running in the background instead of this you will see a prompt for process id to attach.

Do it mean, we modified the task for nothing?

The answer is no. The problem here is not a task but a way how parameters are resolved. When we start debug configuration, as a first step before even starting it, Visual Studio Code attempts to resolve ${command:pickProcess} parameter. That is why, we see a prompt for process id before task is ever started. Fortunately, we can workaround this by using one more feature of Visual Studio Code – compounds.

The idea of compounds is to provide a way to start multiple debug configurations in parallel i.e., to debug multiple targets. However, in our case it is more interested from a side that compound are allowed to own preLaunchTask which runs before any of the included debug configuration:

"configurations": [
  {
    "name": "Attach C++",
    "type": "cppvsdbg",
    "request": "attach",
    "processId": "${command:pickProcess}",
    "preLaunchTask": "debug-test"
  }
],
"compounds": [
  {
    "name": "Debug Tests",
    "configurations": [
      "Attach C++"
    ],
    "preLaunchTask": "debug-test"
  }
]

Now, if we start “Debug Test” debug configuration everything will work as expected – “debug-test” task will be started first and then a prompt for test name and then a prompt for process to attach.

You can hide “Attach C++” debug configuration from the menu by playing with presentation property of debug configuration and compounds.

Author’s note

Bonus #3: Removing compiler errors


If you did all of the above steps, you might notice a bunch of IntelliSense errors in Visual Studio Code problems windows. Most of them are probably about unresolved types. This happening because by default Visual Studio Code IntelliSense can’t correctly pickup definitions for Visual Studio headers. The fix is straightforward – include them in c_cpp_properties.json configuration:

{
  "configurations": [
    {
      "name": "Win32",
      "includePath": [
        "${workspaceFolder}/**",
        "C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Auxiliary/VS/UnitTest/include",
        "C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Auxiliary/VS/include"
      ],
      "defines": [
        ...
      ],
      "compilerPath": "C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.28.29910/bin/Hostx64/x64/cl.exe",
      "cStandard": "c17",
      "cppStandard": "c++17",
      "intelliSenseMode": "windows-msvc-x64"
    }
  ],
  "version": 4
}

You also might want to include a full compiler path in compilerPath property. However, in most cases it is filled automatically when c_cpp_configuration.json file is generated.

Conclusion


It was an interesting journey and I hope this post helped you to configure your environment.

Let’s do a quick summary.

In this post we configured all major aspects of what to do to build, test and debug Visual Studio C++ project in Visual Studio Code without loosing Visual Studio compatibility. Besides that, in Step #1 we configured a task shell to ensure Developer Command Prompt for Visual Studio is started automatically. This quality of life improvement has a critical role in Remote Development scenarios because that is how I started this journey – I was trying to work on a Visual Studio C++ project on Windows PC using Visual Studio Code on my Mac.

History


  • 2021/08/20 – Introduced a warning banner at the top of the post saying that current version of Visual Studio Code documentation contains instructions on how to configure automated start of Developer Command Prompt as described in Step #1.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s