Zach Burlingame

HOWTO: Generate and Publish Doxygen Documentation in TeamCity

Written by

ZachB

in

,

I’ve started using Doxygen and JavaDoc style comments on my native C/C++ applications for documentation generation. In keeping with my goal to have everything “just work” on checkout with minimal dependencies (ideally just Visual Studio and version control) I wanted to get it integrated directly into the project. That way anyone can generate the latest version of the documentation from their working copy whenever they need it. Since I use TeamCity for my continuous integration server, it was natural to have the CI server generate and publish the latest documents during the build process.

Setup Doxygen

Setup Doxygen in a Working Copy of your Project

  1. Download the Windows Doxygen zip package
  2. Create build/doxygen/bin
  3. Place DoxyFile in build/doxygen/
  4. Create build.xml in build/doxygen/
    5. +---build
|   \---doxygen
|       \---bin
+---code
\---documentation

    build.xml

    <?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="3.5" DefaultTargets="Doxygen" 
xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <PropertyGroup>
    <OutputDirectory>..\..\documentation\doxygen\</OutputDirectory>
  </PropertyGroup>
  <Target Name="Doxygen">
    <MakeDir Directories="$(OutputDirectory)" />
    <Exec Command="bin\doxygen.exe" />
  </Target>
</Project>
  5. Test the script using MSBuild in a Visual Studio Command prompt

    6. [batch]
    cd /path/to/project/build/doxygen
    msbuild build.xml
    [/batch]

  6. Add the documentation/doxygen output folder to your VCS ignore pattern. (e.g. for Mercurial, add the following to the .hgignore file)
    7. glob:documentation/doxygen/*
  7. Add the all the files (including doxygen.exe, build.xml, and DoxyFile) to version control, commit the changes and publish them to the master repo (e.g. for Mercurial)

    8. [batch]
    hg add
    hg commit -m"Added Doxygen"
    hg push ssh://user@reposerver://srv/hg/project
    [/batch]

Setup Public Key Authentication

The following steps must be done on the BUILD SERVER. If you have multiple build servers/build agents for TeamCity, then you’ll need to duplicate most of these steps on each one. Alternatively, you can use a shared RSA key.

Generate an RSA public/private key pair

  1. Download and Install PuTTY.Look under “A Windows installer for everything except PuTTYtel”
  2. Open puttygen
  3. Click Generate
  4. Click Save Private Key
  5. Choose a location to save (I’ll use C:\keys for this example)
  6. Name the file (I’ll use buildserver.ppk for this example)
  7. Click Save Public Key
  8. Choose the same location used for the private key
  9. Name the file (I’ll use buildserver.pub for this example)

The following steps must be done on the WEB SERVER

Add an account for the buildserver

sudo useradd buildserver

Setup public key authentication for the user

  1. Setup the necessary files, directories, and permissions
    2. su buildserver
mkdir ~/.ssh
chmod 700 ~/.ssh
touch ~/.ssh/authorized_keys
chmod 600 ~/.ssh/authorized_keys
vim ~/.ssh/authorized_keys
  2. In PuTTYGen, copy the entire contents of the box labeled: “Public key for pasting into OpenSSH authorized_keys file:”
  3. Paste the contents into authorized_keys on the Web Server and save the file.

Setup Rsync

The following steps must be done on the BUILD SERVER.
Since Windows doesn’t natively have rsync, I use the cygwin packaged cwrsync.

  1. Download cwrsync http://www.itefix.no/i2/cwrsync

    2. I ran into problems with cwrsync being used in conjunction with plink where I received the following error:

    “Unable to read from standard input: The parameter is incorrect.”

    The problem apparently is when cygwin programs use redirected stdin/stdout handles. The solution I found to this was to use cygnative. From their website:

    Cygnative.exe is a wrapper for cygwin programs which call native Win32 programs with stdin and/or stdout handle redirected. Without cygnative.exe it is possible that the native program can not read from the handle and receives a “invalid handle” error.

  2. Download cygnative
  3. Create a script which will call cwrsync and pipe the rsync over an SSH connection from the build server to the web server.

    4. I placed this script during testing in project\build\doxygen\cwrsync.cmd I was only using it for testing before I put it into a TeamCity build step so I had no plans of commiting it to source control since it ultimately needs the private key which I don’t want in version control. If you aren’t going to use a TeamCity build step to publish the documentation, you can use this script as a starting point for your solution.

    [batch]
    @ECHO OFF
    REM *****************************************************************
    REM
    REM CWRSYNC.CMD – Batch file template to start your rsync command (s).
    REM
    REM By Tevfik K. (http://itefix.no)
    REM *****************************************************************
    REM Make environment variable changes local to this batch file
    SETLOCAL

    SET LOCAL_DIR=../../documentation/doxygen/html
    SET REMOTE_SERVER=yer_remote_machine_name
    SET REMOTE_USER=yer_remote_user_name
    SET REMOTE_DIR=/var/cache/doxygen/yer_project_name
    SET SSH_KEY=yer_ssh_key
    SET RSYNC_ARGS=-arz
    SET PLINK_CMD=cygnative plink -i %SSH_KEY% -batch
    SET REMOTE_CHMOD=chmod -R a+rx %REMOTE_DIR%

    REM Specify where to find rsync and related files (C:\CWRSYNC)
    SET CWRSYNCHOME=%PROGRAMFILES(x86)%\CWRSYNC

    REM *****************************************************************
    REM Don’t Change Below This Line
    REM *****************************************************************

    REM Set HOME variable to your windows home directory. That makes sure
    REM that ssh command creates known_hosts in a directory you have access.
    SET HOME=%HOMEDRIVE%%HOMEPATH%

    REM Make cwRsync home as a part of system PATH to find required DLLs
    SET CWOLDPATH=%PATH%
    SET PATH=%CWRSYNCHOME%\BIN;%PATH%

    REM Publish the files
    rsync %RSYNC_ARGS% -e "%PLINK_CMD%" "%LOCAL_DIR%" %REMOTE_USER%@%REMOTE_SERVER%:%REMOTE_DIR%

    REM Fix the permissions on the files
    %PLINK_CMD% %REMOTE_USER%@%REMOTE_SERVER% %REMOTE_CHMOD%
    [/batch]

    In a command prompt, cd to the directory that the cwrsync.cmd script is and run it

    cd /path/to/cwrsync/script/
cwrsync.cmd

    It should ‘just work’. If you get an error running the script or your Web Server isn’t serving up the content, try turning up the verbosity of plink and and rsync by adding -v like this:

    SET RSYNC_ARGS=-arzvvvv
SET PLINK_CMD=cygnative plink -v -i %SSH_KEY% -batch

Configure TeamCity

  1. Create a build step to generate the documentation using the build.xml file created earlier in your project’s build configuration.

    2. Runner type: MSBuild
    Build file path: build\doxygen\build.xml
    Working Directory:
    MSBuild version: Microsoft .NET Framework 4.0
    MSBuild ToolsVersion: 4.0
    Run platform: x86
    Targets: Doxygen
    Command line parameters:
    Reduce test failure feedback time:
    .NET Coverage tools:
    Report type:

    Click “Save”

  2. Create a build step to publish the documentation to the web server.

    3. Rather than use a CMD file in version control or pushing it out to all the build agents, I prefer to use a build step in the build configuration for the project in TeamCity. To use the script in the TeamCity build step, you have to use %% rather than % because TeamCity will treat the % as a TeamCity build property.

    Runner type: Command Line
    Working directory:
    Run: Custom Script
    Custom script: < the contents of your cwrsync.cmd from earlier, with every '%' replaced with '%%' >
    Report type:

    Click “Save”

  3. Run your build and verify that everything works!

References

Comments

6 responses to “HOWTO: Generate and Publish Doxygen Documentation in TeamCity”

  1. Elliott Herz Avatar
    Elliott Herz

    Dear Zach,

    I was going through your tutorial of using Doxygen with Teamcity and have a few questions. When creating the build.xml file, how do we incorporate our project that we want documented? Is this done through TeamCity somehow or do we have to specify where the project is or the project file (or solution). As of right now I have teamcity using msbuild to build the solution project from CVS and after a build is successful I wanted to incorporate Doxygen to auto-document all of the code. Also as a side note, I didn’t see the reason in checking in the build and documentation folders into cvs? Is that how the connection happens between Doxygen and the version controlled project?

    I guess my main question is in what step is doxygen actually doing the documentation and storing it into documentation\doxygen? As of right now everytime I run MSBuild on the build.xml it seems like it creates a stub in build\doxygen\html but doesn’t do anything with documentation\doxygen.

    Thank you for your time. If you could answer these questions I would be so greatful!

    Best,
    Elliott Herz

    Reply
    1. ZachB Avatar
      ZachB

      Elliott,

      Glad you’ve found it useful!

      You’ve got the basics of it correct. Let me try and answer your questions one by one.

      When creating the build.xml file, how do we incorporate our project that we want documented?
      I have a TeamCity build step in my project’s build configuration that uses the MSBuild runner on the build\doxygen\build.xml file. I place this build step second to last in my build configuration, after all the various version of the project are build (e.g. Debug, Release, Win32, x64) and after all the unit tests have been run. The last build step is the one that (optionally) publishes the results to a webserver.

      Also as a side note, I didn’t see the reason in checking in the build and documentation folders into cvs?
      The generated documentation I don’t version with my VCS. In fact, I explicitly ignore the folder that the generated documentation lands in (\documentation in my example). The build.xml, Doxyfile, and Doxygen binaries, however, I do explicitly want in version control. There are two reasons for this. The first is that it allows everyone to build the documentation simply by having the source tree (and a Windows machine). They don’t have to go install Doxygen to their host machine. The second is that when you have multiple projects you work on, sometimes you get into version headaches where one project is using version X.X of a build tool while another project is using Y.Y. If there are any incompatibilities between these two, then it makes it complicated to manage. By having the binaries self-contained within the project along with the configuration and build script, it simplifies my life of making it work consistently.

      I guess my main question is in what step is doxygen actually doing the documentation and storing it into documentation\doxygen? As of right now everytime I run MSBuild on the build.xml it seems like it creates a stub in build\doxygen\html but doesn’t do anything with documentation\doxygen.
      When the TeamCity build step mentioned above runs (build\doxygen\build.xml via MSBuild), build\doxygen\bin\doxygen.exe is called, with the MakeDir parameter specified as ..\..\documentation. This is when doxygen actually generates the documentation and stores it in to \documentation. Note that it’s putting it up two directories relative to build.xml, which in my example, is a folder named documentation in the root directory.

      I hope this clears things up a bit.

      Reply
      1. Elliott Herz Avatar
        Elliott Herz

        Zach,

        Thank you for your response, it helped clear up some, but I am still a bit confused.

        As of right now, in teamcity my first build step is running msbuild on the project file (sln) and then my second step would be using MSBuild to call doxygen. It is successful in building the entire build steps but the doxygen build step doesn’t seem to do any documentation (still builds tho). So my first question is how does msbuild know what code needs to be documented? There is no reference to the project code in the build\doxygen\build.xml file except for it being in the project file for you. Does teamcity create this link bewteen the msbuild doxygen and the project (because it is the second step of the build process) or does it work because your build fodler is in your project?

        Like I keep mentioning, everytime I use msbuild on the build\doxygen\build.xml file either in teamcity or through command line only a stub is created in build\doxygen\html and even though the ouput is linked up to the documentation folder, it seems like there isn’t any output (the documentation folder is empty and the documentation\doxygen folder is also empty; there isn’t even a documentation\doxygen\html file also). So my second question is whether you are able to use Msbuild to run the build\doxygen\build.xml file through command line and it still works to document your code?

        I just don’t understand where the link occurs from the msbuild with doxygen to the actually project code?

        Thanks again,
        Elliott Herz

        Reply
        1. ZachB Avatar
          ZachB

          Ahh, OK. I think the piece you are missing is the Doxyfile. The build.xml file only tells MSBuild how to call Doxygen.exe. Neither TeamCity nor MSBuild have any clue where the project code is or what Doxygen does. By convention, Doxygen.exe looks for a configuration file, named Doxyfile, in it’s current working directory. You can also pass the path to the Doxyfile via a command-line parameter. The Doxyfile configuration file is what tells Doxygen.exe what to do including where the code directories are (i.e. the INPUT parameter) and the file extensions to consider (i.e. the FILE_PATTERNS parameter).

          As to your second question, yes. I am able to change directories to where the build.xml script is and call MSBuild from the command line and it successfully creates the documentation.

          The link from MSBuild with doxygen to the actual project code occurs in the doxygen configuration file named Doxyfile that is in the same directory as the build.xml script.

          Reply
          1. Elliott Herz Avatar
            Elliott Herz

            Thank you so much! Finally got everythink hooked up and its running perfectly with teamcity. Great tutorial and the help was much appreciated.

            Now for jsdoc…lol

            Best,
            Elliott

          2. ZachB Avatar
            ZachB

            Excellent, glad you got it all working!

Leave a Reply

Your email address will not be published. Required fields are marked *

CAPTCHA

*

This site uses Akismet to reduce spam. Learn how your comment data is processed.

More posts