xUnit

Enabling source code debugging for your NuGet packages with GitLink

September 23, 2015 Coding 3 comments , , , ,

Enabling source code debugging for your NuGet packages with GitLink

Recently on Twitter, someone was complaining that their CI builds were failing due to SymbolSource.org either being down or rejecting their packages. Fortunately, there’s a better way than using SymbolSource if you’re using a public Git repo (like GitHub) to host your project — GitLink.

Symbols, SymbolSource and NuGet

Hopefully by now, most of you know that you need to create symbols (PDB’s) for your release libraries in addition to your debug builds. Having symbols helps your users troubleshoot issues that may crop up when they’re using your library. Without symbols, you need to rely on hacks, like using dotPeek as a Symbol Server. It’s a hack because the generated source code usually doesn’t match the original, and it certainly doesn’t include any helpful comments (you do comment your code, right?)

So you’ve updated your project build properties to create symbols for release, now you need someplace to put them so your users can get them. Up until recently, the easiest way has been to publish them on SymbolSource. You’d include the pdb files in your NuGet NuSpec, and then run nuget pack MyLibrary.nuspec -symbols. NuGet then creates two packages, one with your library and one just with the symbols. If you then run nuget push MyLibrary.1.0.0.nupkg, if there’s also a symbols package alongside, NuGet will push that to SymbolSource instead of NuGet.org. If you’re lucky, things will just work. However, sometimes SymbolSource doesn’t like your PDB’s and your push will fail.

The issues

While SymbolSource is a great tool, there are some shortcomings.
* It requires manual configuration by the library consumer
* They have to know to go to VS and add the SymbolSource URL to the symbol search path
* It slows down your debugging experience. VS will by default check every configured Symbol Server for matching PDB’s. That leads many people to either disable symbol loading entirely or selectively load symbols. Even if you selectively load symbols, the load is still slow as VS has know way to know which Symbol Server a PDB might be on and must check all of them.
* Doesn’t enable Source Code debugging. PDB’s can be indexed to map original source code file metadata into them (the file location, not contents). If you’ve source-indexed your PDB’s and the user has source server support enabled, VS will automatically download the matching source code. This is great for OSS projects with their code on GitHub.

GitLink to the Rescue

GitLink provides us an elegant solution. When GitLink is run after your build step, it detects the current commit (assuming the sln is in a git repo clone), detects the provider (BitBucket and GitHub are currently supported) and indexes the PDB’s to point to the exact source location online. Of course, there are options to specify commits, remote repo location URLs, etc if you need to override the defaults.

After running GitLink, just include the PDB files in your nuspec/main nupkg alongside your dll files and you’re done. Upload that whole package to NuGet (and don’t use the -symbols parameter with nuget pack). This also means that users don’t need to configure a symbol server as the source-indexed PDB’s will be alongside the dll — the location VS will auto-load them from.

An example

Over at xUnit and xUnit for Devices, we’ve implemented GitLink as part of our builds. xUnit builds are setup to run msbuild on an “outer” .msbuild project with high-level tasks; we have a GitLink task that runs after our main build task.

As we want the build to be fully automated and not rely on exe’s external to the project, we “install” the GitLink NuGet package on build if necessary.

Here’s the gist of our main CI target that we call on build msbuild xunit.msbuild /t:CI (abbreviated for clarity):

<PropertyGroup>
  <SolutionName Condition="'$(SolutionName)' == ''">xunit.vs2015.sln</SolutionName>
  <SolutionDir Condition="'$(SolutionDir)' == '' Or '$(SolutionDir)' == '*Undefined*'">$(MSBuildProjectDirectory)</SolutionDir>
  <NuGetExePath Condition="'$(NuGetExePath)' == ''">$(SolutionDir)\.nuget\nuget.exe</NuGetExePath>
</PropertyGroup>

<Target Name="CI" DependsOnTargets="Clean;PackageRestore;GitLink;Build;Packages" />

<Target Name="PackageRestore" DependsOnTargets="_DownloadNuGet">
  <Message Text="Restoring NuGet packages..." Importance="High" />
  <Exec Command="&quot;$(NuGetExePath)&quot; install gitlink -SolutionDir &quot;$(SolutionDir)&quot; -Verbosity quiet -ExcludeVersion -pre" Condition="!Exists('$(SolutionDir)\packages\gitlink\')" />
  <Exec Command="&quot;$(NuGetExePath)&quot; restore &quot;$(SolutionDir)\$(SolutionName)&quot; -NonInteractive -Source @(PackageSource) -Verbosity quiet" />
</Target>

<Target Name='GitLink'>
  <Exec Command='packages\gitlink\lib\net45\GitLink.exe $(MSBuildThisFileDirectory) -f $(SolutionName) -u https://github.com/xunit/xunit' IgnoreExitCode='true' />
</Target>

<Target Name='Packages'>
  <Exec Command='"$(NuGetExePath)" pack %(NuspecFiles.Identity) -NoPackageAnalysis -NonInteractive -Verbosity quiet' />
</Target>

There are a few things to note from the snippet:
* When installing GitLink, I use the -ExcludeVersion switch. This is so it’s easier to call later in the script w/o remembering to update a target path each time.
* I’m currently using -pre as well. There’s a number of bugs fixed since the last stable release.

The end result

If you use xUnit 2.0+ or xUnit for Devices and have source server support enabled in your VS debug settings, VS will let you step into xUnit code seamlessly.

If you do this for your library, your users will thank you.

xUnit for Xamarin is dead, long live xUnit for Devices!

March 16, 2015 Coding 1 comment , ,

xUnit for Xamarin is dead, long live xUnit for Devices!

In conjunction with today’s release of xUnit 2.0 RTM, I’m happy to announce the initial release of xUnit for Devices (GitHub | NuGet). This has been a long time coming and I’d like to thank Brad Wilson and James Newkirk for their tremendous efforts over the years.

Project rename

xUnit for Devices started out as xUnit for Xamarin. Over the course of development however, it became apparent that what we really have is an MVVM-based test runner where the view was only an implementation detail. Current support is limited to platforms Xamarin Forms supports, but in the future it’s pretty easy to add a desktop/WPF view and support any additional GUI platform as needed.

Upgrading your projects to RTM

If you’ve been using xUnit for Xamarin, the easiest way to update is by using NuGet. There’s a final xunit.runner.xamarin package that pulls in the new xunit.runner.devices package. After upgrading, you can simply remove the old package and keep the dependencies. Windows Phone 8 users will need to make one additional change in the MainPage.xaml to update the assemly name from xunit.runner.xamarin to xunit.runner.devices.

Getting Started

The RTM 1.0 release is available on NuGet and on GitHub.

For iOS and Android, create a new blank Xamarin app project to host the unit test runner. Make sure to give any capabilities/permissions you need in the appropriate manifest.

For WP8, create a new Unit Test Project and then remove the MSTestFramework reference.

Then for all platforms, install/update the xunit.runner.devices package via the GUI or Package Manager Console:
Install-Package xunit.runner.devices

Then look for the .cs.txt, xaml.txt files that are the templates for your platform and copy/paste the contents into the app. Specifically,
– iOS: replace the contents of AppDelegate.cs with AppDelegate.cs.txt
– Android: replace the contents of MainActivity.cs with MainActivity.cs.txt
– WP8: replace the contents of MainPage.xaml.cs with MainPage.xaml.cs.txt and MainPage.xaml with MainPage.xaml.txt

xUnit Device Runners 1.0 RC3

February 23, 2015 Coding 1 comment , ,

Following the release of xUnit 2.0 RC3, the Xamarin Device Runners have been updated to work with RC3.

One note for Android users: due to a dependence on Xamarin Forms, your runner app project needs to use API level 21 as its target and SDK. You can target down to API level 15 if you wish. You can also reference other MonoAndroid or Portable Class Libraries if you want to keep your unit tests at a different API level. You also might need to specify a default theme on some devices to workaround a different Xamarin Forms bug. Please see the updated MainActivity.cs.txt for the specifics.

Wait, what happened to RC2?

If you blinked, you missed it. RC2 of the Device Runners came out Saturday. With xUnit RC3 being a quick update from RC2, it’s best to skip to the latest.

As always, if you run into any issues, feel free to reach out to @onovotny on Twitter or post an issue on GitHub.

Getting Started

RC3 is available on NuGet and on GitHub.

For iOS and Android, create a new blank Xamarin app project to host the unit test runner. Make sure to give any capabilities/permissions you need in the appropriate manifest.

For WP8, create a new Unit Test Project and then remove the MSTestFramework reference.

Then for all platforms, install/update the xUnit.Runner.Xamarin package via the GUI or Package Manager Console:
Install-Package xunit.runner.xamarin -Pre

Then look for the .cs.txt, xaml.txt files that are the templates for your platform and copy/paste the contents into the app. Specifically,
– iOS: replace the contents of AppDelegate.cs with AppDelegate.cs.txt
– Android: replace the contents of MainActivity.cs with MainActivity.cs.txt
– WP8: replace the contents of MainPage.xaml.cs with MainPage.xaml.cs.txt and MainPage.xaml with MainPage.xaml.txt

Announcing: xUnit Device Runner RC1

January 31, 2015 Coding 1 comment , , , ,

xUnit Device Runner 1.0 RC1

I’m pleased to announce the release of the xUnit Device Runners Release Candidate 1. This release adds support for the Xamarin.iOS Unified profile, required for all new iOS applications now and updates starting in July.

Other notable enhancements include a filter for searching test cases by name and status (pass/fail/not run).

To get started, please see the following posts:

If you run into any issues, please file a report in the issue tracker.

Now available: xUnit for Windows Phone 8 Silverlight

November 21, 2014 Coding 1 comment ,

With the release of xUnit 2 beta 5, Windows Phone 8 Siverlight is now supported; here’s how to get started.

Side note: If you’ve previously had the xUnit Extension VSIX installed, you need to remove it for beta 5. You no longer need it. See the release notes for details.

Unfortunatey, due to limitations in the Visual Studio Test Explorer’s architecture, we can’t yet integrate into the Test Explorer window like we can for Universal or Dekstop apps. Instead, we have to run the unit tests as an app on the device, exactly like we do for Xamarin. In fact, with Xamarin Forms providing the UI, we’re able to bring the same runner support to WP8.

Steps to create a Windows Phone 8 Silverlight test project

  1. Read the post on getting started with Xamarin. It’s almost identical.
  2. Use the latest version of the runner, 0.99.5-beta5 at the time of this writing.
  3. When creating the WP8 App, use the Windows Phone 8 Silverlight (Blank) template.
  4. Replace the contents of MainPage.xaml.cs with the contents of MainPage.xaml.cs.txt added to your project

To run tests, simply run the app either in the debugger or deploy and run.

Known issues

The Xamarin Forms-based runner is bare bones. It needs a lot of work to add features. But it does work to execute xUnit tests and run them in a debugger to figure out why stuff’s not working 🙂

How you can help

Pull requests are very much welcome over at the project site. If you need help getting up-and-running, just ping me on Twitter.

Now Shipping: xUnit support for Windows 8 and Windows Phone App 8.1

August 11, 2014 Coding 2 comments , ,

After years of unofficial hacks to get xUnit working for Windows Store apps, I’m happy to announce that xUnit v2 beta 4 ships with official support. As an added bonus, Windows Phone App 8.1 support is included as well.

To use xUnit for Store, you need the latest xUnit.net runner for Visual Studio installed.

Steps to create a Windows 8 Store Unit Test for xUnit:

  1. Install the runner
  2. Create a new Windows Store Unit Test project
  3. In the project references, remove the MSTestFramework reference and delete the UnitTest1.cs sample test
  4. Use NuGet to Install-Package xunit -Pre and install at least version 2.0.0-beta4-build2738
  5. Create your tests
  6. When you compile, you’ll see the tests in the Test Explorer window (make sure to show that window if it’s not visible)

Steps to create a Windows Phone App Unit Test for xUnit:

  1. Install the runner
  2. Create a new Windows Phone App Unit Test project (not Windows Phone 8.1 Silverlight)
  3. In the project references, remove the MSTestFramework reference and delete the UnitTest1.cs sample test
  4. Use NuGet to Install-Package xunit -Pre and install at least version 2.0.0-beta4-build2738
  5. Create your tests
  6. When you compile, you’ll see the tests in the Test Explorer window (make sure to show that window if it’s not visible)

To Do

  • Windows Phone 8 support – technical issues to overcome
  • Project Templates

Limitations

Right now, only Any CPU/x86 is supported by the VS Runner due to limitations in the VS Runner’s extensibility model. We plan on shipping a device runner, similar to the Xamarin Runner that will enable on-device ARM testing.

I’d like to thank @bradwilson and @jamesnewkirk for their patience and persistence with merging these large pull requests and keeping the release bar so high.

Getting Started with xUnit for Xamarin

July 10, 2014 Coding 3 comments , , ,

xUnit.net 2.0 supports both Portable Class Library (PCL) and platform specific projects for iOS and Android.
Unit tests for Xamarin have two main components, which may reside in the same assembly. This post will show you how to get started.

Requirements

  • Xamarin Studio 5.0
  • Visual Studio 2013 Update 2 with the latest Xamarin for Visual Studio

Architecture

Xamarin support consists of two logical components, which may be in the same assembly or may be split:
1. Assemblies containing tests. This may be either a Xamarin project or a PCL. This assembly contains your test classes.
2. App for running tests on a device or simulator. This bootstraps the tests. It’s important that you set any permissions, such as internet access, geo locations, notifications, etc, if your app requires it.

The Goods

For simplicity here, we’ll put both pieces in the same project, but you can also put your tests in a separate library and reference it in your runner app. We’ll use Android as an example, but it’s exactly the same for iOS.

  1. Create a new blank Android project:

  2. Add the NuGet references for xUnit and xunit.runner.xamarin. Make sure the Prerelease option is enabled as xUnit 2.0 is still in beta.


  3. After installing the package, you’ll see a file MainActivity.cs.txt (on Android or AppDelegate.cs.txt for iOS). Copy/paste the contents of that file into your real MainActivity.cs file. Congratulations, your runner app is now ready, you just need to add some tests!

  4. Create a new test class and start creating your tests. For more info on xUnit, check out the Getting Started page.

  5. When you’re ready to run your tests, just debug/run your app as you would any other. You can use a device or simulator.

Current status on the Runners

The runners today are functional, but they’re not pretty. The iOS and Android runners do share quite a bit of code, but the UI is duplicated as it was created before Xamarin.Forms was announced. There’s a ton of room for improvement and I welcome any help in the form of a Pull Request. The code is all on GitHub here.

Using xUnit to test Metro Style Applications

July 4, 2012 Coding No comments ,

Many people would like to use xUnit to test their Metro-style applications. While this isn’t yet currently possible with the main xUnit binaries, I have updated the code to get it working on WinRT. Without getting into the nitty-gritty, I’ve created a Project Template that’ll make it a snap to get started.

You’ll need two things

  1. The xUnit.net runner for Visual Studio 2012. This lets the Unit Test Explorer recognize and run xUnit tests.
  2. The xUnit Test Library Template for Windows Store apps. With this, you can easily create a unit test project to test your metro style applications and libraries.
  3. I haven’t yet created a template installer VSIX for the Code Gallery, but that’s coming soon – need to resolve a few issues with it first.For now, just extract the zip file into your Documents\Visual Studio 2012\Templates directory. It’ll put a file into the Documents\Visual Studio 2012\Templates\ProjectTemplates\Visual C#\Windows Metro style directory.

    UPDATE (8/29/2012): For the RTM version, I’ve now created a VSIX installer for the template, so just grab it off of the gallery and you’ll be good to go.

If you have Visual Studio running, you’ll have to restart it to pickup the new template; once you do, you’ll see the following in the File –> New Project dialog:

That’s all folks, just add a reference to the libraries you want to test and get started. There’s a passing test already in the template so you can verify that it works for you.

Enjoy!