Azure DevOps Integration

From this topic you will learn how to run Rapise tests with Node-Tap and Azure DevOps.

Node-Tap

Prerequisites

Download and install Node.js. Open command prompt and run the following commands to make sure you have Node.js and npm installed:

C:\Users\inflectra>node --version
v10.16.3

C:\Users\inflectra>npm --version
6.9.0

Install Node-Tap package with npm:

npm install tap -g

Discover the folder where Node-Tap is installed:

C:\Users\inflectra>npm list tap -g
C:\Users\inflectra\AppData\Roaming\npm
-- tap@14.6.9

Configure system variable NODE_PATH and set it to node_modules subfolder. In our case it is:

C:\Users\inflectra\AppData\Roaming\npm\node_modules

Relaunch Command Prompt and verify the system variable:

C:\Users\inflectra>echo %NODE_PATH%
C:\Users\inflectra\AppData\Roaming\npm\node_modules

Test Framework

As a starting point you may use sample framework we published on GitHub.

The root folder of the framework contains:

Each Rapise test in this framework is a direct subfolder of the root folder:

AzureDevOpsFriendlyArchhitecture 
|
|-CreateNewBook
|-FailedLogin

test folder contains entries for Node-Tap. These are files with predefined content and names equal to names of Rapise tests.

AzureDevOpsFriendlyArchhitecture 
|
|- tests
    |
    |-CreateNewBook.js
    |-FailedLogin.js

There is also tap-parallel-not-ok in tests folder that instructs Node-Tap to run tests sequentially.

Run Tests with Tap

To run tests on a local machine open command prompt in the root folder of the framework and execute:

tap

You should see tests running: Node-Tap Execution

To pass parameters to a Rapise test specify --test-arg. The value is JSON stringified object. Every property of this object will be evaluated by Rapise as a global variable:

tap --test-arg="{'g_browserLibrary':'Selenium - Chrome'}"

Azure DevOps Pipeline v1

To run tests in Azure DevOps you need to configure a pipeline. Here is the template to run Rapise tests:

azure-pipelines.yml

# Starter pipeline
# Start with a minimal pipeline that you can customize to build and deploy your code.
# Add steps that build, run tests, deploy, and more:
# https://aka.ms/yaml

trigger:
- master

pool:
  name: 'default'

steps:

- script: |
    echo Running tests...
    tap --test-arg="{'g_browserLibrary':'Selenium - Chrome'}"
  displayName: 'Execute Tests'
  continueOnError: true

- script: |
    call tap2xunit.cmd
  displayName: "Convert TAP to XUnit"

- script: |
    cscript.exe fixunit.js
  displayName: 'Fix XUnit Reports'

- task: PublishTestResults@2
  inputs:
    testResultsFormat: 'JUnit'
    testResultsFiles: 'results/*.xml'
    testRunTitle: 'Basic Tests'
    failTaskOnFailedTests: true

The pipeline above assumes that default agent pool contains a Self-hosted Windows Agent installed on a machine wuth Rapise and Node-Tap (see Prerequisites).

See the video to learn how to install and configure a self-hosted agent.

The pipeline consists of the following steps:

  1. Run tests via tap command. You can pass parameters if needed.
  2. Convert TAP formatted execution results to XUnit formatted reports.
  3. Adjust XUnit reports for better processing by Azure DevOps.
  4. Upload test results to Azure with Publish Test Results task.

After execution of the pipeline one can review test results.

Test Report

Azure DevOps Pipeline v2

If you do not want each step of a Rapise test to be reported as test case in Azure DevOps use modified pipeline version:

azure-pipelines-summary.yml

# Starter pipeline
# Start with a minimal pipeline that you can customize to build and deploy your code.
# Add steps that build, run tests, deploy, and more:
# https://aka.ms/yaml

trigger: none

pool:
  name: 'default'

steps:

- script: |
    echo Running tests...
    tap --no-coverage --test-arg="{'g_browserLibrary':'Chrome HTML'}"
  displayName: 'Execute Tests'
  continueOnError: true

- script: |
    call tap2xunit.cmd
  displayName: "Convert TAP to XUnit"

- script: |
    cscript.exe summarize.js
  displayName: 'Fix XUnit Reports'

- task: PublishTestResults@2
  inputs:
    testResultsFormat: 'JUnit'
    testResultsFiles: 'results/*.xml'
    testRunTitle: 'Basic Tests'
    failTaskOnFailedTests: true

It calls summarize.js to adjust XUnit reports. Azure DevOps report for this pipeline looks like:

Test Report Summary