Bi-Directional Traceability in Jira With X-Ray

Image for post

Photo by Marten Bjork on Unsplash

A while ago I wrote about using Notion to build a test reporting structure that allows for bi-directional traceability from User Story to Test Execution and vice-versa.

I then turned that blog into a talk — ‘Test Reporting — Why I Hate Piecharts’ and gave it at my local Ministry of Testing meet-up where I argued the case against test reports being just a number of tests run and a pass rate.

When I started building up a backlog in Jira for the app I’m currently working on I knew that I wanted to have bi-directional traceability between the user stories I was adding and the automated test suite I’d be eventually writing to check everything was working.

The decision I had to make was then — which Jira based test reporting tool to use, Zephyr or X-Ray?

I’ve used Zephyr in the past and found it to be really powerful but the custom ZQL language that it uses to query test cases was a little bit of a put off.

X-Ray on the other hand uses Jira’s JQL so you can build custom views easily anywhere you can use JQL queries.

Getting Started with X-Ray

X-Ray uses Jira tasks as a backing for the entities it uses:

  • **Test **— A test case to verify behaviour
  • **Test Set **— A grouping of tests
  • Precondition — Set up for a test case
  • Test Execution — The actual running of a test case
  • Test Plan — A plan of what to test

If you don’t already have the relevant issue types created you’ll need to create them and configure X-Ray’s mapping, if you’re using a Next-Gen project then you might find that this involves some work as you don’t start with many issue types.

Once you have the mapping in place you’ll then be able to create test cases for your user stories.

X-Ray supports three types of test case structures; Gherkin, Manual and Unstructured which can be used to define how to carry out the test and be used in automation.

I’m not a big fan of Gherkin so I chose to go with Unstructured format and link the description to the name of the test in Jest, the test runner I’m using in my project.

Making things traceable

Having floating test cases won’t give you much benefit but you can link them to your user stories and epics using Jira’s issue linking tool and defining a ‘tests’ and ‘tested by’ relationship between the test case and the user story/epic.

You can define the relationship to use for traceability in X-Ray’s settings if you want to use a custom one but the default ‘tests’/’tested by’ relationship should cover most use cases.

Once the relationship between the user story/epic and the test case is created you should then see the Coverage section of the user story populated with the test case, over time this will show the status of test executions as they are run.

In Jira Next-Gen projects there’s one new feature that I really like as someone acting as Product Owner, which is the Roadmap view.

The Roadmap view shows the epics in a Gantt chart and shows you how much of the epic’s stories have been completed.

If you create a relationship between the test case and the epic then you can also see the test coverage of all stories under the epic which brings your test reporting into this higher level and gives a really powerful insight into the quality of what you’re building.

Automating test reporting

The basic relationship setup will allow you to carry out your manual tests and provide traceability on those but the real benefit of a tool like X-Ray is that you can import test run data and automate this process to run tests across multiple environments.

This automation is something I had done previously with Zephyr, running tests against 20 environments in parallel and using Jira as a dashboard to show the status of the tests for reporting this to the team.

X-Ray offers a number of means for automating test case and test execution creation, allowing you to create in bulk via an import or to create on the fly via calls to a set of API endpoints.

If you’re running a suite of tests then I would recommend the bulk import approach as you only need to hit the API once which keeps your test run fast, if you do hit up the API as you run your tests then you’ll have the overhead of the network requests which will add time.

For my project I’m using an upload of a JUnit report from Jest generated using the jest-junit test reporter which once configured correctly allowed me to link a test case in Jest to an Acceptance Criteria in my user stories.

Creating JUnit Reports

I found with jest-junit I had some additional work to do in order to set it up how I wanted it, here’s the config I used:

module.exports = {
  reporters: [
    'default',
    [
      'jest-junit',
      {
        suiteName: 'Tests',
        classNameTemplate: '{classname}', titleTemplate: '{title}'
      }
    ]
  ],
}

By default the titleTemplate includes the classname which for my use meant I had duplicated titles which threw the test linking off.

Authenticating with X-Ray to upload test results

In order to use the X-Ray API to upload test results you need to first create an API key, this can be done by going to:

  1. Settings (cog icon in top right, not to be confused with project settings)
  2. Apps (bottom of list)
  3. API Keys (under the X-Ray heading)

Once you’ve created a client_id and client_secret you’ll then use those to create an auth token which will be used when talking to the API.

You can use the following cURL command to get an authentication token:

token=$(curl -H "Content-Type: application/json" -X POST --data @"xray_auth.json" https://xray.cloud.xpand-it.com/api/v1/authenticate| tr -d '"')

The xray_auth.json file looks like:

{"client_id": "[ADD CLIENT ID HERE]", "client_secret": "[ADD CLIENT SECRET HERE]"}

Once you’ve got a token you can then use it in your calls to upload your tests:

curl -H "Content-Type: text/xml" -X POST -H "Authorization: Bearer $token"  --data @"junit.xml" https://xray.cloud.xpand-it.com/api/v1/import/execution/junit?projectKey=[PROJECT KEY]

The junit.xml file is the output from jest-junit or however you’ll be generating your JUnit format test report.

When running this setup in CI I stored the client_id and client_secret in the CI’s secret store and used echo to construct the xray_auth.json file before calling the cURL to generate the token, this allows for the credentials to be stored and used securely without committing to git.

Test Coverage Reports

Within Jira you can make use of the test coverage data to help prioritise your backlog items in order to fix technical debt & bugs and you can also use it to see the health of a particular piece of functionality as it progresses through different environments.

Ultimately how you use the test coverage will be dependent on your situation but for my app I’m using it to automate my Acceptance Criteria testing and then using the test results across a range of devices to ensure that my app is working and to highlight where additional development effort may be needed to support older or more obscure set ups.

Out of the box X-Ray offers a number of traceability reports and even allows for custom templates for the PDF test run reports, I don’t use these though as I’m of the opinion that test reports are never actually used and just go into inboxes to die.

For me the real power of a tool like X-Ray is to be able to use dashboards and other dynamic reporting tools to see the data, query it and explore the links between different tickets to uncover the real quality issues impacting the product instead of just checking that some tests have been run.

The Startup

Medium’s largest active publication, followed by +685K people. Follow to join our community.

Follow

#agile #jira #product-ownership #testing #product

What is GEEK

Buddha Community

Bi-Directional Traceability in Jira With X-Ray

Explore your JIRA Data with Power BI

JIRA Software provides bug tracking, issue tracking, and project management capabilities for teams and organizations. The JIRA content pack for Power BI helps you quickly import JIRA data so you can get an instant dashboard to analyze workloads, see how quickly you’re resolving issues, visualize velocity over time, and more. Power BI helps you quickly filter by project or component to generate new insights into your JIRA data.

To get in-Depth knowledge on Power BI you can enroll for a live demo on Power BI online training

This is image title

To connect to the JIRA content pack, simply choose JIRA from the list of available content packs. You will be asked to provide your JIRA URL and credentials. Once the connection has completed, Power BI will automatically create an out-of-the-box dashboard, report, and dataset with data from JIRA.

dashboard

The dashboard for the JIRA content pack shows you key metrics about your workloads, velocity over time, and the breakdown of your bugs and issues by status and assignee. Clicking on any of the dashboard tiles will open the JIRA content pack report, where you can interact and explore your data.

This is image title

You can slice the data in the report using the ‘Item Type’ slicer on the top, so you can find exactly the items you are looking for. Power BI reports are interactive, so if you select a ‘Fix Version’, a ‘Component’, or an ‘Assignee’, the rest of the report page will update per the selection.Learn more from Power BI online course

The report has a second page that focuses on the velocity of creating and resolving issues, and also shows the list of recently resolved items.

This is image title

After the initial import, the dashboard and the reports continue to update daily so you are always seeing up-to-date data. You can control the refresh schedule on the dataset, and configure it to refresh at the exact times you choose.
Take your career to new heights of success with Power BI online training Hyderabad

#power bi training #power bi course #learn power bi #power bi online training #microsoft power bi training #power bi online course

sophia tondon

sophia tondon

1620885491

Microsoft Power BI Consulting | Power BI Solutions in India

Hire top dedicated Mirosoft power BI consultants from ValueCoders who aim at leveraging their potential to address organizational challenges for large-scale data storage and seamless processing.

We have a team of dedicated power BI consultants who help start-ups, SMEs, and enterprises to analyse business data and get useful insights.

What are you waiting for? Contact us now!

No Freelancers, 100% Own Staff
Experienced Consultants
Continuous Monitoring
Lean Processes, Agile Mindset
Non-Disclosure Agreement
Up To 2X Less Time

##power bi service #power bi consultant #power bi consultants #power bi consulting #power bi developer #power bi development

sophia tondon

sophia tondon

1619670565

Hire Power BI Developer | Microsoft Power BI consultants in India

Hire our expert Power BI consultants to make the most out of your business data. Our power bi developers have deep knowledge in Microsoft Power BI data modeling, structuring, and analysis. 16+ Yrs exp | 2500+ Clients| 450+ Team

Visit Website - https://www.valuecoders.com/hire-developers/hire-power-bi-developer-consultants

#power bi service #power bi consultant #power bi consultants #power bi consulting #power bi developer #power bi consulting services

Origin Scale

Origin Scale

1614331774

Traceability Software

At OriginScale traceability is not optional and it comes by default. We use blockchain technology to track and trace key critical events of your products and ingredients at the batch-lot level and provide real-time actionable insights along with a visualization tool.

#traceability of products #traceability #traceability software #product traceability #traceability software for manufacturing

Bi-Directional Traceability in Jira With X-Ray

Image for post

Photo by Marten Bjork on Unsplash

A while ago I wrote about using Notion to build a test reporting structure that allows for bi-directional traceability from User Story to Test Execution and vice-versa.

I then turned that blog into a talk — ‘Test Reporting — Why I Hate Piecharts’ and gave it at my local Ministry of Testing meet-up where I argued the case against test reports being just a number of tests run and a pass rate.

When I started building up a backlog in Jira for the app I’m currently working on I knew that I wanted to have bi-directional traceability between the user stories I was adding and the automated test suite I’d be eventually writing to check everything was working.

The decision I had to make was then — which Jira based test reporting tool to use, Zephyr or X-Ray?

I’ve used Zephyr in the past and found it to be really powerful but the custom ZQL language that it uses to query test cases was a little bit of a put off.

X-Ray on the other hand uses Jira’s JQL so you can build custom views easily anywhere you can use JQL queries.

Getting Started with X-Ray

X-Ray uses Jira tasks as a backing for the entities it uses:

  • **Test **— A test case to verify behaviour
  • **Test Set **— A grouping of tests
  • Precondition — Set up for a test case
  • Test Execution — The actual running of a test case
  • Test Plan — A plan of what to test

If you don’t already have the relevant issue types created you’ll need to create them and configure X-Ray’s mapping, if you’re using a Next-Gen project then you might find that this involves some work as you don’t start with many issue types.

Once you have the mapping in place you’ll then be able to create test cases for your user stories.

X-Ray supports three types of test case structures; Gherkin, Manual and Unstructured which can be used to define how to carry out the test and be used in automation.

I’m not a big fan of Gherkin so I chose to go with Unstructured format and link the description to the name of the test in Jest, the test runner I’m using in my project.

Making things traceable

Having floating test cases won’t give you much benefit but you can link them to your user stories and epics using Jira’s issue linking tool and defining a ‘tests’ and ‘tested by’ relationship between the test case and the user story/epic.

You can define the relationship to use for traceability in X-Ray’s settings if you want to use a custom one but the default ‘tests’/’tested by’ relationship should cover most use cases.

Once the relationship between the user story/epic and the test case is created you should then see the Coverage section of the user story populated with the test case, over time this will show the status of test executions as they are run.

In Jira Next-Gen projects there’s one new feature that I really like as someone acting as Product Owner, which is the Roadmap view.

The Roadmap view shows the epics in a Gantt chart and shows you how much of the epic’s stories have been completed.

If you create a relationship between the test case and the epic then you can also see the test coverage of all stories under the epic which brings your test reporting into this higher level and gives a really powerful insight into the quality of what you’re building.

Automating test reporting

The basic relationship setup will allow you to carry out your manual tests and provide traceability on those but the real benefit of a tool like X-Ray is that you can import test run data and automate this process to run tests across multiple environments.

This automation is something I had done previously with Zephyr, running tests against 20 environments in parallel and using Jira as a dashboard to show the status of the tests for reporting this to the team.

X-Ray offers a number of means for automating test case and test execution creation, allowing you to create in bulk via an import or to create on the fly via calls to a set of API endpoints.

If you’re running a suite of tests then I would recommend the bulk import approach as you only need to hit the API once which keeps your test run fast, if you do hit up the API as you run your tests then you’ll have the overhead of the network requests which will add time.

For my project I’m using an upload of a JUnit report from Jest generated using the jest-junit test reporter which once configured correctly allowed me to link a test case in Jest to an Acceptance Criteria in my user stories.

Creating JUnit Reports

I found with jest-junit I had some additional work to do in order to set it up how I wanted it, here’s the config I used:

module.exports = {
  reporters: [
    'default',
    [
      'jest-junit',
      {
        suiteName: 'Tests',
        classNameTemplate: '{classname}', titleTemplate: '{title}'
      }
    ]
  ],
}

By default the titleTemplate includes the classname which for my use meant I had duplicated titles which threw the test linking off.

Authenticating with X-Ray to upload test results

In order to use the X-Ray API to upload test results you need to first create an API key, this can be done by going to:

  1. Settings (cog icon in top right, not to be confused with project settings)
  2. Apps (bottom of list)
  3. API Keys (under the X-Ray heading)

Once you’ve created a client_id and client_secret you’ll then use those to create an auth token which will be used when talking to the API.

You can use the following cURL command to get an authentication token:

token=$(curl -H "Content-Type: application/json" -X POST --data @"xray_auth.json" https://xray.cloud.xpand-it.com/api/v1/authenticate| tr -d '"')

The xray_auth.json file looks like:

{"client_id": "[ADD CLIENT ID HERE]", "client_secret": "[ADD CLIENT SECRET HERE]"}

Once you’ve got a token you can then use it in your calls to upload your tests:

curl -H "Content-Type: text/xml" -X POST -H "Authorization: Bearer $token"  --data @"junit.xml" https://xray.cloud.xpand-it.com/api/v1/import/execution/junit?projectKey=[PROJECT KEY]

The junit.xml file is the output from jest-junit or however you’ll be generating your JUnit format test report.

When running this setup in CI I stored the client_id and client_secret in the CI’s secret store and used echo to construct the xray_auth.json file before calling the cURL to generate the token, this allows for the credentials to be stored and used securely without committing to git.

Test Coverage Reports

Within Jira you can make use of the test coverage data to help prioritise your backlog items in order to fix technical debt & bugs and you can also use it to see the health of a particular piece of functionality as it progresses through different environments.

Ultimately how you use the test coverage will be dependent on your situation but for my app I’m using it to automate my Acceptance Criteria testing and then using the test results across a range of devices to ensure that my app is working and to highlight where additional development effort may be needed to support older or more obscure set ups.

Out of the box X-Ray offers a number of traceability reports and even allows for custom templates for the PDF test run reports, I don’t use these though as I’m of the opinion that test reports are never actually used and just go into inboxes to die.

For me the real power of a tool like X-Ray is to be able to use dashboards and other dynamic reporting tools to see the data, query it and explore the links between different tickets to uncover the real quality issues impacting the product instead of just checking that some tests have been run.

The Startup

Medium’s largest active publication, followed by +685K people. Follow to join our community.

Follow

#agile #jira #product-ownership #testing #product