Skip to content

API Documentation

Jump to bottom Edit New page
Vesa Poikajärvi edited this page May 16, 2013 · 29 revisions

QA Reports has a RESTful HTTP API for programmatically importing test reports, updating the test cases of an existing report, and merging new result files to an existing report. In addition, QA Reports has some API like features which make fetching report data easier. These are documented in the last section.

Authentication token

A personal authentication token is required for operating with the API. The token can be fetched via QA Reports web UI, by clicking the user name link at the upper right center of the page while logged in.

The API key is shown at the bottom of the User Preferences page:

API Responses

All API methods return with HTTP error codes, mostly accompanied by a JSON payload.

HTTP response codes used are:

Response Description
200 Success
403 Invalid authentication token
404 Report not found
422 Invalid result file(s) or missing required fields

JSON response fields are:

  • ok: either 1 when request was processed without errors, 0 if an error occurred
  • url: URL to created report (only from /api/import) if created succesfully
  • errors: If an error occurred, more information about the error is returned.

Example JSON responses:

  • Import successful: *
{"ok": "1", "url": "http://qa-reports.meego.com/1.2/Core/Sanity/Cedartrail/3554"}
  • Import failed: *
{"ok": "0", "errors": {"target": "can't be blank"}}`

/api/import

Description: Import test results Method: POST [encoding multipart/form-data]

Parameters:

Parameter name Description Values
auth_token Personal authentication token
release_version Release version 1.0, 1.1, 1.2
target Target profile Core, Handset, Netbook, IVI
testset Test type / test set Sanity, Acceptance, Basic Feature Testing etc
tested_at Timestamp of the test in format yyyy-mm-dd Optional. If not given, current date is used.
Example value: 2010-12-14
product Product / hardware name Ia-Russellville, N900, Pinetrail etc
result_files[] Report file in XML format
attachments[] Files to be attached with the report: Screenshots, logs etc Optional
title Report title Optional. Default: value generated from other information if user defined title is not given.
objective_txt Test objective. Web UI markup can be used. Optional. Default: value is set from previous report if it exists.
build_txt Build (image) section content. Web UI markup can be used. Optional. Default: value is set from previous report if it exists.
environment_txt Test environment. Web UI markup can be used. Optional. Default: hwproduct name
qa_summary_txt Quality summary. Web UI markup can be used. Optional. Default: value is set from previous report if it exists.
issue_summary_txt Issue summary. Web UI markup can be used. Optional. Default: value is set from previous report if it exists.
patches_included_txt Patches included in SW build being tested. Web UI markup can be used. Optional. Default: value is set from previous report if it exists.
hide_summary Option to hide the result summary section from the created report. Values: true/false Optional. Default: false
hide_metrics Option to hide the report metrics section from the created report (Note: metrics section appears only if metrics have been reported). Values: true/false Optional. Default: false
copy_template_values Option to disable copying values from previous report. By default, QA Reports copies content for report level fields (test objective etc.) and test case comments (if result has not changed) from previous report if new values are not provided. Values: true/false Optional. Default: true

The input can contain multiple result_files[] and attachments[].

There are CSV equivalents for issue_summary_txt and patches_included_txt - issue_summary_csv and patches_included_csv. Both of these accept a comma separated list of plain IDs which are converted to a list of links using the markup syntax. So, sending 1234,2345,4567 as issue_summary_csv would put the following in Issue Summary field:

* [[1234]]
* [[2345]]
* [[4567]]

Caveats in these parameters are that if both _txt and _csv versions are provided, only _txt is used, and if administrator has configured a default prefix for these fields it will be added to plain IDs (i.e. 1234 would become * [[BZ#1234]]). The defined prefix may not be the same as the default used in the web UI.

Because multiple files can be uploaded, HTTP POST with multipart/form-encoding is used.

An example using cURL for testing the API:

curl \
  --form result_files[][email protected]  \
  --form result_files[][email protected] \
  --form attachments[]=coredump \
  --form auth_token=<your_token> \
  --form release_version=1.2 \
  --form target=Core \
  --form testset=Acceptance \
  --form hardware=N900 \
  http://<server>/api/import

/api/update/<Report_ID>

Description: Update the test cases of an existing report Method: POST [encoding multipart/form-data]

Parameters:

Parameter name Description
auth_token Personal authentication token
result_files[] Report file in XML or CSV format.
Report_ID The ID of the target report

The ID of the target report can be fetched from the URL of the corresponding report page, as shown in the picture below:

The input can contain multiple result_files[].

Because multiple files can be uploaded, HTTP POST with multipart/form-encoding is used.

An example using cURL for testing the API:

curl \
--form result_files[][email protected] \
--form result_files[][email protected]
--form auth_token=<your token>
http://<server>/api/update/<Report_ID>

/api/merge/<Report_ID>

Description: Merge new result files to an existing report Method: POST [encoding multipart/form-data]

Parameters:

Parameter name Description
auth_token Personal authentication token
result_files[] Report file in XML or CSV format.
Report_ID The ID of the target report

The input can contain multiple result_files[].

An example using cURL for testing the API:

curl \
--form result_files[][email protected] \
--form result_files[][email protected] \
--form auth_token=<your token> \
http://<server>/api/merge/<Report_ID>

Because multiple files can be uploaded, HTTP POST with multipart/form-encoding is used.

GET APIs

JSON

Report listings

JSON responses can be received from hiearchy level Target profile onwards. Reports listings JSON contains an array of items with fields id, title, tested_at, and url. JSON response can be received by appeding .json to the URI. For example http://server/1.2/SDK/Basic/Tablet.json could return a response like

[{"id":3243,"tested_at":"2011-08-01T00:00:00Z","title":"Sdk Test Report: Tablet Basic 2011-08-01","url":"http://server/1.2/SDK/Basic/Tablet/3243"},
 {"id":3242,"tested_at":"2011-08-01T00:00:00Z","title":"Tablet Image - Fedora 14 - 1.2.0.90.10.20110726.2","url":"http://server/1.2/SDK/Basic/Tablet/3242"},
 {"id":3241,"tested_at":"2011-08-01T00:00:00Z","title":"Tablet Sysroot - Fedora 14 - 1.2.0.90.10.20110726.2","url":"http://server/1.2/SDK/Basic/Tablet/3241"}]

Individual reports

For an individiual report you can fetch a JSON summary, or a full JSON representation of the report (includes the summary as well). To get a summary append the report URI with /summary.json, and for the test case report append the URI with just .json, e.g. http://server/1.2/SDK/Basic/Tablet/3243/summary.json or http://server/1.2/SDK/Basic/Tablet/3243.json.

A summary could look like this:

{"title":"Sdk Test Report: Tablet Basic 2011-08-01",
 "objective":"",
 "build":"",
 "build_id":"1.2.0.90.10.20110726.2",
 "environment":"* Product: Tablet",
 "qa_summary":"",
 "issue_summary":"",
 "summary": { "Total":32, "Pass":30,"Fail":2,"N/A":0,"Measured":0 },
 "features": [
   {"name":"XRPM", "summary": {"Total":10,"Pass":10,"Fail":0,"N/A":0,"Measured":0}},
   {"name":"Sysroot", "summary": {"Total":10,"Pass":10,"Fail":0,"N/A":0,"Measured":0}},
   {"name":"QtCreator", "summary": {"Total":6,"Pass":6,"Fail":0,"N/A":0,"Measured":0}},
   {"name":"Install tests", "summary": {"Total":6,"Pass":4,"Fail":2,"N/A":0,"Measured":0}}
]}

where summary contains case counts for the whole report, and features.summarycase counts per feature. The full report JSON has an additional field testcases under each feature, and the field contains a list of test cases. A single test case could look like this:

{"name":"Double install","tc_id": "TT-1259", "result":"Pass","comment":"","bugs":[{"id": "123", "prefix": "BZ", "type": "bugzilla", "url": "http://bugzilla/show_bug.cgi?id=123"}]}

It also scans all bugs from the report level text fields (issue summary etc.) and collects them to an additional field bugs, i.e.

{"title":"Sdk Test Report: Tablet Basic 2011-08-01",
 ...
 "bugs": [
  {
    "id": "123",
    "prefix": "BZ",
    "type": "bugzilla",
    "url": "http://bugzilla/show_bug.cgi?id=123"
}]

Cumulative report

Cumulative JSON reports can be received from product level, e.g. http://server1.2/Core/Acceptance/N900/cumulative?oldest=ID&latest=ID where ID's are the corresponding report IDs. Cumulative report response could look like this:

{
  "sequences": {
    "titles": [
      "Core Test Report: N900 Acceptance 2011-07-14",
      "Core Test Report: N900 Acceptance 2011-07-15"
    ],
    "dates": [
      "2011-07-14T02:38:02Z",
      "2011-07-15T02:08:37Z"
    ],
    "summaries": [
      { "Total": 45, "Pass": 21, "Fail": 8, "N/A": 16, "Measured": 0 },
      { "Total": 45, "Pass": 32, "Fail": 4, "N/A": 9,  "Measured": 0
      }
    ],
    "features": {
      "Security": [
        { "Total": 2, "Pass": 0, "Fail": 0, "N/A": 2, "Measured": 0 },
        { "Total": 2, "Pass": 0, "Fail": 0, "N/A": 2, "Measured": 0 }
      ]
    }
  },
  "features": [
    {
      "name": "Security",
      "summary": { "Total": 2, "Pass": 0, "Fail": 0, "N/A": 2, "Measured": 0 },
      "testcases": [
        {
          "name": "Access control checkpoint not defined yet",
          "tc_id": null,
          "result": "N/A",
          "comment": "No test case.",
          "last_report": "Core Test Report: N900 Acceptance 2011-07-15",
          "prev_result": "N/A",
          "last_executed": null
        },
        {
          "name": "Accounts & SSO checkpoint not defined yet",
          "tc_id": null,
          "result": "N/A",
          "comment": "No test case.",
          "last_report": "Core Test Report: N900 Acceptance 2011-07-15",
          "prev_result": "Pass",
          "last_executed": null
        }
      ]
    }
  ],
  "summary": { "Total": 45, "Pass": 32, "Fail": 4, "N/A": 9, "Measured": 0 }
}

This response contains three major sections: sequences, features, and summary. sequences contains report titles, dates, and report and feature summaries for each report included. The lists are in the same order, e.g. the first date in dates is for the first report in titles and so on. features contains the cumulative result for each feature and test case, and summary holds the total cumulative summary over selected reports.

Getting allowed values for fields

Some of the fields, namely release, target, and result have restricted values. Some tools may want to provide correct options to users, or prevalidate before uploading. Thus an API exists for querying these values.

  • /api/query/releases *
[{"id":3,"name":"1.0"},{"id":2,"name":"1.1"},{"id":1,"name":"1.2"}]
  • /api/query/targets *
[{"id":1,"name":"Core"},{"id":2,"name":"Handset"},{"id":4,"name":"IVI"},{"id":3,"name":"Netbook"},{"id":5,"name":"SDK"}]
  • /api/query/results *
["Fail","N/A","Pass","Measured"]`

RSS

QA Reports provides RSS feeds from all hiearchy levels except over all releases, and of course for a single test report. Add /rss to the URI to get the feed. Examples:

  • http://server/1.2/rss
  • http://server/1.2/Core/rss
  • http://server/1.2/Core/Sanity/rss
  • http://server/1.2/Core/Sanity/Cedartrail/rss

If you further append the /rss with /tested_at the RSS feed is sorted by tested_at date instead of the default created_at. Example: http://server/1.2/Core/Sanity/rss/tested_at.

You can also use some wildcards to get combined feeds for e.g. reporting tool purposes. Wildcards can be used for release and target parameters. For release, you can use a trailing asterisk (some chars must be present in front of it) to get combined feeds from different releases, e.g. http://server/1.*/Core/rss to get combined feed for Core from 1.0, 1.1, and 1.2 releases. On target level you can use just an asterisk to get results regardless of the target, e.g. http://server/1.1/*/Sanity/rss to get all Sanity results for 1.1. These are not very flexible as they were implemented for a specific purpose, but at least provide some additional features.

Add a custom footer
Add a custom sidebar
Clone this wiki locally