Agile Sync Usage with Confluence

Agile Sync is an extension for the Matrix Instance for content synchronization between Confluence or other Wiki - like web source systems, and Matrix Requirements projects. It consists of a two stage syncing process:

Content is fetched from Confluence using a command line tool and uploaded to Matrix and stored in an intermediate category (Sync category).
The imported content can be viewed in Matrix and linked to Matrix items. Once linked, changes that come in at a later point can be compared to what is available in Matrix and updated selectively.

This process makes it easy to pull external content into Matrix but still provides control over what becomes part of the “regular” Matrix content.

The sync is usually run in regular intervals and there are labels that can be used to filter for changed items that need to be pulled in.

Prerequisites

To be able to use Agile Sync you need the following:

Matrix feature licenses
- Agile Sync module
- Variants and Labels module
The Confluence Sync command line tool
A machine to run the above tool on with Java installed (minimum version 8) which has access to both the Confluence and Matrix servers

General usage

Agile Sync pulls in pages from outside sources, either individually or in a tree structure. The synced pages will appear in a special category (sync items) and can then be linked to a standard category in Matrix. The sync items will show information if the source and target have diverged, the link to the original item etc.

The sync items display some extra information that makes it easier to track the status:

Status label: This will be automatically set depending on the status and can be used to filter the tree or search for items at this state
Link status: This is where you can update the linked items, create a linked item (if new) etc.
Source info: Information about where this originally came from so you can jump back to the original source

New Items

When the sync tool finds new items in Confluence then it creates a new sync item in Matrix. These new items are in a “Not linked” state and are marked as New. You can choose to sync with an existing item or create a new item.

To sync to an existing item, press “Select item to sync with”. In the dialog that opens up you can select an existing item from the Matrix tree.

To create a new item, press “Create Requirements”. This will open up a create dialog with title and description prefilled based on the Matrix Sync Item content. Saving will create the new item in Matrix, synced with the external source item, via the Matrix Sync Item. In the screenshot above, this creates a new "requirement" Matrix item. However, in the Administration Client you can configure agile sync to create a different category of Matrix item (e.g. Risk or Specification; see instructions below).

Changed Items

If linked and imported items are changed in the external source, then the Matrix Sync Item changes into Out of Sync state. You can sync the content again, overwriting potential changes on the target Matrix item, or open a merge dialog to view differences and select which fields to copy (green buttons in the screenshot below):

Finding Changed Items

To quickly get a list of all items that have changed, you can use a MRQL query in the Matrix tree. Type this query into the search box above the tree and set the display to only show matching items. This will give you a quick way to review everything that is no in the synced state:

CODE

mrql:label!=synced AND category=CONF

Setups steps

Create a sync category in Matrix
Set up the sync tool
Run initial sync

Configure a project for AgileSync

An item category is defined as the “sync control category” for the Confluence / Agile system Sync content (in this article only referred to as “Confluence”).

Also, the Item Categories which shall be using the AgileSync content (“Targets”) are defined.

The sync control items will contain a rich text field which holds the content to be synchronized between Confluence and Matrix, and buttons to control the merging of the content between the (agile) source and the Matrix item (usually the target).

Each sync control item “knows” the URL of the external content source. For hierarchical structures, “Sync control” folders hold the URL of the external hierarchy.

Configure the Matrix Sync Item

Create a new category for incoming items, using the Design item as a starting point. You can freely choose the ID, we use “CONF” in the examples below.
Add the following fields:
- Status (syncStatus)
- Source (syncSourceInfo)
Leave the following default fields active
- Description
- Labels
Remove the following fields (or hide them)
- Tasks
- Links
Open the Category Settings and click on “Sync Configuration” to configure the target type. This is the list of Categories you can generate from the Matrix Sync Items. You can choose multiple target categories.
Add a tracking field in the FOLDER category
- Go to Folder category
- Add a new field SyncControlField of type TextLine
- Open advanced settings and add the line "visibleOption":"CONF" to prevent the field from showing in other categories
- Copy the field ID and keep for the sync config
Configure Labels for the project using the template below. If you use the sample JSON below and you decided not to use “CONF” as Matrix Sync Item category then make sure you change to the correct Sync Item category.

JSON

{
  "labels":[
    {
      "label":"NEW",
      "categories":[
        "CONF"],
      "editors":[
        "wolfgang"],
      "reportName":"New",
      "style":{
        "filter":{
          "off":{
            "displayName":"New",
            "foreground":"orange",
            "background":"transparent",
            "icon":"",
            "tooltip":"click to show unsynced items"
          },
          "on":{
            "displayName":"New",
            "foreground":"white",
            "background":"orange",
            "icon":"",
            "tooltip":"click to show all items"
          }
        },
        "label":{
          "off":{
            "displayName":"New",
            "foreground":"orange",
            "background":"transparent",
            "icon":"",
            "tooltip":"has been synced or ignored before"
          },
          "on":{
            "displayName":"New",
            "foreground":"orange",
            "background":"transparent",
            "icon":"check",
            "tooltip":"new item, not yet synchronized or ignored"
          }
        }
      }
    },
    {
      "label":"IGNORE",
      "categories":[
        "CONF"],
      "editors":[
        "wolfgang"],
      "reportName":"Ignore",
      "style":{
        "filter":{
          "off":{
            "displayName":"Ignore",
            "foreground":"orange",
            "background":"transparent",
            "icon":"",
            "tooltip":"click to show items not to be synced"
          },
          "on":{
            "displayName":"Ignore",
            "foreground":"white",
            "background":"orange",
            "icon":"",
            "tooltip":"click to show all items"
          }
        },
        "label":{
          "off":{
            "displayName":"Ignore",
            "foreground":"orange",
            "background":"transparent",
            "icon":"",
            "tooltip":"is not ignored"
          },
          "on":{
            "displayName":"Ignore",
            "foreground":"orange",
            "background":"transparent",
            "icon":"check",
            "tooltip":"is ignored"
          }
        }
      }
    },
    {
      "label":"SYNCED",
      "categories":[
        "CONF"],
      "editors":[
        "wolfgang"],
      "reportName":"Synced",
      "style":{
        "filter":{
          "off":{
            "displayName":"Synced",
            "foreground":"orange",
            "background":"transparent",
            "icon":"",
            "tooltip":"click to show synced items"
          },
          "on":{
            "displayName":"Synced",
            "foreground":"white",
            "background":"orange",
            "icon":"",
            "tooltip":"click to show all items"
          }
        },
        "label":{
          "off":{
            "displayName":"Synced",
            "foreground":"orange",
            "background":"transparent",
            "icon":"",
            "tooltip":"is not synchronized"
          },
          "on":{
            "displayName":"Synced",
            "foreground":"orange",
            "background":"transparent",
            "icon":"check",
            "tooltip":"has been synchronized"
          }
        }
      }
    },
    {
      "label":"UNSYNC",
      "categories":[
        "CONF"],
      "editors":[
        "wolfgang"],
      "reportName":"Out Of Sync",
      "style":{
        "filter":{
          "off":{
            "displayName":"Out Of Sync",
            "foreground":"orange",
            "background":"transparent",
            "icon":"",
            "tooltip":"click to show out of sync items"
          },
          "on":{
            "displayName":"Out Of Sync",
            "foreground":"white",
            "background":"orange",
            "icon":"",
            "tooltip":"click to show all items"
          }
        },
        "label":{
          "off":{
            "displayName":"Out Of Sync",
            "foreground":"orange",
            "background":"transparent",
            "icon":"",
            "tooltip":"not changed after synchronization"
          },
          "on":{
            "displayName":"Out Of Sync",
            "foreground":"orange",
            "background":"transparent",
            "icon":"check",
            "tooltip":"changed, needs to be re-synchronized"
          }
        }
      }
    }],
  "groups":[
    {
      "selection":"or",
      "labels":[
        "NEW",
        "IGNORE",
        "SYNCED",
        "UNSYNC"]
    }],
  "design_reviews":[
  ],
  "useFilterMenu":false
}

Running synchronization

The sync process is executed using a AgileSync client, in this case the Confluence client. It needs a configuration XML file that contains the path and access information to the Confluence instance.

Extract the provided ZIP file into a directory. It should contain the ExtractConfluence.jar and a syncConfig.xml file

Modify the configuration file for the Sync tool:

XML

<config>
    <matrix
            project='SYNC_TEST' <- Matrix project to sync into
            instance='https://<yuourinstance>.matrixreq.com' <- URL of the instance
            token='<Matrix token>' <- Username
            />
    <confluence path='https://<youratlassian>.atlassian.net/wiki'> <- Root URL of Confluence
        <auth user='username' pwd='<token>' />
    </confluence>
    <category short='CONF' <- The Category to sync into
              folderControlField="SyncControlField"> <- The textline field on the folder category
        <pages> <- Pages to sync
            <page id='25755659' <- The ID of the page to sync
              root="1"/> <- Treat this page as a folder and include subpages
            <page id='18481192' /> <- Import this page individually
        </pages>
        <spaces> <- Spaces to sync
            <space label='TST' />
        </spaces>
    </category>
</config>

In a command shell, in the extracted directory, run the .jar file using the modified configuration file. This will read the specified pages from Confluence and create the required Sync Items in Matrix.
CODE
```
java -jar ExtractConfluence.jar syncConfig.xml
```
Check the new Matrix Sync Items which were created in the specified Matrix project
From each Matrix Sync Item, use the control in that item to create the target items (REQ, SPEC….) - see the instructions above.
For later synchronization, re-run the script with the same configuration, this will update the Matrix Sync Items if content in Confluence has changed. You can run this with an automation to make sure it is executed in regular intervals (e.g. cron or similar automation tools)

Example configuration:

An example can be found here: https://demo.matrixreq.com/AGILE_CONFLUENCE.