Microsoft Dynamics Blog

Welcome to Armanino’s Microsoft Dynamics Blog, where you’ll find information on topics across Microsoft Dynamics 365 ERP and CRM, Dynamics GP, Power BI, PowerApps, Flow and more. Use these tips, tricks and insights to get the most out of your applications! Want these articles delivered straight to your inbox? Subscribe to our newsletter below.

Tuesday, November 27, 2018

Test Automation Using the Regression Suite Automation Tool

Posted by Sandy Strong

Now that Microsoft is going ‘evergreen’ with continuous application and platform updates, test automation has become more critical to help ensure that your Dynamics 365 for Finance and Operations functionality continues to operate as expected.  This is particularly important for clients in FDA-regulated industries, due to software validation requirements.

To facilitate the automated testing process, on October 1st Microsoft released their new Regression Suite Automation Tool (RSAT).  The intention for this tool is business cycle testing, as well as integration testing that generally occurs at the end of the development cycle.  The RSAT tool uses tests created with the Dynamics 365 for Finance and Operations Task Recorder functionality, and provides a simple means of executing those tests.  With it, tests can be ordered as desired for execution.  Furthermore, with the RSAT tool test data parameters are stored in Excel, meaning a power user can simply edit those values when needed and no coding is necessary.  Although task recorder tests can be executed directly from the task recorder tool in Dynamics 365 for Finance and Operations, the RSAT tool reduces the time and effort to run those tests.

The following flowchart of the entire process is pulled from the RSAT user manual:

Flowchart from RSAT ManualHere’s a bit more about the process, with a few hyperlinks to additional information.

  1. Create a business process modeler (BPM) library in Lifecycle Services (LCS). This is essentially an outline of the automated tests you plan to create.
  2. Record tests using Task Recorder.
  3. When you finish a task recording, the Task Recorder will prompt you for how you wish to save the recording. Chose to save the tests to your BPM library in LCS:
    Task Recorder - Save to Lifecycle Services
  4. Select the BPM library you created for your automated tests, select the correct test, and click OK.
  5. Once you’ve finished recording your tests and have saved to LCS, you’re ready to sync your BPM library with Azure DevOps. This creates a test case in Azure DevOps for each of your tests in your Lifecycle Services BPM.
    • From BPM, select the View this page in the updated BPM experience link:
    • Select the More library actions icon (…) and VSTS sync.
    • Once this completes, select More library actions and Sync test cases:
    • You’ll know the process is complete once the sync buttons are no longer grayed out. Also, the Requirements tab of each tests will have a link to the epic and test case that were automatically created:
  6. Create a test suite in Azure DevOps.
    • On the toolbar, select Test Plans:
    • In the left pane, select + > Static suite:
    • Enter a name for the suite.
    • Click Add existing and query the tag LCS:Test Cases.
    • Click Run query.
    • Select the test cases you wish to include the in the test suite and click Add test cases.
  1. Launch the regression suite automation tool (RSAT). See the RSAT user manual for details on the settings.
  2. Click Load to load the tests from the Azure DevOps test suite.
  3. If desired, use the up and down arrow buttons to adjust the order of the tests:
  4. Click New to create the Excel parameter files for each test.
  5. If desired, mark a test and click the Edit button to edit the parameter file for that test:
  6. Mark all test cases you wish to run and click Run:
  7. Once all the tests have been run, the Result column will be updated to indicate whether each test passed or failed. For any failed tests, you can click on the result to view the error message.
  8. Additional test run results are available in Azure DevOps under Test > Runs.
« | »