Quality

We know quality is very important. We hold ourselves accountable to the highest quality standards. Our Quality Engineering Team assess software at every stage of development and release.   We follow a rigorous process to ensure that our software adheres to our CORE quality standards and our customer expectations.

At resolver, the maturity and effectiveness of our product quality process is continuously measured and improved. To ensure we have visibility, consistency and confidence, we are always focusing on increased test coverage, more business-centric automation, more reliable automation, easier releases and earlier feedback. The metrics we look at regularly to maintain and improve our confidence and quality are:

  1. Test coverage: continuous test coverage close 100%.
  2. Qualified builds: automated build validation close to 100%.
  3. Regression target: automated regression suite greater than 90% of total tests and run them frequently.
  4. Defect trends: Customer found defects < 9 per quarter.
  5. Automation stability: flaky test trends close to < 5%.
  6. Early Bug Discovery: > 90% of bugs discovered before release regression for major releases.

We follow an agile development process. Quality Engineers works side by side with developers on each feature and utilizes the early development testing principle. We focus on early feature functionality testing to find the issues sooner than later.

All tests are created, reviewed, executed and automated by the Quality Engineers during the development cycle.   All automated tests are added to one or more of the automation suites that get executed by our feature, release and automation team continuously (feature, smoke, build, release, performance or security).

We run our automation frequently and early.     The automation regression tests are run before and after every merge, and the full suite of build and release automated regression tests (3400+) are run every single night.   Full suite performance tests for all our application are run weekly, with easy to compare and easy to interpret visual results.

We use Jira Xray to track and maintain all our tests.     Both manual and automated test results are recorded in Jira Xray.   Features, requirement, bugs, tests and executions are all linked together for traceability.   Real-time dashboards are used to plan, track and report our feature team quality work and release testing.   These metrics helps us continuously monitor and improve on test coverage, time to market, reliability and efficiency of automated tests and overall quality.

Releases metrics are looked at continuously during the release.   All releases are released with 100% execution rate and 99%+ pass rate.   Any bugs discovered during release testing are reviewed with product team to determine the severity and if the product can be released with it.