Skip to end of metadata
Go to start of metadata

Jerry's goal:

  • Performance degradation notification on a per commit basis.

 

Current per commit gap:

  • Performance test are run on env14 which is shared with AFTs. High variability in passing/failing due to inconsistent load from running in a shared environment.

Possible solutions:

  • Run performance tests on dedicated env.
    • Work required:
      • Create another env.
      • Point existing AFTs at new env, using env14 which already has AppDynamics configured and increased JVM Memory settings as the dedicated performance testing env.
  • Schedule JMeter and AFTs to run after each other, due to AFT per commit execution times JMeter will only be run every 4 hours. When nightly AFTs are run on Saucelabs load can impact JMeter tests.
    • Work required:
      • Chain AFT and JMeter Jenkins Jobs.

 

Current performance gap:

  • Running JMeter tests under 1 Jenkins job means the server JVM GC is not in a clean state for tests other than the first.

Possible solutions: 

  • Create 1 JMeter Jenkins job per JMeter test that bounces the test env - shared env will still impact
    • Work required:
      • Create and Chain 1 JMeter test per Jenkins Job
      • Remotely bounce env without having to do full build and redeploy

 

Current notification gap:

  • JMeter Jenkins plugin notification is based on percentage of the JMeter tests in the job that fail.

Possible solutions:

  • Create 1 JMeter Jenkins job per JMeter test - shared env will still impact
    • Work required:
      • Create and Chain 1 JMeter test per Jenkins Job
  • Create 1 AppDynamics Business Transaction per JMeter test - shared env will still impact
    • Work required
      • Create 1 AppDynamics Business Transaction per JMeter test, with supporting Health Rules based
  • Both to get the most out of both tools.
  • No labels