Stanki Guild

Guide · Jira

How we test approval rules in Jira: a million random cases a night

Approval Flow for Jira 3.0.0 is the production version (30 October 2026). Last updated 1 November 2026. On the Atlassian Marketplace.

By Bartosz Stankiewicz, founder, Stanki Guild.

An approval rule is a promise to an auditor. "The person who wrote the requirement never signs it off." "Two different people, not one person twice." "Once three people say no, it stays rejected." Each of these is one line in a spec. In code they meet each other: a substitute signing for someone on holiday, a person who abstains and then changes their mind, a rejection that was cleared by a later approval. The bugs live where two rules touch.

Example tests don't find those well. You write the case you thought of. The case nobody thought of is the one that ships.

What we run

#

Every night a server generates random approval requirements: a random quorum from 1 to 4, random named approvers or none, one of the consensus modes (a number of people, every named person, the first decision wins, a percentage), sometimes abstaining allowed, sometimes "rejected only after two people say no", separation of duties on or off. Then it plays 30 random actions on each one. Approve, reject, abstain, revoke. Sometimes the click comes from a substitute acting for someone else.

Every action goes through the same permission check the app runs before it writes anything. After every single action the test checks that these still hold:

  • one person counts once toward the quorum, also when a substitute signed for them
  • nobody holds an approval and an active rejection at the same time, or an approval and an abstention
  • with named approvers, only those people decided anything
  • with separation of duties on, the author never approved or rejected, and a substitute never signed twice
  • "first decision wins" never ends up with two decisions
  • for a plain quorum, "approved" matches a count done the dumb way, outside the app's own code
  • an approval never turns an approved requirement back to open

That's a million requirements, 30 million actions, about two minutes on a small server. Each night starts from a different random seed. When something fails, the report gives the seed and the exact steps, so the case can be replayed on a laptop.

Does the test actually catch anything?

#

A test that never fails looks exactly like a test that checks nothing. So before trusting it we broke the app on purpose.

We switched off the rule that stops the author from approving their own requirement. The test failed straight away.

Then we switched off the check that ignores a second sign-off from the same person. The test did not fail. That looked bad for a minute. The reason: the permission check refuses the second click before the write ever happens, so the write-level guard is a second lock behind the first. Both have to break before a double vote gets through. I'm fine with that answer, but it's the kind of thing you only learn by trying.

The first real failure it ever reported was a bug in the test, not in the app: with "first decision wins" and a threshold of two rejections, a single rejection doesn't close the requirement. That's by design, and the test was wrong. So far it has not found a bug in the app. It has been running since 1 November 2026, which is not long.

What this doesn't cover

#

Groups and project roles are resolved live from Jira, and the test doesn't generate them yet. It also doesn't test the screens, only the rules underneath. And this suite runs against the next release; the version you have installed runs the regular test suite every night (about 1,250 tests).

If you run approvals in Jira today, which rule would you most want proven: separation of duties, the quorum, or what happens to old sign-offs when the requirement text changes?

#