A common requirement among Salesforce teams is being able to schedule Run All Tests daily in their Salesforce orgs.
There are free and commercial tools to do this, but I wasn't satisfied with any of them, so I built my own free and open-source using GitHub Actions, the
sf CLI and the Slack API.
This is available to you right now and it takes 5 min to get it working!
With this app, you'll get a slack notification every morning (or when you desire) letting you know that the tests runs for any number of orgs have finished, and whether the tests passed or failed, like this:
Before I show you how to set it up, let's talk about why anyone would want to do this in the first place. If you don't care about this and just want to go straight to the implementation, read this section.
In traditional software development, there's the concept of a nightly build, which ChatGPT explains as follows:
Translating this to Salesforce terms means running all tests in the org every night so that when we come to work in the morning, we can tell if any changes made the day before broke our tests (or "broke the build").
If the build is broken (or tests are failing), we know the first thing to do is to fix that. Ideally, all development would stop until the build is fixed (i.e. tests are passing again).
This is one of the core principles of DevOps: Nothing is more important than to keep the build stable.
Why on a schedule and not on demand
Ideally, Salesforce developers would run all tests when they commit changes to an integration branch as part of a Continuous Integration workflow.
However, in large Salesforce orgs, running all tests can easily take 4 hours or more! There are many things teams can do to reduce the test run times, but that's a whole different topic.
If you run all tests on a schedule, you can then run specified tests in your deployments. The idea is your deployments only run specific tests, while every night, all tests are run (assuming people aren't working, it's ok if it takes 4 hours).
Then, as I explained earlier, you'd have to check in the morning that all tests indeed pass before you continue developing new features.
Long story short, you get the best of both worlds: run all tests and keep the build stable, quick deployments with specified tests.
Why I built my own instead of using existing tools
First of all, I don't think this functionality alone is worth paying for. If this comes as part of an existing DevOps platform, then yes, it's definitely a nice addition!
However, most free and commercial tools in this domain are reinventing the wheel. They display code coverage results, lines of code that are covered/uncovered, percentage of coverage per class, etc.
Believe it or not, all of this is available for free in the developer console. I'm not going to cover this in detail, but here are some screenshots of how the developer console already solves these problems:
See? Why reinvent all this? I wanted to focus on the core problem, which I define as follows:
- I want the tests to run automatically
- I don't want to install anything on any org!
- I want this to scale to any number of orgs
- I want to be notified when the tests are done running, and I want to know if there are any failures
- I don't want to pay for this :)
I realised none of the solutions out there met the above criteria for me, so I had no choice but to build my own (poor me...).
Using sfdx and GitHub Actions
Let's break down the solution bit by bit
I'm going to assume you have a basic understanding of GitHub Actions. For this article, all you need to know is that I'm using GitHub actions as a scheduler, and as a run time to run
sf CLI commands against my orgs.
The sf CLI
First, to run the tests, I wanted to use the sf CLI, as opposed to using the Tooling API objects that expose test coverage information.
The reason is the sf CLI already uses these objects behind the scenes, so following the "don't reinvent the wheel" pattern, I wanted to reuse the existing functionality.
To run tests with the CLI, you can use the
apex run test command
sfdx apex run test --test-level RunLocalTests --result-format human
Because I built this with large orgs in mind where tests can take hours to run, I'm using the async version of this command, which returns a test run ID that I can use later to check the results. If I used the sync version of this and the tests take 4 hours to run, that's 4 hours of GitHub Actions processing...not something you want to do!
Two GitHub Actions workflows
One challenge is that if the run takes 4 hours, how and when do you get the results? The first command returns a test run ID, but I need to run a second command to get the results.
This brings certain challenges:
- Where do I store the test run ID?
- How do I run a scheduled action that gets the results?
- What if the test run isn't finished? do I fail, or retry?
- If this is meant to work for multiple orgs, how I know which test run ID belongs to which org?
I don't want to explain in detail how I solved this because that could be a whole article on its own, so I'll only give a few points:
- I'm using GitHub Actions Matrices to be able to scale this to many different orgs without having to create new actions.
- One job schedules the test run, and another checks the ID and sends the slack message. You have full control on how far the 2nd job should run after the first one
- I'm using GitHub Actions artifacts to upload a file with the test run ID. Then, the 2nd workflow queries this file and checks the test run.
- I'm using a GitHub Action to send slack messages.
There's a bit more too it. If you want to see the implementation, you can read the workflow files here.
Setting it up
Ok, after that wall of text, let's see how to actually set it up.
I admit it's not the easiest setup, but it's really a one time thing and then it'll work forever and scale to any number of orgs. So either you pay for a product, or spend a few min setting it up 😄
1- Fork the GitHub repo
Then, clone this to your local machine. Replace
pgonzaleznetwork with your GitHub username
git clone https://github.com/pgonzaleznetwork/salesforce-daily-unit-tests
2- Get the authentication URL for your orgs
Then, use the
sf CLI to get an sfdx authentication URL for one of your orgs, let's say production
sfdx force:org:display -u <OrgUsername> --verbose --json
You will get a result like this
Copy the value of
3- Create GitHub Secrets for each org
Now, enter this URL as a GitHub secret, with a name for your org, for example
Your secrets must be named SFDX_<some_short_name>_URL
Replace <some_short_name> with the name of your org. If you don't follow this pattern, nothing will work. You've been warned.
Repeat the process for all the orgs you want to run tests for. My short names are
4- Edit the workflow files with the orgs' shortnames
Open the project in vscode, and open the
There, replace the following section with the short names you used in your secrets
The must match exactly the name you gave it when creating the secret and they should be upper case, like the secret as well.
Now, open the other file
.github/workflows/matrix-check-test-runs.yml and do exactly the same
5- Specify when the tests should run
Now, we need to specify 2 things: a) when the job that requests the tests is run, and b) when the second job, which queries the test results, runs.
You have full flexibility here. In my sample scenario, the first job runs at 12 AM GMT, and the 2nd one, at 4 AM GMT. This is because I'm assuming it'll take at least 4 hours for the test run to complete.
In your case, you can specify one hour, two, or whatever makes sense for you.Let's do it.
Open the first file
.github/workflows/matrix-schedule-test-runs.yml and edit the
cron expression to be your desired time (ask ChatGPT to give you a cron expression for any time+timezone)
Now, edit the 2nd file
.github/workflows/matrix-check-test-runs.yml and edit the
cron expression to be the time you want the tests to be collected.
6- Create a Slack App and get the API token
To send the slack notification, I'm using this Slack action.
Follow the steps here to create a simple Slack app, which will provide you with an access token
Copy the access token and add it as a GitHub secret called
Then, figure out which channel you want to send messages to, and copy the channel ID
Paste the channel ID in the 2nd file, here
7- Commit changes and push to the repo
Commit all the changes and use
git push to push them to your remote repo.
You are done! See? It was that bad!
Seeing it in action
Now you have 2 options. Either you wait for the scheduled run, or you can run the jobs manually to see if they are working. To run them manually:
1- go to your repo > Actions and there you will see the 2 workflows
2- Click on Schedule Test Runs and click Run Workflow
3- You will now see the job in progress
If you click on the job, you'll see 2 jobs running, one per org. This is possible thanks to GitHub Actions Matrixs
The job is done when Salesforce returns a Test Run ID, not when the tests actually finish running. You will see the test run Id for each org as an artifact (just reload the page)
4- Now, assuming enough time has passed, you can run the 2nd job to collect the results and send the slack notification.
Remember, all of this is meant to happen automatically based on the schedules you configured; I'm only showing you how to run it manually so that you can confirm it works.
Go to the Check Test Runs and follow the same steps to run the job manually. If the tests runs have completed, and everything is working, you'll get a slack notification!
And that's it! Enjoy free Salesforce unit test automation!