To the people running gitlab-runners: Please upgrade to 13.12 to avoid future incompatibilities when we migrate to GitLab 14

GitLab tests in the Continuous Integration (CI) context

Test suite parallelization on the CI

Our current CI parallelization setup is as follows:

  1. The retrieve-tests-metadata job in the prepare stage ensures we have a knapsack/report-master.json file:
    • The knapsack/report-master.json file is fetched from the latest master pipeline which runs update-tests-metadata (for now it's the 2-hourly scheduled master pipeline), if it's not here we initialize the file with {}.
  2. Each [rspec|rspec-ee] [unit|integration|system|geo] n m job are run with knapsack rspec and should have an evenly distributed share of tests:
    • It works because the jobs have access to the knapsack/report-master.json since the "artifacts from all previous stages are passed by default".
    • the jobs set their own report path to "knapsack/${TEST_TOOL}_${TEST_LEVEL}_${DATABASE}_${CI_NODE_INDEX}_${CI_NODE_TOTAL}_report.json".
    • if knapsack is doing its job, test files that are run should be listed under Report specs, not under Leftover specs.
  3. The update-tests-metadata job (which only runs on scheduled pipelines for the canonical project takes all the knapsack/rspec*_pg_*.json files and merge them all together into a single knapsack/report-master.json file that is saved as artifact.

After that, the next pipeline uses the up-to-date knapsack/report-master.json file.

Monitoring

The GitLab test suite is monitored for the master branch, and any branch that includes rspec-profile in their name.

CI setup

  • Rails logging to log/test.log is disabled by default in CI for performance reasons. To override this setting, provide the RAILS_ENABLE_TEST_LOG environment variable.

Return to Testing documentation