Kubernetes hosted tests

[karlmutch]

@sgotti

I’ll just setup one or more “setup” tasks that will create and push the images. Then other “setup” tasks that will spun up the k8s environment (create a “temporary” namespace, with an unique id to make it unique to the run) and the various deployments/statefulsets etc… For doing this you can just continue to use keel or create the various deployments/statefulsets/services etc… by yourself as you prefer calling the k8s api. The k8s api secrets can just be saved in an agola secret and exposed as a variable to the tasks (you could provide a full kubecfg file saving it base64 encoded in a secret).

Then create additional tasks that will depend on the “setup” tasks that will execute the tests.

As the final “cleanup”, we though about it many time. Currently you can create a “cleanup” task that will be called at the end of the run regardless how it’s final result. In your case you can just delete the k8s namespace and k8s will automatically delete all the resources. For doing this you should make this task depend on every other task both on_success and on_failure conditions. This isn’t a very clean approach and I opened a proposal to improve this here:

https://github.com/agola-io/agola/issues/5

Another approach will be to externally delete the old k8s namespaces (i.e. a job that will delete namespace older than N hours or for example locking them using mechanisms like etcd locks).

Let us know what you think about this or propose alternative approaches.

My sense is that using the Kubernetes API might be the best approach for initiation of the tests. Adding keel.sh is complicating the solution so I am looking for ways to avoid it. I am looking to use agola to reduce the variety of technologies a consumer of open source needs in order to successfully use CI/CD.

I would then have a step that waits for the test driver pod inside the test namespace to exit and then scrape the logs from the test driver pod and incorporate them as output inside the agola pod output. Finally deleting the test namespace as an additional final step.

As time permits I’ll try to prototype this approach.

Thanks again,
Karl