Thanks!
You should be able to use also makisu without requiring privileged container, there isn’t any example since at the time of testing makisu was very young and kaniko (with its rough edges) worked better. We’ll try to create a makisu example when we have time or feel free to contribute it
I’ll just setup one or more “setup” tasks that will create and push the images. Then other “setup” tasks that will spun up the k8s environment (create a “temporary” namespace, with an unique id to make it unique to the run) and the various deployments/statefulsets etc… For doing this you can just continue to use keel or create the various deployments/daemonsets by yourself as you prefer calling the k8s api. The k8s api secrets can just be saved in an agola secret and exposed as a variable to the tasks (you could provide a full kubecfg file saving it base64 encoded in a secret).
Then create additional tasks that will depend on the “setup” tasks that will execute the tests.
As the final “cleanup”, we though about it many time. Currently you can create a “cleanup” task that will be called at the end of the run regardless how it’s final result. In your case you can just delete the k8s namespace and k8s will automatically delete all the resources. For doing this you should make this task depend on every other task both on_success and on_failure conditions. This isn’t a very clean approach and I opened a proposal to improve this here:
Another approach will be to externally delete the old k8s namespaces (i.e. a job that will delete namespace older than N hours or for example locking them using mechanisms like etcd locks).
Let us know what you think about this or propose alternative approaches.