Workspace usage, dir select beteen multi-flow tasks

General Discussions
1
  • save_to_workspace (source, dest, path)
  • restore_workspace (only dest_dir param)

TaskFlow

In one of my build as the code follows, there be 4 tasks: clone-> goBuild/nodeBuild -> dockerImg.

agola-task-flow-2019-11-04_233623
agola-task-flow-2019-11-04_2336231048×226 4.21 KB 

##clone task
steps: [
            { type: 'clone' }, 
            { type: 'save_to_workspace', contents: [{ source_dir: '.', dest_dir: './clone', paths: ['**'] }] },

##golang build
  steps: [
    { type: 'restore_workspace', dest_dir: '.' },
    ..
    { type: 'save_to_workspace', contents: [{ source_dir: './clone/dist', dest_dir: './target_gin', paths: ['**'] }] },

##nodejs build
  steps: [
    { type: 'restore_workspace', dest_dir: '.' },
    ...
    { type: 'save_to_workspace', contents: [{ source_dir: './clone/dist/binary', dest_dir: './target_ui', paths: ['ui/**'] }] },

##docker-img merge
          steps: [
            { type: 'restore_workspace', dest_dir: '.' },

WorkspaceUsage

I just found that restore_workspace copy all the workspace tar and then unTar. When some file/dir repeated, then failed in unTar.

To avoid this err I have to define diff topDir when save_to_workspace with clone/target_gin/target_ui by using dest_dir.

Suggest

We can find that the clone topDir in workspace isn’t used by the docker-img task, But still restored to the taskPod, cost more file transfer work and few build-time cost.

Can we just restore what we want to the building-container. Or any good ideas?

2

What’s more Suggest/Question:

  • Is there a way we delete some topDir or clean_workspace?

  • Whether step { type: 'clone' }, can use a local cache? (when git-repo is big or with a poor bandwidth between gitea and agola, that cost much time in each build)

Whats in my mind, In order to reUse the cloned repo:

  • 1.when first build, it not exist, just clone.
  • 2.then the repo cloned in local(building-volume/workspace) -> git fetch -> with the build branch: git rebase origin/master to avoid merge conflict. (push -f in some git branch).