Multi-stage builds execute CI in the last stage context instead of reverting to the job image.

[ivanchubb]

Actual behavior

When executing a multi-stage build (particularly evident when the last stage is FROM scratch) after the executor finishes, the rest of the script in the CI job maintains the context of the built image instead of reverting back to the context of the image defined at the job level. This causes unexpected errors when trying to do any parsing or log aggregation after running the exeuctor.

Expected behavior

Expected behavior is after the executor completes, the shell would return to the context of the image specified in the CI job.

To Reproduce
Steps to reproduce the behavior:

1. Create a multi-stage Dockerfile where the first stage builds the binary and the second stage is FROM scratch
2. Create a ci job with image: kaniko:1.23.2 where the executor is ran.
3. In the same job, after the executor runs, add to the script command that should exist in kaniko:1.23.2, but not in the final stage of the Dockerfile (for example: sed). The command will fail with "sed not found".

Additional Information

• Dockerfile.txt
• build-container.txt
• Kaniko Image (1.23.2)

Triage Notes for the Maintainers

Description	Yes/No
Please check if this a new feature you are proposing
Please check if the build works in docker but not in kaniko
Please check if this error is seen when you use --cache flag
Please check if your dockerfile is a multistage dockerfile

[erikdao]

Upvote for this report since I'm facing a very similar issue.

[mzihlmann]

could you be looking for the --cleanup flag?

https://github.com/GoogleContainerTools/kaniko#flag---cleanup

[ivanchubb]

could you be looking for the --cleanup flag?

https://github.com/GoogleContainerTools/kaniko#flag---cleanup

Just tried that and it doesn't help

[feifei325]

I have a similar problem. I observed that when using kaniko to build multiple levels,
Example command: /kaniko/executor --context dir:///{codePath} ...
When the Dockerfile contains the following command
COPY --from={baseImage} /{targetPath} After the COPY command is executed, {codePath} will be cleared. I am not sure why

After executing COPY --from=, execute ADD . .

[feifei325]

I have a similar problem. I observed that when using kaniko to build multiple levels, Example command: /kaniko/executor --context dir:///{codePath} ... When the Dockerfile contains the following command COPY --from={baseImage} /{targetPath} After the COPY command is executed, {codePath} will be cleared. I am not sure why

After executing COPY --from=, execute ADD . .

Use kaniko to package, put the code in /kaniko/{code_dir} and kaniko/executor --context dir:///kaniko/{code_dir} .... This can be executed normally. But I don't know why it can't be placed in other directories

[mzihlmann]

If you're still looking for a solution to this you could give my fork a try https://github.com/mzihlmann/kaniko/releases/
It fixes this issue and a few more, mostly related to caching, if you have other issues you would like to see resolved please let me know. I know that this is not ideal and I hope we can get the changes merged here eventually but for now that's the best I can offer.
If you like what you see you can support me with a star, thank you 🙇