Summary of Secrets: write-up best practices, do's and don'ts, roadmap · Issue #13490 · moby/moby

Summary Secrets: write-up best practices, do's and don'ts, roadmap · Issue #13490 · moby/moby · GitHub github.com

4,622 words - html page - View html page

One Line

The text explores the challenges and uncertainties of managing secrets in Docker, particularly through volume recreation during build steps.

Slides

Slide Presentation (6 slides)

Copy slides outline Copy embed code Download as Word

Secrets: Best Practices for Handling Secrets in Docker

Source: github.com - html - 4,622 words - view

The Challenges of Handling Secrets in Docker

• Docker discourages the use of insecure or non-designed features for handling secrets

• Security maintainers should provide guidance on handling secrets in Docker

• Uncertainties exist in the effectiveness of storing secrets in volumes

Leveraging Volume Recreation for Storing Secrets

• Some people have found ways to store secrets by leveraging volume recreation for each build step

• This method allows for the isolation of secrets within the container

• The effectiveness of storing secrets in volumes is uncertain

Build Time Secrets with Buildkit

• Build time secrets are now possible when using buildkit as the builder

• The RUN -mount option used for secrets will soon become the default Dockerfile syntax

• This enables greater control and security when handling secrets during the image building process

Seeking Feedback on Handling Secrets

• The author seeks feedback on their approach to handling secrets

• Their approach involves "unplugged" shared volumes to protect sensitive data

• The goal is to ensure no sensitive data is left on the host or in the container after the service starts

Best Practices for Handling Secrets in Docker

• Docker discourages insecure and non-designed features for handling secrets

• Leveraging volume recreation can provide isolation for storing secrets

• Buildkit enables build time secrets and improved security

• Seek feedback and consider "unplugged" shared volumes to protect sensitive data

Key Points

Docker discourages the use of insecure or non-designed features for handling secrets.
Some people have found ways to store secrets by leveraging the recreation of volumes for each build step.
The effectiveness of storing secrets in volumes is uncertain.
The author suggests that security maintainers should provide guidance on handling secrets in Docker.
Build time secrets are now possible when using buildkit as the builder.
The author seeks feedback on their approach to handling secrets, which involves "unplugged" shared volumes to protect sensitive data.

Summaries

26 word summary

The excerpt discusses handling secrets in Docker, including limitations and concerns. Storing secrets through volume recreation during build steps is mentioned, but its effectiveness is uncertain.

47 word summary

The excerpt discusses the handling of secrets in Docker and the current limitations and concerns with existing features. It mentions that some people have found ways to store secrets by leveraging volume recreation during build steps, but the effectiveness of this method is uncertain. The author suggests

258 word summary

Handling secrets in Docker is a recurring topic, with many pull-requests being hijacked by people wanting to use specific features for handling secrets. Currently, Docker discourages the use of these features because they are insecure or not designed for handling secrets. However

Some people have found ways to store secrets by taking advantage of the fact that volumes are re-created for each build step, but the effectiveness of this method is uncertain. Alternatively, containers can be manually built without using a Dockerfile and the results can be

In this excerpt from a GitHub issue, the author discusses the importance of considering different situations for run-time secrets and build-time secrets. They mention that they are not a security expert and suggest that the security maintainers should provide guidance on this topic. Another

The excerpted text discusses various aspects of using secrets in Docker image building and deployment. The first part of the text highlights the need for secrets during the image building process, such as passing an SSH key for source code checkout from a private GitHub repository.

Build time secrets are now possible when using buildkit as the builder, as mentioned by author thaJeztah on Sep 17, 2020. The RUN -mount option used for secrets will soon become the default Dockerfile syntax. Users

The author is seeking feedback on their approach to handling secrets in containers. They mention that their approach involves "unplugged" shared volumes to ensure that no sensitive data is left on the host or in the container after it is built and the service starts

Raw indexed text (30,067 chars / 4,622 words / 927 lines)

Secrets: write-up best practices, do's and don'ts, roadmap Issue #13490 moby/moby GitHub

Toggle navigation

Product

Actions

Automate any workflow

Packages

Host and manage packages

Security

Find and fix vulnerabilities

Codespaces

Instant dev environments

Copilot

Write better code with AI

Code review

Manage code changes

Issues

Plan and track work

Discussions

Collaborate outside of code

Explore

All features

Documentation

GitHub Skills

Blog

Solutions

For

Enterprise

Teams

Startups

Education

By Solution

CI/CD & Automation

DevOps

DevSecOps

Resources

Customer Stories

White papers, Ebooks, Webinars

Partners

Open Source

GitHub Sponsors

Fund open source developers

The ReadME Project

GitHub community articles

Repositories

Topics

Trending

Collections

Pricing

Search or jump to...

Search code, repositories, users, issues, pull requests...

Clear

Search syntax tips

Provide feedback

We read every piece of feedback, and take your input very seriously.

Include my email address so I can be contacted

Cancel

Submit feedback

Saved searches

Use saved searches to filter your results more quickly

Name

Query

To see all available qualifiers, see our

documentation

Cancel

Create saved search

You signed in with another tab or window.

Reload

to refresh your session.

You signed out in another tab or window.

Reload

to refresh your session.

You switched accounts on another tab or window.

Reload

to refresh your session.

Dismiss alert

moby

Public

Notifications

Fork

18.9k

Star

66.7k

Code

Issues

2.9k

Pull requests

357

Discussions

Actions

Projects

Wiki

Security

Insights

Code

Issues

Pull requests

Discussions

Actions

Projects

Wiki

Security

Insights

New issue

Have a question about this project?

Pick a username

Email Address

Password

By clicking Sign up for GitHub, you agree to our

and

privacy statement

. Well occasionally send you account related emails.

Already on GitHub?

to your account

Jump to bottom

Secrets: write-up best practices, do's and don'ts, roadmap

#13490

Open

thaJeztah

opened this issue

May 26, 2015

222 comments

Open

Secrets: write-up best practices, do's and don'ts, roadmap

#13490

thaJeztah

opened this issue

May 26, 2015

222 comments

Labels

area/security

status/needs-attention

Calls for a collective discussion during a review session

Comments

Copy link

Member

thaJeztah

commented

May 26, 2015

Handling secrets (passwords, keys and related) in Docker is a recurring topic. Many pull-requests have been 'hijacked' by people wanting to (mis)use a specific feature for handling secrets.

So far, we only

discourage

people to use those features, because they're either provenly insecure, or not designed for handling secrets, hence "possibly" insecure. We

don't

offer them real alternatives, at least, not for all situations and

, then without a practical example.

I just think "secrets" is something that has been left lingering for too long. This results in users (mis)using features that are not designed for this (with the side effect that discussions get polluted with feature requests in this area) and making them jump through hoops just to be able to work with secrets.

Features / hacks that are (mis)used for secrets

This list is probably incomplete, but worth a mention

Environment Variables

. Probably the most used, because it's part of the

"12 factor app"

. Environment variables are discouraged, because they are;

Accessible by any proces in the container, thus easily "leaked"

Preserved in intermediate layers of an image, and visible in docker inspect

Shared with any container linked to the container

Build-time environment variables

Proposal: Add support for build-time environment variables to the 'build' API

#9176

Support for passing build-time variables in build context

#15182

). The build-time environment variables were

not

designed to handle secrets. By lack of other options, people are planning to use them for this. To prevent giving the

impression

that they are suitable for secrets, it's been decided to deliberately

not

encrypt those variables in the process.

Mark .. Squash / Flatten layers

. (

flatten images - merge multiple layers into a single one

#332

Add MARK and SQUASH builder instructions

#12198

Add docker squash command

#4232

Adds build option to squash newly built layers

#9591

). Squashing layers will remove the intermediate layers from the final image, however, secrets used in those intermediate layers will still end up in the build cache.

Volumes

. IIRC some people were able to use the fact that volumes are re-created for each build-step, allowing them to store secrets. I'm not sure this actually works, and can't find the reference to how that's done.

Manually building containers

. Skip using a Dockerfile and manually build a container, commiting the results to an image

Custom Hacks

. For example, hosting secrets on a server,

curl

-ing the secrets and remove them afterwards, all in a single layer. (also see

https://github.com/dockito/vault

So, what's needed?

Add documentation on "do's" and "don'ts" when dealing with secrets;

@diogomonica

made some excellent points in

Proposal: Add support for build-time environment variables to the 'build' API

#9176 (comment)

Describe the officially "endorsed" / approved way to handle secrets, if possible, using the

current

features

Provide roadmap / design for officially handling secrets, we may want to make this pluggable, so that we don't have to re-invent the wheel and use existing offerings in this area, for example,

Vault

Keywiz

Sneaker

The above should be written / designed with both

build-time

and

run-time

secrets in mind

@calavera

created a quick-and-dirty proof-of-concept on how the new Volume-Drivers (

#13161

) could be used for this;

https://github.com/calavera/docker-volume-keywhiz-fs

Note:

Environment variables are used as the de-facto standard to pass configuration/settings,

including secrets

to containers. This includes official images on Docker Hub (e.g.

MySQL

WordPress

PostgreSQL

). These images should adopt the new 'best practices' when written/implemented.

In good tradition, here are some older proposals for handling secrets;

"Add private files support"

Add private files support

#5836

"Add secret store"

Add Secret store

#6075

"Continuation of the

docker secret

storage feature"

Continuation of the

docker secret

storage feature

#6697

"Proposal: The Docker Vault"

Proposal: The Docker Vault

#10310

The text was updated successfully, but these errors were encountered:

273

julie-sullivan, johann-petrak, theothermattm, 1beb, sneawo, JasonGhent, muhqu, webwurst, guneysus, torarnv, and 263 more reacted with thumbs up emoji

ianrose14, darkn3rd, athreyapatel, and vankejan reacted with hooray emoji

robhaswell, amingilani, nielssorensen, JorgeRdg, dasMulli, jergason, pramodhkp, toblerpwn, webyneter, cilindrox, and 21 more reacted with heart emoji

All reactions

273 reactions

4 reactions

31 reactions

thaJeztah

added

the

kind/security

label

May 26, 2015

Copy link

Member

Author

thaJeztah

commented

May 26, 2015

ping

@ewindisch

@diogomonica

@NathanMcCauley

This is just a quick write-up. Feel free to modify/update the description if you think that's nescessary :)

All reactions

Sorry, something went wrong.

Copy link

dreamcat4

commented

May 26, 2015

This is useful infos:

hashicorp/vault#165

As is this:

hashicorp/vault#164

All reactions

Sorry, something went wrong.

Copy link

Member

Author

thaJeztah

commented

May 26, 2015

@dreamcat4

there are some plans to implement a generic "secrets API", which would allow you to use either Vault, or Keywiz or you-name-it with Docker, but all in the same way. It's just an early thought, so it will require additional research.

All reactions

Sorry, something went wrong.

Copy link

dreamcat4

commented

May 27, 2015

@thaJeztah

Yep Sorry I don't want to detract from those efforts / discussion in any way. I am more thinking maybe it's a useful exercise also (as part of that longer process and while we are waiting) to see how far we can get right now. Then it shows up more clearly to others the limits and deficiencies in current process. What underlying is missing and needed the most to be added to improve the secrets.

Also it's worth considering about the different situations of run-time secrets VS build-time secrets. For which there is also an area overlap area.

And perhaps also (for docker) we may also be worth to consider limitations (pros/cons) between solutions that provide a mechanism to handle the secrets "in-memory". As opposed to a more heavily file-based secrets methods or network based ones e.g. local secrets server. Which are the current hacks on the table (until proper secrets API). This can help us to understand some of the unique value (for example of stronger security) added by a docker secrets API which could not otherwise be achieved by using hacks on top of the current docker feature set. However I am not a security expert. So I cannot really comment on those things with such a great certainty.

All reactions

Sorry, something went wrong.

Copy link

Member

Author

thaJeztah

commented

May 27, 2015

@dreamcat4

yes, you're right; for the short term, those links are indeed useful.

Also it's worth considering about the different situations of run-time secrets VS build-time secrets. For which there is also an area overlap area.

Thanks! I think I had that in my original description, must have gotten lost in the process. I will add a bullet

However I am not a security expert.

Neither am I, that's why I "pinged" the security maintainers; IMO, this should be something written by them

All reactions

Sorry, something went wrong.

Copy link

Contributor

diogomonica

commented

May 27, 2015

@thaJeztah

great summary. I'll try to poke at this whenever I find some time.

All reactions

Sorry, something went wrong.

thaJeztah

mentioned this issue

Jun 1, 2015

Proposal: The Docker Vault

#10310

Closed

Copy link

Member

Author

thaJeztah

commented

Jun 7, 2015

@diogomonica

although not

directly

related, there a long open feature request for forwarding SSH key agent during build;

#6396

given the number of comments, it would be good to give that some thinking too. (If even to take a decision on it whether or not it can/should be implemented)

All reactions

Sorry, something went wrong.

This was referenced

Jun 10, 2015

Managing secrets

docker/compose#1534

Closed

Moving a file from the host system to a container via docker cp

#5846

Closed

Copy link

ebuchman

commented

Jun 13, 2015

Assuming you could mount volumes as user other than root (I know it's impossible, but humour me), would that be a favourable approach to getting secrets into containers?

If so, I'd advocate for an alternative to

-v host_dir:image_dir

that expects the use of a data-only container and might look like

-vc host_dir:image_dir

(ie. volume-copy) wherein the contents of host_dir are copied into the image_dir volume on the data-only container.

We could then emphasize a

secure-data-only containers

paradigm and allow those volumes to be encrypted

All reactions

Sorry, something went wrong.

thaJeztah

mentioned this issue

Jul 10, 2015

Proposal: Machine Declaration

docker/machine#773

Open

thaJeztah

mentioned this issue

Jul 26, 2015

Highlight the security risk of setting the environment variable containing passwords

docker-library/mysql#88

Closed

ghost

mentioned this issue

Sep 4, 2015

[rancher-compose] env_file on hosts

rancher/rancher#1971

Closed

pwaller

mentioned this issue

Sep 4, 2015

Add support for encrypted environment variables

docker/compose#908

Closed

thaJeztah

mentioned this issue

Sep 8, 2015

Any php program in a linked container can read the mysql root password

MariaDB/mariadb-docker#21

Closed

jessfraz

added

area/security

and removed

kind/security

labels

Sep 8, 2015

hhorak

mentioned this issue

Sep 9, 2015

RFE: Better way to pass secret data into container

sclorg/mysql-container#91

Open

Copy link

kepkin

commented

Nov 13, 2015

I've recently read a good article about that from

@jrslv

where he propose to build a special docker image with secrets just to build your app, and than build another image for distribution using results from running build image.

So you have two Dockerfiles:

Dockerfile.build (here you simply copy all your secrets)

Dockerfile.dist (this one you will push to registry)

Now we can build our distribution like that:

!/bin/sh

docker build -t hello-world-build -f Dockerfile.build

docker run hello-world-build

build.tar.gz

docker build -t hello-world -f Dockerfile.dist ^

Your secrets are safe, as you never push

hello-world-build

image.

I recommend to read

@jrslv

article for more details

http://resources.codeship.com/ebooks/continuous-integration-continuous-delivery-with-docker

guneysus, playerjamesbattleground, roylee0704, johntellsall, zambon, koflerdavid, Vanuan, and papanito reacted with thumbs up emoji

All reactions

8 reactions

Sorry, something went wrong.

lamroger

mentioned this issue

Nov 13, 2015

Does a docker-squash remove the intermediate layers from the build cache?

jwilder/docker-squash#38

Closed

Copy link

lamroger

commented

Nov 13, 2015

Thanks for sharing

@kepkin

Just finished reading the article. Really concise!

I like the idea of exporting the files and loading them in through a separate Dockerfile. It feels like squashing without the "intermediate layers being in the build cache" issue.

However, I'm nervous that it'll complicate development and might require a third Dockerfile for simplicity.

All reactions

Sorry, something went wrong.

thaJeztah

mentioned this issue

Nov 17, 2015

Support for passing build-time variables in build context

#15182

Merged

blaggacao

mentioned this issue

Nov 22, 2015

Secret Management

rancher/rancher#1269

Closed

Copy link

Contributor

TomasTomecek

commented

Nov 24, 2015

@kepkin

no offense but that doesn't make any sense. Secrets are definitely not safe, since they are in the tarball and the tarball is being

ADD

ed to production image -- even if you remove the tarball, without squashing, it will leak in some layer.

Vanuan and ploh reacted with thumbs down emoji

All reactions

2 reactions

Sorry, something went wrong.

Copy link

Member

Author

thaJeztah

commented

Nov 24, 2015

@TomasTomecek

if I understand the example correctly, the tarball is

not

the image-layers, but just the binary that was built inside the build container. See for example;

https://github.com/docker-library/hello-world/blob/master/update.sh

(no secrets involved here, but just a simple example of a build container)

All reactions

Sorry, something went wrong.

Copy link

kepkin

commented

Nov 25, 2015

@TomasTomecek

I'm talking about secrets for building Docker image. For instance, you need to pass ssh key to checkout source code from your private GitHub repository. And the tarball contains only build artifacts but doesn't contain GitHub key.

All reactions

Sorry, something went wrong.

Copy link

Contributor

TomasTomecek

commented

Nov 25, 2015

@kepkin

right, now I read your post again and can see it. Sorry about that. Unfortunately it doesn't solve the issue when you need secrets during deployment/building the distribution image (e.g. fetching artifacts and authenticating with artifact service). But it's definitely a good solution for separation between build process and release process.

All reactions

Sorry, something went wrong.

Copy link

kepkin

commented

Nov 25, 2015

@TomasTomecek

that's exactly how I fetch artifacts actually.

In Docker.build image I download some binary dependencies from Amazon S3 image which require AWS key & secret. After retrieving and building, I create a tarball with everything I need.

All reactions

Sorry, something went wrong.

224 hidden items

Copy link

caub

commented

Aug 14, 2018

edited

The biggest pain point is secret rotations for me

you've to maintain a graph of secret to services dependencies, and update twice each service (to get back to original secret name)

listing secrets from services doesn't seem to be easy (I gave up after some attempts around

docker service inspect --format='{{.Spec.TaskTemplate.ContainerSpec.Secrets}}'

), listing services dependencies from

docker secret inspect

would be useful too. So I just maintain that (approximate) graph manually for now.

You also have to specify the secret destination, when it's not the default

/run/secrets/

in the docker service update command

I just hope for a simpler way to rotate secrets

All reactions

Sorry, something went wrong.

Copy link

BretFisher

commented

Aug 14, 2018

@caub

here's some CLI help:

Docker docs for formatting

help come up with the rest of your inspect format:

docker service inspect --format='{{range .Spec.TaskTemplate.ContainerSpec.Secrets}}{{println .SecretName}}{{end}}'

That'll list all secret names in a service. If you wanted both name and ID, you could:

docker service inspect --format='{{range .Spec.TaskTemplate.ContainerSpec.Secrets}}{{println .SecretName .SecretID}}{{end}}' nginx

I always have my CI/CD (service update commands) or stack files hardcode the path so you don't have that issue on rotation.

With labels you can have CI/CD automation identify the right secret if you're not using stack files (without needing the secret name, which would be different each time).

caub reacted with thumbs up emoji

All reactions

1 reaction

Sorry, something went wrong.

Copy link

Member

AkihiroSuda

commented

Nov 11, 2018

docker build --secret

is finally available in Docker 18.09

https://medium.com/@tonistiigi/build-secrets-and-ssh-forwarding-in-docker-18-09-ae8161d066

@thaJeztah

Are we ready to close this issue?

rob-mccann, PatrickShaw, ivorscott, mmoya, and pdemro reacted with hooray emoji

All reactions

5 reactions

Sorry, something went wrong.

Copy link

andriy-f

commented

Nov 11, 2018

For older versions of docker, using multistage build with copying secrets before build command is viable option, right?

FROM debian as build

COPY ./secret.conf /path/on/image/

RUN build.sh

...

FROM debian

COPY --from=build ...

All reactions

Sorry, something went wrong.

Copy link

Member

Author

thaJeztah

commented

Nov 11, 2018

@andriy-f

yes, that works, as long as you;

(obviously) don't copy the secret to the final stage , or:

use the

build

stage / stage in which a secret is present as a "parent" for the final image

never

push

the build-stage to a registry

trust the host on which your daemon runs; i.e. taking into account that your "build" stage is preserved as an image; someone with access to that image would be able to get access to your secret.

andriy-f, lucasbasquerotto, gmfx, mwschall, prasrvenkat, and mercutiodesign reacted with thumbs up emoji

All reactions

6 reactions

Sorry, something went wrong.

real34

mentioned this issue

Feb 26, 2019

Dploiement sur lhbergement final

lachouettecoop/participation#3

Closed

Copy link

Member

Author

thaJeztah

commented

Sep 17, 2020

build time secrets are now possible when using buildkit as builder; see the blog post here

https://medium.com/@tonistiigi/build-secrets-and-ssh-forwarding-in-docker-18-09-ae8161d066

and the documentation;

https://docs.docker.com/develop/develop-images/build_enhancements/

the

RUN --mount

option used for secrets will graduate to the default (stable) Dockerfile syntax soon

pdemro, hlobit, and TobiasWenzel reacted with hooray emoji

coip, Chili-Man, pdemro, bryanasdev000, and tsafs reacted with heart emoji

All reactions

3 reactions

5 reactions

Sorry, something went wrong.

Copy link

pdemro

commented

Sep 17, 2020

Thank you

@thaJeztah

I did just a little more digging and found that article shortly after posting (previous post is now deleted). Thanks again!

coip and Chili-Man reacted with thumbs up emoji

coip and Chili-Man reacted with hooray emoji

coip, Chili-Man, and thaJeztah reacted with heart emoji

All reactions

2 reactions

3 reactions

Sorry, something went wrong.

Copy link

Vanuan

commented

Sep 18, 2020

edited

Cool. That closes the build time secrets question. Anything for runtime/devtime (ssh in OS X)?

brybalicious reacted with thumbs up emoji

All reactions

1 reaction

Sorry, something went wrong.

Copy link

brybalicious

commented

Mar 4, 2021

edited

What is the current best-practice for run-time secrets to pass e.g. to an entrypoint script, without the need to set up some external service? Right now, it seems the build-time secret solution is easier than run-time. I could just mount a local

credentials.json

from the build context and parse it for any required build-time secrets. Run-time secrets, however are very confused, and there's no single idea of what constitutes best-practice.

All reactions

Sorry, something went wrong.

Copy link

DXist

commented

Mar 4, 2021

Best practice is to encrypt secrets with a public key and allow only your runtime controller to decrypt and mount them. Encrypted secrets can be stored in a Git repository.

For Kubernetes there is

https://github.com/bitnami-labs/sealed-secrets

brybalicious reacted with thumbs up emoji

All reactions

1 reaction

Sorry, something went wrong.

Copy link

brybalicious

commented

Mar 4, 2021

Best practice is to encrypt secrets with a public key and allow only your runtime controller to decrypt and mount them. Encrypted secrets can be stored in a Git repository.

For Kubernetes there is

https://github.com/bitnami-labs/sealed-secrets

Nice. What about in a simple setup that is not a cluster? Just a single Dockerfile..?

All reactions

Sorry, something went wrong.

Copy link

DXist

commented

Mar 5, 2021

edited

There are basic tools to store secrets in a repository like

https://git-secret.io/

Possible workflow:

Generate public/private gpg key pair during Docker image build.

Import gpg public key from the image, reencrypt secrets, using CI private key for decryption

mount encrypted keys, decrypt them in Docker entrypoint

All reactions

Sorry, something went wrong.

Copy link

binarytemple

commented

Mar 5, 2021

via email

edited by thaJeztah

Ive seen this pattern many times and wondered to myself, why would the software development team (who are busy installing npm and god knows what on their machines), be entrusted with runtime secrets such as the magic key that allows you to run up a $50k google bill overnight?

Surely this is a problem for the operations/security teams? Do they really need the application software developers handling this stuff?

eman1986 reacted with thumbs up emoji

All reactions

1 reaction

Sorry, something went wrong.

This comment has been minimized.

Copy link

jan-hudec

commented

Mar 5, 2021

Surely this is a problem for the operations/security teams?

Yes. They need the tools and documentation the most. When they deploy with docker, they need to inject a lot of secrets in them. And they often version the compose files, kubernetes manifests, helm charts, terraform manifests, ansible playbooks or whatever they use to deploy the containers in git, so storing encrypted bundles there is useful to them too.

Do they really need the application software developers handling this stuff?

Operations don't need them to, but developers themselves do. The build and test infrastructure also involves a bunch of secrets.

All reactions

Sorry, something went wrong.

Copy link

petef19

commented

Sep 19, 2021

edited

In our

docker-compose

file, we've been using

shared volumes

read-only

mode to pass secrets to the container that are needed for service configuration.

Once the container has successfully started and been configured, we then remove the shared files/folders on the host side. to leave no secrets on the host or in the container.

The container (and service running in it) has no issue with that as it only uses these secret files at start up.

Would love to get feedback about this approach. Anything wildly wrong with that ?

Thanks.

All reactions

Sorry, something went wrong.

Copy link

Contributor

lirantal

commented

Sep 19, 2021

@petef19

you might benefit from the bit of knowledge on mounting secrets using the new DOCKER_BUILDKIT support in Docker:

https://snyk.io/blog/10-best-practices-to-containerize-nodejs-web-applications-with-docker/

petef19 reacted with thumbs up emoji

All reactions

1 reaction

Sorry, something went wrong.

Copy link

petef19

commented

Sep 20, 2021

edited

@lirantal

thanks for the link, great read.

Regarding using

secrets

, whether the newer

build-secrets

or the

docker-compose-v3-secrets

, the downside with that approach is that you still leave sensitive data inside the container - and the secrets can be read and accessed just like plain text files. If an attacker manages to hijack a container, it's open season.

With our approach ("unplugged" shared volumes) - and please chime in with opinions - there is no sensitive data left on the host or in the container (after it was build and service started). But the drawback may be scalability, as we currently only run a couple of servers, so not stress tested in that regard.

All reactions

Sorry, something went wrong.

Copy link

Contributor

lirantal

commented

Sep 20, 2021

you still leave sensitive data inside the container

Well yes, but the point is that you mount secrets for a build stage that you don't actually use in the resulting image, and it's a nicer way of mounting data into that layer only.

All reactions

Sorry, something went wrong.

Copy link

petef19

commented

Sep 20, 2021

you still leave sensitive data inside the container

Well yes, but the point is that you mount secrets for a build stage that you don't actually use in the resulting image, and it's a nicer way of mounting data into that layer only.

right, but what if you need the sensitive data to configure the service when the final container starts ?

All reactions

Sorry, something went wrong.

Copy link

Member

Author

thaJeztah

commented

Sep 20, 2021

Correct, with the

build-secrets

, the secrets will not be stored in the image-layer. They will be mounted during build, and no longer there after the build-step completes (there may be an empty file afterwards, due to how Linux mount points work (they require the destination to exist), but the secret themselves are not there).

For

git

operations, you can also use the

--ssh

option when using BuildKit, which will forward an ssh-agent into the container (

https://docs.docker.com/develop/develop-images/build_enhancements/#using-ssh-to-access-private-data-in-builds

petef19 reacted with thumbs up emoji

All reactions

1 reaction

Sorry, something went wrong.

Copy link

Contributor

lirantal

commented

Sep 20, 2021

It looks like

@petef19

is seeking a run-time solution for when containers are running, not when they are built. To achieve that, there are the usual suspects:

Put secrets in source code

Pass environment variables

Mount a directory for the secrets

Use run-time API to fetch secrets (like say hashicorp vault and such)

** The above are organized from the least secure way of managing secrets to the most (my opinion only) **

All reactions

Sorry, something went wrong.

Copy link

petef19

commented

Sep 20, 2021

edited

It looks like

@petef19

is seeking a run-time solution for when containers are running, not when they are built. To achieve that, there are the usual suspects:

Put secrets in source code

Pass environment variables

Mount a directory for the secrets

Use run-time API to fetch secrets (like say hashicorp vault and such)

** The above are organized from the least secure way of managing secrets to the most (my opinion only) **

yes, runtime secrets to configure services as they start up. I can extend your list...

(5) use cmd args

(6) if the service allows to read secrest from a file, use that option

(7) use shell scripts to init the service

Out of these, (3), (6) and (7) can all be used with

shared volumes

that can be "unplugged" after the service has started successfully. leaving nothing behind on host or container.

Question is could an attacker that hijacks the container still get out through the "unplugged" shared volume, although the "tunnel" does not exist anymore.... ? Especially if all secret mounts are

read-only

and are direct

file mounts

(not directories), so even if the shared volume was not "unplugged", wouldn't the attacker need a

shared dir

in order to attempt to traverse up... ?

Re (4): this seems to be a secure approach and easily scalable, but since the API connection creds are in the container (which are a secret itself) couldn't a hijacker use those creds to query the API for secrets.... ?

All reactions

Sorry, something went wrong.

This comment has been minimized.

theahmadov

mentioned this issue

Jan 18, 2023

False Positive with Email Address

theahmadov/slash#2

Open

neersighted

mentioned this issue

May 30, 2023

[Feature Request] Runtime secrets

#45642

Closed

Copy link

lonix1

commented

May 31, 2023

edited

Since my issue

#45642

was closed and points to this one, I want to add that since

"runtime secrets"

are not possible, the recommendation is to use a one-node swarm to access swarm secrets. But this is not easy to do and maintain, and no longer works if we use docker rootless (no overlay network, so no swarm mode).

Now that

compose

is part of the main cli tool (

docker compose

), please please please give us runtime secrets. (Exactly the same as swarm runtime secrets, just without swarm.)

stoneclaims-jon, suntorytimed, mlalpho, hecd, henri, briandefiant, pkudrel, and timocov reacted with thumbs up emoji

All reactions

8 reactions

Sorry, something went wrong.

to join this conversation on GitHub

Already have an account?

Assignees

No one assigned

Labels

area/security

status/needs-attention

Calls for a collective discussion during a review session

Projects

None yet

Milestone

No milestone

Development

No branches or pull requests

99 participants

and others

Footer

2023 GitHub, Inc.

Footer navigation

Terms

Privacy

Security

Status

Docs

Contact GitHub

Pricing

API

Training

Blog

About

You cant perform that action at this time.