How to copy files into a Docker container without using the COPY command

Brice Fotzo
Towards Dev
Published in
5 min readJun 12, 2024

If you’ve ever written a Dockerfile, you’ve likely used the COPY command to transfer files and folders into your container.

While useful, there are times when you need to add files to a running container without stopping it or rebuilding the image. Let’s explore alternative methods for transferring files into your Docker container.

Copy of files from host to container

This guide is ideal for any Data Guy or Software Engineers that want to develop in a container or try and update code with various data without stoping their container.

Let’s go!

Prerequisites

Before starting, make sure you have:

Getting Started

Here are 3 methods to copy files into a running container without using the COPY command in your Dockerfile:

  1. Using docker cpcommand
  2. Using bind mounts
  3. Using wget or curl

Preparations

Before diving into the methods, let’s do some preparation:

  1. Download this csv data that we’ll use and save it to your workspace
  2. Run a FastAPI container where we’ll push our data and check their presence using a specific endpoint.

Pull the Docker image for tutorial

docker pull ghcr.io/bricefotzo/docker-init-example:main

Run the container

docker run -d --name my-api -p 8000:8000 ghcr.io/bricefotzo/docker-init-example:main

Navigating to the URL localhost:8000 in your browser, you’ll see this:

The 3 endpoints exposed by the image docker-init-example:main

The one we’ll use to check data is the second(/tree).

You can try this endpoint to check what is inside the folder of the app: app.

Listing of files and folders present in the folder /app

As you can see, listing the directory of the project(/app), we have 4 files and 1 folder. In the following methods, we’ll use different methods to check data availability.

Let’s start transferring files !

Method 1: Using docker cp Command

With my-apicontainer running, we’ll add data in it without stopping it or rebuilding the image.

The docker cp command allows you to copy files or directories between the host and a container's filesystem, even after the container is running.

How to use docker cp:

Usage:

docker cp [OPTIONS] CONTAINER:SRC_PATH DEST_PATH|-

docker cp [OPTIONS] SRC_PATH|- CONTAINER:DEST_PATH

Copy a file from host to container:

docker cp ./iris.csv my-api:/app
#Successfully copied 6.66kB to my-api:/app

Copy a directory from host to container:

docker cp ./data my-api:/app 
#Successfully copied 12.8kB to my-api:/app

As you can see, 2 items were added: iris.csv and data.

Files and directories after copying iris.csv and data

Checking in /app/data, we have 2 files: iris.csv and iris2.csv.

Files under the directory /app/data

Copy a file from container to host:

Let’s copy the file main.py into our host and let’s check what is inside.

docker cp my-api:/app/main.py ./
#Successfully copied 2.56kB to /Users/<username>/workspace/docker-init-example/data/./

When to use docker cp :

  • When you need to update files in a running container without rebuilding the image.
  • For temporary changes or debugging purposes.

Method 2: Using Bind Mounts

Bind mounts allow you to mount a directory from the host filesystem into the container. This method provides a direct mapping between the host and container, so changes in the host directory are immediately reflected in the container and vice versa.

How to use bind mounts:

Create a workspace on our host and add some files:

mkdir api-data
touch api-data/data1.csv
touch api-data/data2.csv

Start another API container with volume mounts:

docker run -d --name my-api2 -v ./api-data:/app/data/api -p 8001:8000 ghcr.io/bricefotzo/docker-init-example:main

The container is named my-api2 and the volume bind is done between the local folder api-data that we just created and the container folder /app/data/api.

Note: The port binding is changed to avoid conflict with the previous container. You can clean up the previous one and continue using the same container name and port if you want.

Let’s just add another file into the local directory api-data and check the content of /app/data/api.

touch api-data/data3.csv

Let’s check:

When to use bind mounts:

  • For development environments where you want to reflect code changes in real-time without rebuilding the image.
  • When you need persistent storage that directly maps to the host filesystem.

Method 3: Using wget or curl

Sometimes, it’s more efficient to download files directly from the internet into your container using tools like wgetor curl.

How to use wget or curl:

Download a file using docker exec and wget:

docker exec -it my-api2 /bin/sh -c "wget https://archive.ics.uci.edu/ml/machine-learning-databases/iris/iris.data -O /app/data/api/iris.csv"

Output:

--2024-06-12 18:29:52--  https://archive.ics.uci.edu/ml/machine-learning-databases/iris/iris.data
Resolving archive.ics.uci.edu (archive.ics.uci.edu)... 128.195.10.252
Connecting to archive.ics.uci.edu (archive.ics.uci.edu)|128.195.10.252|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: unspecified
Saving to: '/app/data/api/iris.csv'

/app/data/api/iris.csv [ <=> ] 4.44K --.-KB/s in 0.001s

2024-06-12 18:29:54 (3.36 MB/s) - '/app/data/api/iris.csv' saved [4551]

Download a file using docker exec and curl:

docker exec -it my-api2 /bin/sh -c "curl -o /app/data/api/iris2.csv https://archive.ics.uci.edu/ml/machine-learning-databases/iris/iris.data"

Output:

% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
Dload Upload Total Spent Left Speed
100 4551 0 4551 0 0 8538 0 --:--:-- --:--:-- --:--:-- 8586

Then let’s check if the data have been downloaded in the target folder.

Both of iris.csv and iris2.csv are present in the folder

When to use wget or curl:

  • When you need to download files from a remote server directly into the container.
  • For small files or configuration scripts needed at runtime.

Conclusion

While the COPY command in a Dockerfile is useful for static file transfers during image build time, these alternative methods offer greater flexibility for various scenarios:

  • docker cp for on-the-fly file transfers.
  • Bind mounts for real-time development and persistent storage.
  • wget or curl for downloading files directly into the container.

By utilizing these methods, you can manage file transfers more effectively in your Docker workflows, making your tasks as a data engineer more efficient and streamlined.

Let’s connect to continue the discussion!

Sign up to discover human stories that deepen your understanding of the world.

Free

Distraction-free reading. No ads.

Organize your knowledge with lists and highlights.

Tell your story. Find your audience.

Membership

Read member-only stories

Support writers you read most

Earn money for your writing

Listen to audio narrations

Read offline with the Medium app

--

--

Published in Towards Dev

A publication for sharing projects, ideas, codes, and new theories.

No responses yet

What are your thoughts?