Hello World: a Tutorial series with C++, Docker, and Ubuntu.

28 Jul 2018 c-plus-plus docker tutorials ubuntu

The end goal is of this tutorial is to release C++ code developed in Ubuntu – and currently on Github – in Docker images, with all of the required libraries, such that others can run, evaluate, and use it. To get there, well, that took a while.

This guide is assembled from my own notes as I was learning Docker. I most frequently program in C++ on Ubuntu, with OpenCV, OpenMP, Eigen, and other libraries. I give the information from the tutorial in the text, and you can also get the examples as directories from Github, here, or

git clone https://github.com/amy-tabb/docker-tutorial-files.git

First, of all, I’m using

Docker version 18.06.0-ce, build 0ffa825

Resources

Good resources are available from the Docker website, my personal favorites being Getting started and Build the app. The remainder of this document assumes that you have read and tried the examples on these two pages. Oh! And that you have a version of Docker installed. After a half-hearted attempt to install the latest from Github, I installed Docker CE for Ubuntu – painless on Ubuntu 16.04 (2x).

Later, the best practices page can be useful. Vladislav’s site is also good, though I discovered it too late in this particular quest.

To break this task into smaller sections, this tutorial has the following structure:

First, I’ll mention some notes that turned out to be useful deugging tools. If you haven’t touched Docker yet, though, you are safe to skip them at the present time.

Already frustrated people, tips from the trenches. Newbies, skip this section.

This section contains some items I wished that I had known earlier. However, it might be confusing if you’re only first starting to use Docker – hence the title.

This tutorial is offered exclusively from the Dockerfile perspective, because that is needed for automated builds (more on that later). However, for debugging purposes, I have found it sometimes useful to work out all the kinks by running an image:

docker run -it ubuntu bash

And then you’re in bash in ubuntu, or whatever other image you have selected (could be your own!). Then execute your commands using the command line. To save as a new image, you can commit (Note, you need to open a new bash shell to get the image id).

If you have a slow internet connection or large images, docker pull, push, and build (which includes pull) will stall out. You may get some unhelpful hints from the docker command about how fix that – but to be clear, first you need to kill the Docker daemon dockerd. If you installed Docker from the method above and haven’t restarted, you didn’t start this deamon yourself, so you’ll have to find it the usual way:

$ ps aux | grep dockerd
atabb     1537  0.0  0.0  15948   932 pts/27   R+   09:34   0:00 grep --color=auto dockerd
root     38974  0.0  0.0  75396  2172 pts/18   S+   Jul26   0:00 sudo dockerd
root     38975  0.5  0.0 3302072 102404 pts/18 Sl+  Jul26   5:59 dockerd

(dockerd is the Docker daemon). I’m including all the steps for all the different levels of users.

Grab the id of the dockerd, sudo kill 38975.

Then restart the daemon with some parameters that will work better for your setting ( Docker daemon doc. ). For instance:

$ dockerd --max-concurrent-downloads 1   --max-concurrent-uploads 1

would change the layers downloaded and uploaded at a time from 5 (default) to 1. This still doesn’t solve all problems for big images – that’s handled to some extent on Page 3, but it may solve some.

Finally, pulling images and building lots of images can take a lot of hard drive space. Some tips for dealing with that are here, appropriately titled “Keeping the whale happy …”.

C++ Program

First, we’ll start with the goal of getting a C++ program to compile and run in Docker. To do so, we’ll use the familiar `Hello World’ template, but then gradually add complexity that will mirror that of the final goal application: passing arguments, assessing the host file system, and writing data to the host file system.

Hello World

Navigate to DockerHelloWorldProject0 in the command line from the Github repository I mentioned above, which contains file Dockerfile and folder HelloWorld. The Dockerfile looks like this:

FROM amytabb/docker_ubuntu16_essentials
COPY HelloWorld /HelloWorld
WORKDIR /HelloWorld/
RUN g++ -o HelloWorld helloworld.cpp
CMD ["./HelloWorld"]

The C++ file, helloworld.cpp is straightforward:

#include <iostream>
using namespace std;
     
int main()
{
  cout << "Hello world 0!" << endl;
  return 0;
}

FROM creates a layer from the amytabb/docker_ubuntu16_essentials image.
COPY adds the local folder HelloWorld to the Docker image’s directory structure
WORKDIR changes the directory to /HelloWorld/. RUN cd HelloWorld does not work, not only because I have tried it, but because at every RUN, you are adding a new layer, which happens at the / level of the image. More info here. I also have a dedicated page about WORKDIR.
RUN ... compiles the .cpp file into an executable.
CMD runs/executes the new executible only when we use docker to run an image, when building, it sets up this expected behavior.

The standard order of operations is to then try to build. On certain connections, the download of the larger layers will time out during the build. I get around this problem by pulling them first:

docker pull amytabb/docker_ubuntu16_essentials

And then, in the directory DockerHelloWorldProject0, try to build with Docker:

$ docker build -t hello0 .

By the way, for this example we could substitute gcc:4.9 for my docker container amytabb/docker_ubuntu16_essentials; but we’ll use the latter container in the other examples. Once it has been pulled to the local machine, docker will use it throughout the session, so we don’t need to download it again. My result looks like:

$ sudo docker build -t hello0 .
Sending build context to Docker daemon  5.632kB
Step 1/5 : FROM amytabb/docker_ubuntu16_essentials
 ---> 8c0f518d0b72
Step 2/5 : COPY HelloWorld /HelloWorld
 ---> Using cache
 ---> 8af16ac6ec94
Step 3/5 : WORKDIR /HelloWorld/
 ---> Running in bc718f92ef92
Removing intermediate container bc718f92ef92
 ---> d60afcf83759
Step 4/5 : RUN g++ -o HelloWorld helloworld.cpp
 ---> Running in 849f5b06d8c7
Removing intermediate container 849f5b06d8c7
 ---> 74ddc7fb4a2a
Step 5/5 : CMD ["./HelloWorld"]
 ---> Running in 33a0ebe24e99
Removing intermediate container 33a0ebe24e99
 ---> 14fe6de84535
Successfully built 14fe6de84535
Successfully tagged hello0:latest

Then, the moment of truth! Try to run:

$ docker run -it hello0
Hello world 0!

Hello World With Arguments

Next challenge: Sending arguments to our simple hello world program. I’ll use the environment variable method sent through explicit -e flags, described in detail here. There are alternate methods, that work better for lots of variables.

Navigating to the next folder in the directory, DockerHelloWorldProject1, we have a Dockerfile, shell script, and a folder with the .cpp file in it.

Dockerfile

First, the Dockerfile:

FROM amytabb/docker_ubuntu16_essentials
ENV NAME VAR1
ENV NAME VAR2
ENV NAME VAR3
COPY run_hello1.sh /run_hello1.sh
COPY HelloWorld /HelloWorld
WORKDIR /HelloWorld/
RUN g++ -o HelloWorld1 helloworld1.cpp
WORKDIR /
CMD ["/bin/sh", "/run_hello1.sh"]

FROM creates a layer from the amytabb/docker_ubuntu16_essentials image.
ENV NAME specifies the environment variables that may be passed from the command line when the container is run. In this example, we have three arguments maximum. If the arguments are not specified, they are not passed to the C++ program, but more on that later.
COPY copies files from the host to the image; syntax is host -> image. Unless we specifically place it in the image, it will not magically get there.
WORKDIR changes the directory to /HelloWorld/ in the image.
RUN ... compiles the .cpp file into an executable.
CMD runs/executes the new executible using a shell script, which interprets the environment varibles and sends them to the C++ program as arguments.

Shell script

Pretty simple: we’re just packaging everything together so that the C++ program can receive it. To make things easier, we wouldn’t have navigated into and out of the HelloWorld directory. However, I did so since the final goal application will require doing so, and I would rather fail early on the easy stuff.

#!/bin/sh
./HelloWorld/HelloWorld1 $VAR1 $VAR2 $VAR3

C++ program

#include <iostream>
#include <string>
using namespace std;
     
int main(int argc, char **argv)
{
  cout << "Hello world 1, with arguments!" << endl;

  string val;
  for (int i = 1; i < argc; i++){
	val = argv[i];
        cout << "Argument " << i << " " << val << endl;
  }
  return 0;
}

Build and run

Choose a different tag, and build:

docker build -t hello1 .

Then, it is time to run and play around with arguments:

$ docker run -it -e VAR1='23' hello1 
Hello world 1, with arguments!
Argument 1 23

If we send no arguments, that’s no problem either:

$ docker run -it  hello1 
Hello world 1, with arguments!

What about arguments we didn’t specify in the Dockerfile, in other words, the user messes up?

$ docker run -it -e VAR1='23' -e VAR2='12' -e VAR3='10000' -e MYSTERY='blah' hello1 
Hello world 1, with arguments!
Argument 1 23
Argument 2 12
Argument 3 10000

Hello World With Args And Access Host

I chose to bindmount a folder on the host to a folder in the image. From the way I have written the Dockerfile, the container’s folder HAS to be /write_directory. The source folder on the host is where the data will be read and written. While in my own programs I remove and create folders of results with system calls – otherwise, there’s just too much accumulated junk in the development/debugging stage when I am working on algorithms – for releases I do not in case a user doesn’t read the fine print. Details of bindmounting and other types of mounting are here.

Dockerfile

First, the Dockerfile:

FROM amytabb/docker_ubuntu16_essentials
ENV NAME VAR1
ENV NAME VAR2
ENV NAME VAR3
RUN mkdir /write_directory
ARG DIRECTORY=/write_directory
ENV VAR_DIR=$DIRECTORY
COPY run_hello2.sh /run_hello2.sh
COPY HelloWorld /HelloWorld
WORKDIR /HelloWorld/
RUN g++ -o HelloWorld2 helloworld2.cpp
WORKDIR /
CMD ["/bin/sh", "/run_hello2.sh"]

Many things are similar from the previous two sections, so I will only cover the new sections.

ARG is a way to specify build-type variables, which you can then copy over to environment varibles. Thanks to vsupalov’s site – here and here specifically – go there for more details.

Shell script

In this case, the first argument for the program is the directory of the Docker container. I have chosen this order because I want it to be that way, not for any other reason.

#!/bin/sh

./HelloWorld/HelloWorld2 $VAR_DIR $VAR1 $VAR2 $VAR3

C++ program

This program is similar to the others, except that we want some evidence that the bindmounting worked – or didn’t. So this program opens a new file in the directory – in the Docker image, which is bound to the directory on the host – and writes some text, and then closes the file.

#include <iostream>
#include <string>
#include <fstream> 
using namespace std;
     
int main(int argc, char **argv)
{
  cout << "Hello world 2, with a directory and arguments!" << endl;
  
  ofstream out;
  string val;
  if (argc >= 2){
	val = argv[1];
	cout << "Directory is : " << val << endl;
       
        string filename = val + "/test_write.txt";
	out.open(filename.c_str());
	out << "HELLO WORLD FROM A BINDMOUNT!" << endl;
	 for (int i = 2; i < argc; i++){
		val = argv[i];
        	cout << "Argument " << i << " " << val << endl;
		out << "Argument " << i << " " << val << endl;
  	}
	out.close();	
  } else

 
  return 0;
}

Build and run

By now, you know the drill. Within the directory, build.

$ docker build -t hello2 .

Now, we’ll try bindmounting! Remember, the image’s folder HAS to be /write_directory, so I’ll try

docker run -it -v /home/atabb/docker/HelloWorldMount:/write_directory -e VAR1=15 hello2 
Hello world 2, with a directory and arguments!
Directory is : /write_directory
Argument 2 15

And the contents of my host folder has

HelloWorldMount]$ ls -l
total 4
-rw-r--r-- 1 root root 44         test_write.txt

So the file’s there, and it has the expected text:

HELLO WORLD FROM A BINDMOUNT!
Argument 2 15

Notice, that the owner of test_write.txt is root – or else the docker group if you have configured docker that way.

What if we run without any directory specified, or the user forgets? The Docker container doesn’t die, a win, but no data is written to the host folder.

$ docker run -it -e VAR1=15 hello2 Hello world 2, with a directory and arguments!
Directory is : /write_directory
Argument 2 15

Another variation, what if the user binds to the wrong folder?

$ docker run -it -v /home/atabb/docker/HelloWorldMount:/write_dir -e VAR1=25 hello2

Here, the user selected write_dir instead of write_directory. The host folder HelloWorldMount was bound to a non-existent write_dir, and so no data was written. In this toy example, nothing broke. However, depending on the application, you should be sure to test whether the files you need to read are in the directory, and exit the C++ code gracefully, which of course you do anyway. In my test, if HelloWorldMount does not exist, it is created when running docker run ..... Of course, it will be empty and be owned by root (or someone else, depending on how you set up Docker).

My favorite for testing whether files are present in C++:

ifstream in;
string filename = read_directory + "number_cameras.txt";
in.open(filename.c_str());
if (!in.good()){
	cout << "Input file is bad for number cameras -- abort." << filename << endl;
	exit(1);
}

Amy Tabb

Hello World: a Tutorial series with C++, Docker, and Ubuntu.

Resources

Contents

Already frustrated people, tips from the trenches. Newbies, skip this section.

C++ Program

Hello World

Hello World With Arguments

Dockerfile

Shell script

C++ program

Build and run

Hello World With Args And Access Host

Dockerfile

Shell script

C++ program

Build and run

Share or discuss.

Amy Tabb

Hello World: a Tutorial series with C++, Docker, and Ubuntu.

Resources

Contents

Already frustrated people, tips from the trenches. Newbies, skip this section.

C++ Program

Hello World

Hello World With Arguments

Dockerfile

Shell script

C++ program

Build and run

Hello World With Args And Access Host

Dockerfile

Shell script

C++ program

Build and run

Share or discuss.

Related Posts

CMake: conditional and logical expression syntax. 06 Dec 2022

CMake: How to force reconfiguration. 29 Nov 2022

CMake: How to specify the compiler in CMakeLists.txt. 28 Nov 2022

Mixing C and C++ in one file. 27 Nov 2022

CMake: create a list and iterate over it to set SOURCE properties. 01 Nov 2022