Week 4 · Day 16 · Docker Starts

Docker Fundamentals

What containers actually are under the hood — namespaces, cgroups, Union FS. Images vs containers. Docker architecture. The lifecycle. Week 4 starts here: understand what you've been running.

⏱ Duration 60 min

📖 Theory 30 min

🔧 Lab 25 min

❓ Quiz 5 min

The Week 3 → Week 4 Bridge

Week 3: you ran containers in CI without knowing how they work. Week 4: you understand the kernel primitives underneath. Containers are processes, not VMs.

Session Overview

What we cover today

01

Containers vs VMs

Not just "lighter VMs." Fundamentally different. Containers share the host kernel. VMs have their own OS.

02

Linux Namespaces

PID, NET, MNT, UTS, IPC, USER. How Docker isolates processes so they can't see each other.

03

cgroups — Resource Limits

How --memory 512m and --cpus 1 actually work. CPU throttling, OOM killer.

04

Union Filesystem & Image Layers

OverlayFS: read-only layers stacked + writable container layer. Why docker pull is fast.

05

Images vs Containers

Image = read-only blueprint. Container = running instance + writable layer. N containers from 1 image.

06

Docker Architecture

CLI → Docker Daemon (dockerd) → containerd → runc. How a docker run command executes.

07

🔧 Lab — Docker Hands-On

Pull, run, exec, inspect layers, volumes, lifecycle. See namespaces and cgroups in action.

Part 1 of 5

Containers vs VMs — fundamentally different

Key differences

Feature	VM	Container
OS	Full Guest OS	Shared host kernel
Size	1–40 GB	5 MB–500 MB
Startup	30–60 sec	<1 sec
Isolation	Full (hardware)	Process (namespace)
Overhead	High	Near-zero

The critical insight

A container is just a process running on the host OS with a restricted view of the system. It's not a mini-VM. The kernel is shared. Namespaces create the illusion of isolation. cgroups create the illusion of dedicated resources.

Part 2 of 5

Linux Namespaces — the isolation mechanism

6 Linux namespaces Docker uses

PID — process isolation. Container sees its own PID 1. Host sees actual PID 4521.
NET — network isolation. Each container gets its own eth0, IP address, port space.
MNT — filesystem isolation. Container sees its own / (the image layers). Cannot see host filesystem.
UTS — hostname isolation. Container has its own hostname.
IPC — inter-process communication isolation.
USER — user ID isolation. root inside container ≠ root on host (with user namespaces).

💡 See namespaces yourself

docker run -d --name test nginx:alpine
docker exec test ps aux — sees PID 1
ps aux | grep nginx — host sees real PID

Part 3 of 5

cgroups — resource limits that make docker run --memory work

cgroups in docker run

# Set memory limit — OOM killed if exceeded
docker run -d \
  --name limited-app \
  --memory 512m \
  --memory-swap 512m \   # no swap
  --cpus 1.5 \           # 1.5 CPU cores max
  --cpu-shares 512 \     # relative weight (default 1024)
  nginx:alpine

# Verify limits are applied
docker inspect limited-app | grep -A5 '"Memory"'
# "Memory": 536870912   ← 512 * 1024 * 1024 bytes

# See live resource usage
docker stats limited-app
# CONTAINER   CPU %   MEM USAGE / LIMIT
# limited-app  0.1%    8MiB / 512MiB

# See cgroup files directly (Linux only)
cat /sys/fs/cgroup/memory/docker/<container-id>/memory.limit_in_bytes
# 536870912 ← Docker wrote this file!

# In Kubernetes — same mechanism:
# resources:
#   limits:
#     memory: "512Mi"
#     cpu: "1500m"
# → K8s translates these to cgroup files

What cgroups control

CPU — --cpus 1.5: max 1.5 cores. --cpu-shares 512: relative weight when competing for CPU.
Memory — --memory 512m: hard limit. Process is OOM-killed if exceeded.
Block I/O — limit disk read/write bandwidth per container.
Network — limit bandwidth (via tc/netem, not pure cgroups).

cgroups = files in /sys/fs/cgroup/. Docker writes these files when you do docker run --memory. The kernel reads them to enforce limits.

Namespaces + cgroups = containers

Namespaces give containers an isolated view of the system (can't see each other).
cgroups give containers limited resources (can't consume everything).

Together they create the container abstraction. Docker is a user-friendly wrapper around these two Linux kernel features.

⚠ Always set memory limits in production

Without --memory, a container can consume all host RAM, starving other containers. In Kubernetes: always set resources.limits.memory. Without it, a single runaway pod can OOM the entire node.

Part 4 of 5

Union Filesystem & Layers — how images are stored efficiently

Why layers matter

Shared layers: If 10 containers use node:20-alpine, the base layer is stored only once on disk.
Fast pulls: docker pull only downloads layers you don't have. Already have layer 1+2? Only layer 3+4 download.
Layer cache in builds: Unchanged layers are reused. This is why Dockerfile instruction order is critical.
CoW (Copy-on-Write): When a container modifies a file from a read-only layer, the file is copied to the writable layer first.

Inspect layers

# See image layers
docker history node:20-alpine
docker history my-devops-app:local

# See layer sizes
docker image inspect my-devops-app:local \
  --format '{{json .RootFS.Layers}}'

# Check disk usage
docker system df         # total Docker disk usage
docker system df -v      # verbose — per image/container

Part 4 of 5 — continued

Images vs Containers — blueprint vs running instance

Images and containers

# === Images — read-only blueprints ===
docker images                     # list all local images
docker pull nginx:alpine           # pull from registry
docker image inspect nginx:alpine  # full metadata
docker history nginx:alpine        # show layers
docker rmi nginx:alpine            # remove image
docker image prune                 # remove dangling images

# One image → multiple containers
docker run -d -p 8080:80 --name web-1 nginx:alpine
docker run -d -p 8081:80 --name web-2 nginx:alpine
docker run -d -p 8082:80 --name web-3 nginx:alpine
# 3 containers, 1 image, ~zero extra disk space

# === Containers — running instances ===
docker ps                          # running containers
docker ps -a                       # all (inc. stopped)
docker ps --format "table {{.Names}}\t{{.Status}}\t{{.Ports}}"

# Container lifecycle
docker stop web-1                  # SIGTERM → SIGKILL after 10s
docker start web-1                 # restart
docker restart web-1               # stop + start
docker pause web-1                 # freeze (cgroup freezer)
docker unpause web-1               # unfreeze
docker kill web-1                  # immediate SIGKILL
docker rm web-1                    # delete (must be stopped)
docker rm -f web-1                 # force delete (stop+rm)
docker rm $(docker ps -aq)        # remove all stopped

The class analogy

Image = recipe. Read-only. You don't eat the recipe.
Container = meal cooked from the recipe. Each time you cook (docker run), you get a fresh meal (container). Many meals, one recipe. Discard the meal (docker rm), recipe is untouched.

Changes made inside a container do NOT modify the image.

Container states

created → running → paused
                ↑          ↓
              start      pause
                ↓          ↓
            stopped   ← unpause
                ↓
             removed

Stopped containers still exist on disk (docker ps -a). They consume disk space but not CPU/memory. Always docker rm containers you no longer need.

Part 5 of 5

Docker Architecture — what happens when you type docker run

The component chain

Docker CLI — what you type. Sends REST API calls to the daemon via a Unix socket (/var/run/docker.sock).
dockerd — the Docker daemon. Manages the high-level features: image caching, networking, volumes, registry auth.
containerd — the container runtime. Manages the container lifecycle, image layers (snapshots), and delegates to runc. Used directly by Kubernetes.
runc — OCI-compliant runtime. The piece that actually calls clone() and unshare() system calls to create namespaces and cgroups.

ℹ Kubernetes dropped Docker in 1.24

Kubernetes 1.24 removed the "dockershim" layer. Now K8s talks directly to containerd (or CRI-O) via CRI. Your Docker images still work — they're just OCI images. The Dockerfile you wrote in Week 3 works in both.

Hands-On Lab

🔧 Docker Hands-On

Pull, run, exec, inspect layers, volumes, container lifecycle — see namespaces and cgroups in action

⏱ 25 minutes

Docker Desktop running ✓

🔧 Lab — Steps

Explore Docker from the inside

1

Run an nginx container and hit it

docker run -d -p 8080:80 --name my-web nginx:alpine → curl localhost:8080. See the welcome page. Containers serving real traffic in under 1 second.

2

Exec into the container — see namespaces

docker exec -it my-web sh → ps aux inside (sees PID 1). Exit. ps aux | grep nginx on host (sees real PID). Same process, two different views.

3

Inspect image layers

docker image history nginx:alpine — see each layer, its size and the instruction that created it. docker system df — total Docker disk usage.

4

Apply cgroup limits — see docker stats

docker run -d --name limited --memory 100m --cpus 0.5 nginx:alpine → docker stats limited. See the MEM LIMIT column showing your limit.

5

Mount a volume — persistent data

Create mkdir ~/html && echo "<h1>My page</h1>" > ~/html/index.html. Run nginx with -v ~/html:/usr/share/nginx/html. Hit port 8081 — see your custom page.

6

Full lifecycle — stop, start, remove

Practice: docker stop my-web → docker ps -a (still exists) → docker start my-web → docker rm -f my-web. Clean up all lab containers.

🔧 Lab — Complete Commands

Full lab script

Steps 1–4

# === 1. Run nginx ===
docker run -d -p 8080:80 --name my-web nginx:alpine
curl http://localhost:8080
# <!DOCTYPE html> ... Welcome to nginx!

# === 2. Namespaces demo ===
docker exec -it my-web sh
# Inside container:
ps aux
#   PID  1 nginx: master process
#   PID  7 nginx: worker process
hostname        # random container ID
exit

# On host — same process, different PID
ps aux | grep nginx
# bastianjoe  4521 nginx: master (real PID!)

# === 3. Inspect layers ===
docker image history nginx:alpine
# IMAGE       CREATED    CREATED BY              SIZE
# <hash>      ...        /bin/sh -c #(nop) CMD   0B
# <hash>      ...        COPY file...            1.7kB
# <hash>      ...        RUN /bin/sh -c ...      4.1MB

docker system df
# TYPE        TOTAL     ACTIVE    SIZE      RECLAIMABLE
# Images         3          1     28.5MB    22MB
# Containers     1          1     0B        0B

# === 4. cgroup resource limits ===
docker run -d --name limited \
  --memory 100m \
  --cpus 0.5 \
  nginx:alpine
docker stats limited --no-stream
# CONTAINER  CPU%   MEM USAGE / LIMIT
# limited    0.0%   4.1MiB / 100MiB ← our limit!

Steps 5–6

# === 5. Volume mount ===
mkdir -p ~/html
cat > ~/html/index.html << 'EOF'
<h1>Hello from Day 16!</h1>
<p>This file lives on the HOST machine.</p>
EOF

docker run -d \
  -p 8081:80 \
  --name my-custom-web \
  -v ~/html:/usr/share/nginx/html:ro \
  nginx:alpine

curl http://localhost:8081
# <h1>Hello from Day 16!</h1>

# Edit the host file — container sees changes!
echo "<h1>Updated!</h1>" > ~/html/index.html
curl http://localhost:8081
# <h1>Updated!</h1>  ← instant update!

# === 6. Full lifecycle cleanup ===
docker stop my-web
docker ps -a           # still exists (stopped)
docker start my-web
curl http://localhost:8080  # back online

# Remove everything
docker rm -f my-web my-custom-web limited
docker ps -a           # empty!

# Clean up unused images
docker image prune     # remove dangling images
docker system prune    # remove all unused

💡 Volumes vs bind mounts

-v ~/html:/container/path = bind mount (host path)
-v myvolume:/container/path = named volume (Docker managed)
Named volumes: better for production (Docker manages lifecycle). Bind mounts: better for development (see host files in container).

Knowledge Check

❓ Quiz Time

3 questions · 5 minutes · image vs container, cgroups, exec command

Day 16 knowledge check →

QUESTION 1 OF 3

What is the key difference between a Docker image and a Docker container?

A

Images are stored on disk; containers are stored in memory

B

An image is a read-only blueprint; a container is a running instance of that image with a writable layer

C

Images run on VMs; containers run on bare metal

D

Containers are stored in registries; images are not

QUESTION 2 OF 3

Which Linux kernel feature does Docker use to limit CPU and memory per container?

A

Namespaces — they isolate what the container can see

B

OverlayFS — the layered filesystem

C

cgroups (Control Groups) — they limit and measure resource usage

D

SELinux — mandatory access control

QUESTION 3 OF 3

What command opens an interactive shell inside an already-running container named my-app?

A

docker run -it my-app sh

B

docker attach my-app

C

docker exec -it my-app sh

D

docker shell my-app

Day 16 — Complete

What you learned today

🏠

Not a VM

Containers share the host kernel. Processes with restricted view. <1 sec startup vs 30–60 sec VMs.

🔒

Namespaces

PID/NET/MNT/UTS/IPC/USER. Isolation mechanism. Container sees PID 1; host sees real PID.

⚖️

cgroups

Resource limits. --memory 512m writes to /sys/fs/cgroup/. OOM killer enforces it.

📚

Union FS

OverlayFS stacks read-only layers + writable container layer. Shared layers = fast pull.

Day 16 Action Items

nginx running, curl localhost:8080 works ✓
PID namespace demo: container sees PID 1, host sees real PID ✓
docker stats limited shows 100MiB limit ✓
Volume mount: edit host file, container reflects change ✓

Tomorrow — Day 17

Writing Dockerfiles — Best Practices

Bad Dockerfile → good Dockerfile. Layer cache optimisation. Non-root users. .dockerignore. Compare image sizes before/after. Build the best possible production image.

layer order .dockerignore distroless docker scout

📌 Reference

Namespaces — complete reference

Namespace	Isolates	Without it	Docker flag
PID	Process IDs	Containers see all host processes	--pid=host (share)
NET	Network interfaces, routing	Containers share host network stack	--network=host
MNT	Filesystem mount points	Container sees entire host filesystem	-v /:/host (bind)
UTS	Hostname, domain name	Container has host's hostname	--hostname=name
IPC	Shared memory, semaphores	Containers share IPC resources	--ipc=host
USER	User/group IDs	root in container = root on host	--userns=host

⚠ --network=host and --pid=host are security risks

Using these flags removes the namespace isolation. --network=host means the container has full access to the host network stack — any port the container binds is bound on the host. Avoid in production unless you have a specific reason.

📌 Reference

Essential Docker CLI — all commands

Container commands

# Run
docker run nginx:alpine          # foreground
docker run -d nginx:alpine       # detached (background)
docker run -it alpine sh         # interactive
docker run --rm nginx:alpine     # auto-remove on exit
docker run -p 8080:80            # port mapping
docker run -v /host:/cont        # volume
docker run -e KEY=value          # env var
docker run --name myapp          # named container
docker run --memory 512m         # memory limit
docker run --cpus 1.5            # CPU limit
docker run --restart always      # auto-restart

# Lifecycle
docker stop myapp                # graceful
docker kill myapp                # immediate
docker start myapp               # restart stopped
docker restart myapp             # stop + start
docker pause myapp               # freeze
docker rm myapp                  # delete stopped
docker rm -f myapp               # force delete

# Inspect
docker logs myapp                # stdout/stderr
docker logs -f myapp             # follow
docker logs --tail 50 myapp      # last 50 lines
docker stats                     # live resource usage
docker top myapp                 # processes in container
docker inspect myapp             # full JSON metadata
docker exec -it myapp sh         # interactive shell
docker cp myapp:/file .          # copy file out

Image + system commands

# Images
docker images                    # list
docker pull nginx:alpine         # pull
docker push ghcr.io/user/app     # push
docker build -t app:1.0 .        # build
docker tag app:1.0 app:latest    # tag
docker rmi app:1.0               # remove
docker history app:1.0           # show layers
docker inspect app:1.0           # metadata
docker save app:1.0 > app.tar    # export
docker load < app.tar            # import

# System
docker system df                 # disk usage
docker system prune              # remove unused
docker system prune -a           # remove all unused
docker image prune               # dangling images
docker container prune           # stopped containers
docker volume prune              # unused volumes
docker network prune             # unused networks
docker info                      # Docker system info
docker version                   # client + server

# Volumes
docker volume create mydata
docker volume ls
docker volume inspect mydata
docker volume rm mydata
docker run -v mydata:/app/data app

📌 When to use what

Container vs VM — choosing the right tool

Use containers when

Running stateless microservices (APIs, web apps)
CI/CD workloads (build agents, test runners)
Dev environment standardisation (everyone gets the same environment)
Fast scaling needed (containers start in <1 sec)
Many isolated services on one host (density)
Kubernetes deployments

Use VMs when

Running a different OS (Windows app on Linux host)
Full kernel isolation required (strong security compliance)
Running legacy software that can't be containerised
Database servers needing persistent local storage with full isolation
Infrastructure/network appliances
Compliance requiring complete OS-level separation

💡 In practice — use both

Most cloud architectures use VMs as the foundation (the compute instances) and run containers inside those VMs. The VM provides OS isolation and security boundary; the containers provide application isolation and density.

DP World context

AKS (Azure Kubernetes Service) = VMs managed by Azure running container workloads. The node is a VM (D4s_v3 with 4 vCPU, 16 GB). Inside each node: 20–30 containers (pods). Best of both worlds.

Week 4 Roadmap

Week 4 — Docker mastery path

Day	Topic	Key Skills	Status
Day 16 ✅	Docker Fundamentals	Namespaces, cgroups, Union FS, architecture, lifecycle	✅
Day 17	Writing Dockerfiles	Layer caching, bad→good, .dockerignore, distroless	Tomorrow
Day 18	Docker Compose	Multi-container stacks, networking, health checks	Wed
Day 19	Container Networking	Bridge, host, overlay networks, DNS resolution	Thu
Day 20	Container Security	Trivy scanning, rootless containers, signing	Fri

Week 3 → Week 4 connection

Week 3: you ran docker build + docker push in CI without knowing what happens inside. Week 4: you now understand every step — namespace creation, cgroup files, layer mounting, containerd snapshots. Week 5: run these containers at scale in Kubernetes.

Commands learned today

docker run -d -p -v --memory --cpus
docker exec -it <name> sh
docker logs -f <name>
docker stats <name>
docker image history <image>
docker system df
docker stop/start/rm/rm -f

📌 Troubleshooting

Common Docker issues & fixes

Problem	Cause	Fix
Container exits immediately	App crashes on startup	`docker logs container-name` to see the crash output.
"port already in use"	Another process using that port	Use different host port: `-p 8081:80`. Or find and stop the conflicting process.
Container killed — OOMKilled	Exceeded memory limit	Increase memory: `--memory 1g`. Or fix memory leak in app.
"Cannot connect to Docker daemon"	Docker Desktop not running	Start Docker Desktop. Wait for whale icon in taskbar. Then retry.
"Permission denied" on volume	Container user ≠ host file owner	Add `--user $(id -u):$(id -g)` or use `chmod 755` on the mounted directory.
Container running but app unreachable	Forgot -p flag or wrong port	`docker ps` — check PORTS column. EXPOSE doesn't publish; only `-p` does.
Low disk space	Accumulated images and stopped containers	`docker system prune -a` reclaims unused space. Check with `docker system df`.
Changes lost after container restart	Data in writable container layer	Use volumes for persistent data: `-v mydata:/app/data`. Container layer is ephemeral.

📌 Quick Reference

Day 16 — everything at a glance

Container fundamentals cheatsheet

# Container internals
Namespace = isolation (what you see)
cgroups   = limits (how much you get)
OverlayFS = layered filesystem

# Container is a PROCESS, not a VM
docker run nginx  →  ps aux | grep nginx
# PID 1 inside  =  PID 4521 outside

# Image → Container relationship
docker run -d -p 8080:80 nginx:alpine  # run-1
docker run -d -p 8081:80 nginx:alpine  # run-2
docker run -d -p 8082:80 nginx:alpine  # run-3
# 3 containers · 1 image · minimal extra disk

# Essential daily commands
docker ps                    # running?
docker ps -a                 # all states
docker logs -f myapp         # debug
docker exec -it myapp sh     # inspect
docker stats                 # resources
docker stop myapp            # graceful
docker rm -f myapp           # force clean

# Resource limits
docker run --memory 512m --cpus 1.5 app
# Writes: /sys/fs/cgroup/memory/.../limit
# Kubernetes: resources.limits.memory: "512Mi"

The 3 kernel features

Namespaces — isolation

Container thinks it's alone on the machine
Has its own PID 1, its own network interface, its own filesystem root

cgroups — limits

Container can't consume all RAM/CPU
--memory 512m → kernel file → OOM killer

OverlayFS — layers

Read-only layers (image) + writable layer (container)
Shared base layers save disk space

💡 Day 16 mental model

When you run docker run nginx:
1. runc creates PID/NET/MNT namespaces
2. cgroup files written for limits
3. OverlayFS mounts image layers
4. nginx process starts inside the namespace
5. You see PID 1 inside, real PID outside