If you want to control traffic flow at the IP address or port level, then you might consider using Kubernetes NetworkPolicies for particular applications in your cluster.
NetworkPolicies are an application-centric construct which allow you to specify how a pod is allowed to communicate with various network “entities” over the network.
NetworkPolicies apply to a connection with a pod on one or both ends, and are not relevant to other connections.
The entities that a Pod can communicate with are identified through a combination of the following 3 identifiers:
When defining a pod- or namespace- based NetworkPolicy, you use a selector to specify what traffic is allowed to and from the Pod(s) that match the selector.
Meanwhile, when IP based NetworkPolicies are created, we define policies based on IP blocks (CIDR ranges).
Network policies are implemented by the network plugin. To use network policies, you must be using a networking solution which supports NetworkPolicy. Creating a NetworkPolicy resource without a controller that implements it will have no effect.
There are two sorts of isolation for a pod:
They concern what connections may be established.
“Isolation” here is not absolute, rather it means “some restrictions apply”.
The alternative, “non-isolated for $direction”, means that no restrictions apply in the stated direction.
The two sorts of isolation (or not) are declared independently, and are both relevant for a connection from one pod to another.
By default, a pod is non-isolated for egress; all outbound connections are allowed.
A pod is isolated for egress if there is any NetworkPolicy that both selects the pod and has “Egress” in its policyTypes; we say that such a policy applies to the pod for egress.
When a pod is isolated for egress, the only allowed connections from the pod are those allowed by the egress list of some NetworkPolicy that applies to the pod for egress. The effects of those egress lists combine additively.
By default, a pod is non-isolated for ingress; all inbound connections are allowed.
A pod is isolated for ingress if there is any NetworkPolicy that both selects the pod and has “Ingress” in its policyTypes; we say that such a policy applies to the pod for ingress.
When a pod is isolated for ingress, the only allowed connections into the pod are those from the pod’s node and those allowed by the ingress list of some NetworkPolicy that applies to the pod for ingress.
The effects of those ingress lists combine additively.
Network policies do not conflict; they are additive.
If any policy or policies apply to a given pod for a given direction, the connections allowed in that direction from that pod is the union of what the applicable policies allow.
Thus, order of evaluation does not affect the policy result.
For a connection from a source pod to a destination pod to be allowed, both the egress policy on the source pod and the ingress policy on the destination pod need to allow the connection.
If either side does not allow the connection, it will not happen.
Create Kubernetes cluster with 3 worker nodes.
Master: 1 node
Worker: 2 node
Create docker hub account. Docker Hub if you already have one skip this step
Open Play with Kubernetes login with your docker hub account.
Click on start
It will start a 4 hr session
create three instance
click on + ADD NEW INSTANCE three time to add three instances
kubeadm init --apiserver-advertise-address $(hostname -i) --pod-network-cidr 10.5.0.0/16
enter below command on first node
kubectl apply -f https://raw.githubusercontent.com/cloudnativelabs/kube-router/master/daemonset/kubeadm-kuberouter.yaml
capture output of kubeadm join XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
you may also use kubeadm token list to find token
use this command on second and third node kubeadm join <IP address of master/first node>:6443 –token
enter captured command in second and third node
kubeadm join XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Task: Create a pod with name web and image nginx and block all incoming traffic
name: web
image: nginx
label app:web
port 80
this command will create pod with image nginx and name web
kubectl run web --image=nginx --labels="app=web" --expose --port=80
verify access to port 80
kubectl run busybox --image=busybox --rm -it --restart=Never -- wget -O- http://web:80 --timeout 2
You will get output similar to below output
Connecting to web:80 (10.97.41.90:80)
writing to stdout
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
- 100% |********************************| 615 0:00:00 ETA
written to stdout
pod "busybox" deleted
create network policy to block all incoming traffic
create a new file deny-all-incoming.yaml
vi deny-all-incoming.yaml
Press i to get in insert mode
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
name: deny-all-incoming
spec:
podSelector:
matchLabels:
app: web
ingress: []
use escape to exit insert mode and :wq to save and exit vi
kubectl apply -f deny-all-incoming.yaml
try to access port 80 again
kubectl run busybox --image=busybox --rm -it --restart=Never -- wget -O- http://web:80 --timeout 2
You will get error as access has been block by policy
If you don't see a command prompt, try pressing enter.
wget: can't connect to remote host (10.107.21.116): Connection refused
pod "busybox" deleted
pod default/busybox terminated (Error)
Remove network policy
kubectl delete -f deny-all-incoming.yaml
Task: Create a pod with name dev and image nginx and allow incoming traffic only from pod with label app=store
name: store
image: nginx
label app:store,role:dev
port 80
this command will create pod with image nginx and name store
kubectl run store --image=nginx --labels="app=store,role=dev" --expose --port=80
verify access to port 80
kubectl run busybox --image=busybox --rm -it --restart=Never -- wget -O- http://store:80 --timeout 2
You will get output similar to below output
Connecting to store:80 (10.96.15.161:80)
writing to stdout
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
- 100% |********************************| 615 0:00:00 ETA
written to stdout
pod "busybox" deleted
create network policy to limit incoming traffic only from pod with label app=store
create a new file limit-imcoming-traffic.yaml
vi limit-imcoming-traffic.yaml
Press i to get in insert mode
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
name: limit-imcoming-traffic
spec:
podSelector:
matchLabels:
app: store
role: dev
ingress:
- from:
- podSelector:
matchLabels:
app: store
use escape to exit insert mode and :wq to save and exit vi
Use below command to apply policy
kubectl apply -f limit-imcoming-traffic.yaml
try to access port 80 again
kubectl run busybox --image=busybox --rm -it --restart=Never -- wget -O- http://store:80 --timeout 2
You will get error as access has been block by policy
If you don't see a command prompt, try pressing enter.
wget: can't connect to remote host (10.96.15.161): Connection refused
pod "busybox" deleted
pod default/busybox terminated (Error)
try to access port 80 again by creating pod with label app=store
kubectl run busybox --image=busybox --labels="app=store" --rm -it --restart=Never -- wget -O- http://store:80 --timeout 2
You will get output similar to below output
Connecting to store:80 (10.96.15.161:80)
writing to stdout
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
- 100% |********************************| 615 0:00:00 ETA
written to stdout
pod "busybox" deleted
Remove network policy
kubectl delete -f limit-imcoming-traffic.yaml
Task: Create a pod with name book and image nginx and deny traffic from other namespaces
name: book
image: nginx
label app:book
port 80
namespace: book
Below command will create namespace book
kubectl create namespace book
this command will create pod with image nginx and name book in namespace book
kubectl run book --image=nginx --labels="app=book" --namespace=book --expose --port=80
verify access to port 80
kubectl run busybox --image=busybox --rm -it --restart=Never -- wget -O- http://book.book:80 --timeout 2
You will get output similar to below output
Connecting to book.book:80 (10.107.44.233:80)
writing to stdout
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
- 100% |********************************| 615 0:00:00 ETA
written to stdout
pod "busybox" deleted
create network policy to limit incoming traffic only from pod with label app=store
create a new file deny-from-other-namespaces.yaml
vi deny-from-other-namespaces.yaml
Press i to get in insert mode
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
namespace: book
name: deny-from-other-namespaces
spec:
podSelector:
matchLabels:
ingress:
- from:
- podSelector: {}
use escape to exit insert mode and :wq to save and exit vi
kubectl apply -f deny-from-other-namespaces.yaml
try to access port 80 again
kubectl run busybox --image=busybox --rm -it --restart=Never -- wget -O- http://book.book:80 --timeout 2
You will get error as access has been block by policy
If you don't see a command prompt, try pressing enter.
wget: can't connect to remote host (10.107.44.233): Connection refused
pod "busybox" deleted
pod default/busybox terminated (Error)
try to access port 80 again by creating pod with label app=store
kubectl run busybox --image=busybox --namespace=book --rm -it --restart=Never -- wget -O- http://book:80 --timeout 2
You will get output similar to below output
Connecting to book.book:80 (10.107.44.233:80)
writing to stdout
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
- 100% |********************************| 615 0:00:00 ETA
written to stdout
pod "busybox" deleted
Remove network policy
kubectl delete -f deny-from-other-namespaces.yaml
Task: Create a pod with name demo and image nginx and allow traffic from a namespaces
name: demo
image: nginx
port 80
namespace: demo
Below command will create namespace demo and test
kubectl create namespace demo
kubectl create namespace prod
Add label for namespace demo and test
kubectl label namespace/demo env=demo
kubectl label namespace/prod env=prod
Below command will create pod with image nginx and name demo in namespace demo
kubectl run demo --image=nginx --namespace=demo --expose --port=80
verify access to port 80
kubectl run busybox --image=busybox --rm -it --restart=Never -- wget -O- http://demo.demo:80 --timeout 2
You will get output similar to below output
Connecting to demo.demo:80 (10.107.211.36:80)
writing to stdout
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
- 100% |********************************| 615 0:00:00 ETA
written to stdout
pod "busybox" deleted
create network policy to limit incoming traffic only from namespace prod
create a new file allow-from-a-namespace.yaml
vi allow-from-a-namespace.yaml
Press i to get in insert mode
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
namespace: demo
name: deny-from-other-namespaces
spec:
podSelector:
matchLabels:
ingress:
- from:
- namespaceSelector:
matchLabels:
env: prod
use escape to exit insert mode and :wq to save and exit vi
kubectl apply -f allow-from-a-namespace.yaml
try to access port 80 again
kubectl run busybox --image=busybox --rm -it --restart=Never -- wget -O- http://demo.demo:80 --timeout 2
You will get error as access has been block by policy
If you don't see a command prompt, try pressing enter.
wget: can't connect to remote host (10.107.211.36): Connection refused
pod "busybox" deleted
pod default/busybox terminated (Error)
try to access port 80 again by creating pod in namespace prod
kubectl run busybox --image=busybox --namespace=prod --rm -it --restart=Never -- wget -O- http://demo.demo:80 --timeout 2
You will get output similar to below output
Connecting to demo.demo:80 (10.107.211.36:80)
writing to stdout
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>
<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>
<p><em>Thank you for using nginx.</em></p>
</body>
</html>
- 100% |********************************| 615 0:00:00 ETA
written to stdout
pod "busybox" deleted
Remove network policy
kubectl delete -f allow-from-a-namespace.yaml
Task: Create a pod with name car and image nginx and allow only DNS traffic for outbound
name: car
image: nginx
Below command will create pod with image nginx and name car
kubectl run car --image=nginx --labels="app=car"
Connect to pod and try to access example.com
kubectl exec -it car -- bash
You will get output similar to below output
root@car:/# curl example.com
<!doctype html>
<html>
<head>
<title>Example Domain</title>
<meta charset="utf-8" />
<meta http-equiv="Content-type" content="text/html; charset=utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<style type="text/css">
body {
background-color: #f0f0f2;
margin: 0;
padding: 0;
font-family: -apple-system, system-ui, BlinkMacSystemFont, "Segoe UI", "Open Sans", "Helvetica Neue", Helvetica, Arial, sans-serif;
}
div {
width: 600px;
margin: 5em auto;
padding: 2em;
background-color: #fdfdff;
border-radius: 0.5em;
box-shadow: 2px 3px 7px 2px rgba(0,0,0,0.02);
}
a:link, a:visited {
color: #38488f;
text-decoration: none;
}
@media (max-width: 700px) {
div {
margin: 0 auto;
width: auto;
}
}
</style>
</head>
<body>
<div>
<h1>Example Domain</h1>
<p>This domain is for use in illustrative examples in documents. You may use this
domain in literature without prior coordination or asking for permission.</p>
<p><a href="https://www.iana.org/domains/example">More information...</a></p>
</div>
</body>
</html>
Create network policy to deny all outbound traffic
vi allow-only-dns-outbound.yaml
Press i to get in insert mode
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
name: allow-only-dns-outbound.yaml
spec:
podSelector:
matchLabels:
app: car
policyTypes:
- Egress
egress: []
use escape to exit insert mode and :wq to save and exit vi
kubectl apply -f allow-from-a-namespace.yaml
try to access example.com from inside the pod
kubectl exec -it car -- bash
root@car:/# curl example.com
curl: (6) Could not resolve host: example.com
pod was not able to resolve DNS for example.com
Modify network policy to allow DNS for outbound traffic
vi allow-only-dns-outbound.yaml
Press i to get in insert mode add last 5 line and remove [ ]
kind: NetworkPolicy
apiVersion: networking.k8s.io/v1
metadata:
name: allow-only-dns-outbound.yaml
spec:
podSelector:
matchLabels:
app: car
policyTypes:
- Egress
egress:
- ports:
- port: 53
protocol: UDP
- port: 53
protocol: TCP
use escape to exit insert mode and :wq to save and exit vi
Use below command to apply network policy
kubectl apply -f allow-from-a-namespace.yaml
Connect to pods and try to access example.com and google.com
kubectl exec -it car -- bash
root@car:/# curl example.com
curl: (7) Failed to connect to example.com port 80: Connection refused
root@car:/# curl google.com
curl: (7) Failed to connect to google.com port 80: Connection refused
root@car:/#
Remove network policy
kubectl delete -f allow-from-a-namespace.yaml
Task: Delete all open nodes/instances and close session
What you can’t do with network policies (at least, not yet)
As of Kubernetes 1.23, the following functionality does not exist in the NetworkPolicy API, but you might be able to implement workarounds using Operating System components (such as SELinux, OpenVSwitch, IPTables, and so on) or Layer 7 technologies (Ingress controllers, Service Mesh implementations) or admission controllers.
In case you are new to network security in Kubernetes, its worth noting that the following User Stories cannot (yet) be implemented using the NetworkPolicy API.