This is an example with Deployment, sevice, netpol and ingress
¶ Network Diagram
¶ Flow
Let's say you have a simple python web application that runs on port 8000. When running this on your desktop, you would access it via port 8000. In order to run this in kubernetes, you would containerize this application. When you create the container for your python application, you expose port 8000 for the container.
In Kubernetes, you don't access a pod directly. In order to communicate with a pod, you must go through what is called a service. Think of a service as a load balancer inside the kubernetes cluster that sits in front of a defined set of pods. Since you can run multiple pods, the service maintains traffice to the pods. The service will have it's own port which you define. Best practices state the port number for the service should match the port number of the pods it's serving.
Now that you have a pod running and a service running inside the cluster, you need to be able to access them from outside the cluster. This is done through the clustter ingress. Clusters will normally run an ingress, such as Nginx or Contour, to allow external access to internal pods. Normally, cluster ingresses run on ports 80, 443 or 8443.
In order to tell the cluster ingress where to send incoming requests, you will need to create an ingress.yaml (for nginx) or httpproxy.yaml (for contour). These files will allow the ingress to route incoming to the appropiate services, based on the criteria in the yaml.
¶ In the example below
- I have a cluster called mycluster which has an ingress setup to accept traffic to *apps.mycluster.net.
- Inside the cluster, I have an httpproxy setup to redirect any incoming request to the cluster ingress for helloworld.apps.mycluster.net to the hello-world service running on port 8000.
- I have a service running on port 8000 that is setup to forward traffic to any pod with a label of app: hello-world and send traffic to port 8000 on that pod.
- I have pods running labeled app: hello-world exposing port 8000.
- Those pods are running a simple python http server and creating pages based on configmaps.
When a call is made to https://helloworld.apps.mycluster.net,
- the call will go into the cluster ingress.
- The cluster ingress will check to see if it has a FQDN registered for helloworld.apps.mycluster.net (this was created using the httpproxy.yaml).
- It will then redirect traffic to the service defined for that FQDN and route (we are just using / for this example).
- The service will then distribute traffic to the pods registered with that service and a helloworld.html page will be displayed back to the user from the pod.
¶ Configuration
- Create an ingress Use the FQDN that will point to the cluster ingress on port 443. That ingres will define one or more routes. Each route will point to one or more services (in this example, we have 1 route for "/" pointing to the hello-world service.)
- Create a Service. This service is called from the ingress and is used to determine where to send traffic. This is done via the "selector" section.
- Create network policy to allow traffic from the ingress namespace to your pods container port.
¶ ConfigMap
apiVersion: v1
kind: ConfigMap
metadata:
name: app-config
namespace: ns-name
data:
ENV: Dev
APP_ID: "MY_APP"
---
apiVersion: v1
kind: ConfigMap
metadata:
name: index-cm
namespace: ns-name
data:
index.html: '<html>
<h1>Hello World</h1> <br>
<hr>
This index.html file was mounted via a configmap volume.<br>
The index.html file is actually a configmap. <br>
The ENV and SEAL_ID are also configmap Environment variables, mounted as configmaps.<br>
<br>
<b>ENV</b>: Dev <br>
<b>APP_ID</b>: My_APP<br><br>
<a href="hello.html">jump to hello page</a>
</html>'
hello.html: '<html>
<h1>Hello</h1><br>
This is a second page called from <a href="index.html">hello.html</a>
<br>
<b>ENV</b>: Dev <br>
<b>APP_ID</b>: My_APP<br><br>
<br>
<a href="index.html">Back to main page.</a>
</html>'
¶ Deployment
apiVersion: apps/v1
kind: Deployment
metadata:
name: helloworld
namespace: ns-name
spec:
selector:
matchLabels:
app: helloworld
strategy:
type: RollingUpdate
rollingUpdate:
maxSurge: 1
maxUnavailable: 1
template:
metadata:
labels:
app: helloworld
spec:
securityContext:
runAsUser: 999
fsGroup: 999
containers:
- name: helloworld
image: app-image
ports:
- containerPort: 8000
envFrom:
- configMapRef:
name: app-config
imagePullPolicy: Always
# resources:
# limits:
# cpu: 20
# memory: 20Mi
# requests:
# cpu: 10m
# memory: 20Mi
readinessProbe:
httpGet:
path: /
port: 8000
initialDelaySeconds: 5
periodSeconds: 1
livenessProbe:
httpGet:
path: /
port: 8000
initialDelaySeconds: 5
periodSeconds: 1
volumeMounts:
- name: index-vol
mountPath: /tmp/hw
readOnly: true
volumes:
- name: index-vol
configMap:
name: index-cm
¶ Service
This service is called from the ingress and is used to determine where to send traffic. This is done via the "selector" section.
apiVersion: v1
kind: Service
metadata:
name: helloworld-svc
namespace: ns-name
labels:
app: helloworld
spec:
ports:
- port: 8000
targetPort: 8000
protocol: TCP
name: http
selector:
app: helloworld
Note: Use which ever ingress you have installed.
¶ Ingress
¶ Nginx
This Ingress definition is not required but can be useful in validating the ingress/httpproxy -> servive -> pod setup.
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: hello-world
spec:
rules:
- host: "helloworld.apps.mycluster.net"
http:
paths:
- path: /
backend:
serviceName: hello-world
servicePort: 8000
tls:
- hosts:
- "hw-glb-example.apps.mycluster.net"
secretName: ingress-contour/ingress-contour-default-ssl-cert
or
¶ HTTPProxy
This ingress definition uses an FQDN that points directly to this specific cluster (notice the fqdn ends with the cluster name in it).
apiVersion: projectcontour.io/v1
kind: HTTPProxy
metadata:
name: helloworld
namespace: ns-name
spec:
virtualhost:
fqdn: helloworld.apps.mycluster.net
tls:
secretName: ingress-contour/ingress-contour-default-ssl-cert
routes:
- conditions:
- prefix: /
services:
- name: helloworld-svc
port: 8000
¶ GLB Ingress
¶ HTTPProxy
The FQDN used in this ingress will be used on all clusters running this app that you want to access from the GLB.
apiVersion: projectcontour.io/v1
kind: HTTPProxy
metadata:
name: helloworld
namespace: ns-name
spec:
virtualhost:
fqdn: helloworld-dev.mydomain.net # This will be used on all clusters running this app that you want to access from the GLB.
tls:
secretName: ingress-contour/ingress-contour-default-ssl-cert
routes:
- conditions:
- prefix: /
services:
- name: helloworld-svc
port: 8000
¶ Network Policy
¶ Native
Allow traffice to pass from contour namespace (cluster ingress) to your pod on the container port.
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: ingress-to-hello-world-policy
spec:
ingress:
- from:
- namespaceSelector:
matchLabels:
namespace: ingress-contour
ports:
- protocol: TCP
port: 8000
podSelector:
matchLabels:
app: hello-world
policyTypes:
- Ingress
or
¶ Calico
apiVersion: projectcalico.org/v3
kind: NetworkPolicy
metadata:
name: allow-ingress-to-helloworld-policy
namespace: ns-name
spec:
order: 1000 # must be set (lower values take precedence)
selector: app == 'helloworld' # apply policy on pod(s) with matching label
types:
- Ingress
ingress:
- action: Allow
protocol: TCP
source:
namespaceSelector: cluster_namespace == 'ingress-contour'
destination:
namespaceSelector: cluster_namespace == 'my-dev' # allow ingress to workloads in specific namespace
selector: app == 'helloworld' # allow ingress to pod(s) with matching label
ports: # must define ports on destination
- 8000 # allow ingress on port 8000
¶ HPA
apiVersion: autoscaling/v2beta2
kind: HorizontalPodAutoscaler
#namespace: ns-name
metadata:
name: helloworld-hpa
spec:
minReplicas: 1
maxReplicas: 3
metrics:
- type: Resource
resource:
name: cpu
target:
averageValue: 9m
type: AverageValue
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: helloworld
behavior:
scaleDown:
policies:
- type: Pods
value: 1
periodSeconds: 60
scaleUp:
policies:
- type: Pods
value: 1
periodSeconds: 60