Introduction
Orchestration tools like Kubernetes introduced unprecedented levels of resiliency and versatility to software deployment and management. They also turned the spotlight on the inadequacies of the current security landscape.
Kubernetes’ dispersed architecture provides us with several additional layers. We can use them to enhance and build upon existing security settings. If you configure these layers properly, they can isolate any security threat before it breaks out from its point of origin.
This article focuses on the measures you can take to improve the overall security of your Kubernetes cluster.
1. Enable Role-Based Access Control (RBAC)
The complexity of a system running on multiple devices, with many interconnected microservices managed by hundreds of individuals and utilities is a logistical nightmare.
Exercise strict control over the permissions granted to users. Kubernetes is a proponent of the role-based access control (RBAC) method. The role-based access control method means that no user has more permissions than they need to accomplish their tasks effectively. Kubernetes is all about automation, and RBAC uses an API group to drive authorization decisions through the Kubernetes API.
Note: Automation can be an issue in this regard. A user that has permissions to create pods within a namespace can use that role to create a pod to access config maps or secrets. This irregularity is called privilege escalation, where a user grants themselves additional permissions.
As of Kubernetes version 1.8, RBAC mode is stable and backed by the rbac.authorization.k8s.io/v1 API. Enable RBAC by starting the API server with the following command:
--authorization-mode=RBACKubernetes has a set of predefined roles for both human users and components, such as apps, controllers, and nodes. These predefined roles are numerous and offer a reasonable default level of separate roles for the most common tasks.
2. Keep Your Secrets Secret
In Kubernetes, a Secret is a small object that contains sensitive data, like a password or token.
Even though a pod is not able to access the secrets of another pod, it is crucial to keep the secret separate from an image or pod. Otherwise, anyone with access to the image would have access to the secret as well. Complex applications that handle multiple processes and have public access are especially vulnerable in this regard.
Assign Processes to Different Containers
Each container in a pod must request the secret volume in its volume. Reduce the risk of secrets being exposed by dividing processes into separate containers. Use a front-end container that interacts with users but is unable to see the private key.
Complement that container with a signer container that can see the private key and respond to simple signing requests from the front-end. This partitioned approach forces an attacker to perform a set of complex actions in an attempt to breach your security measures.
Note: Administrators should always use strong credentials from the API servers to their etcd server. Mutual auth via TLS client certificates can isolate the etcd servers behind a firewall that only the API servers may access.
3. Restrict Pod-to-Pod Traffic With a Kubernetes Network Policy
It is a Kubernetes Security best practice to impose the TLS security protocol on each level of the application deployment pipeline. However, it’s also imperative to secure the individual elements that make up the cluster and the elements that control access to the cluster.
Pods accept traffic from any source by default. When you define Network Policies, you set specific rules on how pods communicate within a cluster and with external resources.
Network policies do not conflict but are instead additive. Defining a network policy in a namespace means that the pod is going to reject any connection not allowed by that policy, effectively isolating the pod. The pod is limited to what is approved by the combination of multiple network policies.
Note: You shouldn’t run pods with mixed security levels on the same node due to the lack of guaranteed security boundaries between pods.
4. Enhance Pod Security
Securing the Kernel with AppArmor or SELinux
Containers share the same kernel, making it necessary to use additional tools to improve container isolation. Security modules, like AppArmor and systems that enforce access control policies like SELinux, confine user programs and system services. They are also used to deny access to files and limit network resources.
AppArmor limits the set of resources available to a container within the system. Each profile can run in either enforcing mode, which blocks access to disallowed resources or complain mode which only reports violations. A timely report of potential issues significantly improves your systems’ logging and auditing capabilities.
SELinux manages resource allocation independently of the general Linux ACM (Access Control Mechanism). It does not recognize a superuser (root) and is not dependent on setuid/setgid binaries.
Securing a system without SELinux relies on the proper configuration of privileged applications and the kernel itself. A misconfiguration in these areas may compromise the entire system. The security of a system based on an SELinux kernel depends on the correctness of the kernel and its security-policy configuration.
Attacks still pose a significant threat. However, with SELinux, individual user programs and system daemons do not necessarily need to endanger the security of the entire system.
Taints and Tolerations
Kubernetes provides the option to create predefined rules when the system assigns new pods to nodes.
As an orchestration tool, Kubernetes tends to start pods on the most efficient location in the cluster, i.e., location with the smallest workload. It is possible to tweak the placement of pods by defining Taints and Tolerations.
Taints allow nodes to ‘reject’ a pod or set of pods based on the predefined rules. On the other hand, Tolerations are applied at the pod level, and they enable pods to schedule to nodes with matching Taints (the keys and effects are the same).
Taints and tolerations should be used in tandem to make sure that pods do not get scheduled onto inappropriate nodes.
Namespaces
Kubernetes does not have a mechanism that provides security across Namespaces. Limit the use of Namespaces, as a security feature, within trusted domains, and for internal purposes. Do not use Namespaces when you want to deny a user of the Kubernetes cluster access to any of the other Namespaces’ resources.
The watch and list requests allow clients to inspect the values of all secrets that are in that namespace. Only the most privileged, system-level components should have permission to implement these requests.
5. Kubernetes Rolling Updates
It has become impossible to track all potential attack vectors. This fact is unfortunate as there is nothing more vital than to be aware and on top of potential threats. The best defense is to make sure that you are running the latest available version of Kubernetes.
There are several techniques such as rolling updates, and node pool migrations that allow you to complete an update with minimal disruption and downtime.
6. Define Audit Policies
Audit logging records the timeline of events that occur in a Kubernetes cluster. By keeping track of the actions taken by users and the Kubernetes API, administrators can analyze the chain of events that led to a potential issue.
Kubernetes allows you to fine-tune audit policies by defining how often events are being logged, if alerts should be issued, and the procedure for terminating affected pods.
Use audit logging regularly to ensure that your system is up-to-date and that threats are kept under tabs. A container-based deployment adds another dimension to the audit process. A comparison between the original image and the image running in the container can be effectively used to see if any discrepancies can lead to security concerns. Make sure that your software versions always contain the latest security patches.
Note: Automating audit logging and predefined general audit rules are an efficient way to maintain control over your clusters. Minimal images that contain only the files and libraries essential to the application improve audit efficiency.
Conclusion
Now you understand Kubernetes security and its best practices.
Follow the principles of least privilege and defense-in-depth. If an attacker manages to compromise one component, they should not gain full access to the system. Instead, ensure that attackers need to penetrate additional layers before being able to cause significant harm or access sensitive data.
For a more comprehensive overview on how to build efficient clusters, check out our Kubernetes Best Practices article.
To stay informed about the latest IT security news, we recommend you check out our list of Top Internet Security Blogs.
