What to Measure and Control in Kubernetes
The Kubernetes Maturity Model includes seven phases that you’ll undertake to achieve full maturity. In phase six, we discuss measurement and control .
When new to Kubernetes, you’ll be running through the basics — preparing, deploying, improving. Eventually you’ll have gained confidence and will now want to refine what’s been set up. This is the step in maturity where you identify what you need to measure around five key areas:
- Security — You will measure how many and what vulnerabilities exist in your containers or clusters and how often/when you are patching workloads, clusters or add-ons.
- Auditing — You’ll create an audit trail to understand who has performed recent actions and what actions workloads are taking in your clusters. You’ll be able to identify if unauthorized access or actions have occurred.
- Drift — You will be able to identify which workloads do not conform to your standards, what versions of dependencies/cluster add-ons are running and if workloads are compatible with future versions of Kubernetes.
- Efficiency — You’ll measure to track the typical or standard resource usage of your workloads and the typical capacity/usage of the nodes within your clusters. You’ll also know how often your clusters are scaling.
- Velocity — You’ll measure to improve your development velocity. This will include understanding how often deployments are being shipped, how many users access your clusters and the most common actions being taken within your clusters.
Once you start to gather data in these key areas, you may notice some problems. Workloads may be disorganized, impacting other workloads. There may be too much access from workloads causing security issues. There may be reliability or scalability issues (not scaling enough or scaling too frequently). Costs may creep up too high as too many resources are being used or workloads are not being cleaned up.
To overcome this, you’ll want to put controls in place. Here you need to answer some fundamental questions based on data to establish a set of Kubernetes guardrails. You will need to answer questions around security, configurations and workloads:
Kubernetes workload security is essential. How will you control cluster permissions around:
- Who has access to clusters?
- What actions users can take within clusters?
- What actions workloads can take within clusters?
- What level of permissions workloads have within clusters?
- What are the network policies between workloads within your clusters?
Solid Kubernetes environments will have configuration standards for consistency. You should have controls in place around:
- Where Kubernetes resources live and are defined?
- What changes happen and when?
- What is your code review process for resources?
- What type of resources can be deployed in your clusters?
- Which namespaces are usable by which users?
- Which namespaces workloads are deployed to?
- How do you set the amount of resources available to a workload or namespace?
- What are your common standards/defaults across your workloads/deployments?
Similarly, you will have established workflows for how workloads and services are deployed, promotion paths and responsibility:
- Who can deploy workloads and services to your clusters?
- How workloads and services can be deployed to your clusters?
- What is the promotion path between environments?
- Who is responsible for what aspects of your environment?
Once these questions are answered, you will have a set of policies to start implementing configuration changes within your clusters. This is where Kubernetes policy enforcement becomes important. It isn’t “good enough” to simply write the policies down and expect your team to follow through. You need a way to enforce policies across your clusters.
There are a number of open source tools such as Polaris, OPA, Goldilocks and Trivy that can help check configurations for Kubernetes best practices and customized policies. This still requires your team to apply each tool across each cluster. This can become cumbersome fast if you’re managing multiple people and clusters.
Using software like Fairwinds Insights combines vetted open source tools into a single dashboard view. It provides the ability for you to both measure cluster configurations and establish and enforce policies based on this data. You get visibility into severe or medium problems relating to security, reliability and efficiency.
At this stage of your Kubernetes maturity, you will need options to give you measurement and control. Fairwinds Insights is software able to help you measure and implement guardrails. It has a cross-engineering team impact.