Kubernetes DevOps Tip #7: Cost of Repair in Development vs. Production
Written By: Sarah Geisinger
One of the main benefits about Kubernetes is the platform’s ability to increase the speed of development. By using microservices and containers, development happens faster. This is all good news and most certainly a huge benefit. But when you increase development velocity, one major drawback emerges: the cost to repair defects.
The Capers Jones graph below shows the percentage of defects introduced during each phase of the development life cycle. More importantly, it demonstrates how the cost to repair said defect goes from 1x when coding to more than 640x in production.
Kubernetes misconfiguration can be expensive to fix.
Just as problems with code are expensive to fix, so too are Kubernetes misconfigurations. When spinning up clusters in support of an application, there are configurations that just need to be done and run. You need to:
- avoid running your container as root to ensure Kubernetes security
- set the right CPU and memory to control cloud costs
- set liveness and readiness probes to ensure proper autoscaling
Most companies run Kubernetes without configuration best practices in mind, creating security and reliability issues that add to technical debt and can be very expensive to repair.
Cost to Repair Kubernetes Misconfigured Clusters
The Kubernetes Configuration Benchmark report gives us average findings per cluster and workloads:
- Average # of Kubernetes misconfigurations per cluster — 328
- Average # of workloads per Kubernetes cluster — 110
- Average # of findings per workload — 3
Now consider these numbers based on the cost for a DevOps engineer:
- Hourly rate for DevOps engineer — $100
- Cost to fix in coding phase (5 minutes) — $8.33
- Cost to fix a workload Kubernetes misconfiguration at time of Git pull request — $24.85
- Cost to fix a workload misconfiguration in production — $15,903.03.
Let’s just read that again: $15,903!!!!!!
When considering how you are configuring Kubernetes, you MUST consider getting it right in your pre-production environments. You NEED to ensure misconfiguration cannot bleed into production in the first place.
How to Identify Kubernetes Misconfiguration Early
Fairwinds enables organizations to shift configuration validation earlier, reducing the cost to fix up to 640x. We do this by providing continuous and automatic scanning of your clusters for Kubernetes best practices. Our solution can scan your development environments, alert developers to misconfigurations AND show them how to fix the problem-a 5 minute task at the cost of $8.33!
The Fairwinds Insights Admission Controller will reject any Kubernetes resources from entering your cluster if they don’t conform to your organization’s policies. Once again, helping to reduce your production environment cost to repair from $15k down to $8–25.00.
Don’t lose the benefits of Kubernetes by not configuring it correctly.
