How to Fix Kubernetes Probe Failed: Connection Refused
Quick Fix Summary
TL;DRCheck if your application is listening on the correct port and interface, then verify the probe configuration matches.
A Kubernetes probe (liveness or readiness) failed because the kubelet could not establish a TCP connection to the specified port on the Pod's IP address. This indicates the application process inside the container is not accepting connections on the expected endpoint.
Diagnosis & Causes
Recovery Steps
Step 1: Immediate Pod Diagnostics & Logs
First, gather real-time status and logs to see if the application process is running and why it might not be accepting connections.
kubectl describe pod <pod-name> -n <namespace> | grep -A 10 -B 5 "Probe\|Readiness\|Liveness"
kubectl logs <pod-name> -n <namespace> --tail=50
kubectl exec -it <pod-name> -n <namespace> -- netstat -tulpn || ss -tulpnStep 2: Verify Application Binding and Connectivity
Exec into the pod to test if the application is listening correctly and is reachable from within its own network namespace.
kubectl exec -it <pod-name> -n <namespace> -- curl -v http://localhost:<app-port>/<probe-path>
kubectl exec -it <pod-name> -n <namespace> -- wget -O- http://127.0.0.1:<app-port>
kubectl exec -it <pod-name> -n <namespace> -- nc -zv 0.0.0.0 <app-port>Step 3: Audit and Correct Probe Configuration
Compare the live pod's probe spec with the deployment manifest. Ensure port numbers, paths, and host parameters are correct.
kubectl get pod <pod-name> -n <namespace> -o jsonpath='{.spec.containers[0].livenessProbe}' | jq .
kubectl get pod <pod-name> -n <namespace> -o jsonpath='{.spec.containers[0].readinessProbe}' | jq .
# Example corrected probe in deployment YAML:
livenessProbe:
httpGet:
path: /healthz
port: 8080
scheme: HTTP
initialDelaySeconds: 15
periodSeconds: 10Step 4: Test with a Debug Sidecar or Ephemeral Container
For persistent issues, run a temporary debugging container in the pod's network namespace to probe the application from a different perspective.
kubectl debug -it <pod-name> -n <namespace> --image=busybox --target=<container-name>
# Once in debug shell, test connectivity:
wget -O- http://<pod-ip>:<app-port>
nc -zv <pod-ip> <app-port>Architect's Pro Tip
"If your app uses a slow-start framework (e.g., Java/Spring), set 'initialDelaySeconds' generously. A failed liveness probe restarts the pod, which can create a restart loop on a slow-starting app."
Frequently Asked Questions
What's the difference between 'Connection Refused' and other probe failures like timeout?
'Connection Refused' (TCP RST) means nothing is listening on the port. A 'Timeout' means something is listening but didn't respond, indicating application hang or overload.
Can a misconfigured Readiness Probe cause a service outage?
Yes. If ready probes fail, the pod is removed from Service endpoints. Even if the app is functionally working, traffic will stop flowing to it, causing an effective outage.