A failed release is easier to fix when you can separate build errors from runtime, configuration, networking, and data problems. This guide provides a provider-neutral workflow for troubleshooting cloud deployment failures, tracking recurring signals, deciding when to roll back, and improving your release runbook over time.
Overview
Cloud application failures usually become visible at one of three stages: during the build, while the platform starts the new instance, or after the release begins serving real traffic. Treating every failure as a code defect can waste time. A dependency may be absent from the production install, a process may be listening on the wrong port, a health check may be too strict, or an application may be unable to reach its database.
Start by identifying the first stage at which the deployment diverged from the expected path:
- Build stage: source retrieval, dependency installation, compilation, tests, and image creation.
- Startup stage: process launch, environment loading, port binding, migrations, and health checks.
- Serving stage: requests, background jobs, database queries, external API calls, and scaling behavior.
Record the release identifier, commit or image digest, deployment time, changed components, and the first visible error. These details create a reliable boundary between what changed and what merely happened at the same time. For broader context on logs, metrics, and traces, see the cloud app observability guide.
What to track
A useful troubleshooting record connects symptoms to evidence and an approved response. The following table is a practical starting point for a deployment runbook.
| Symptom | Likely causes | First checks | Typical next action |
|---|---|---|---|
| Dependency installation fails | Lockfile mismatch, unavailable package, incompatible runtime, or private registry access | Build log, manifest, lockfile, runtime version, registry credentials | Reproduce with the production install command; align the runtime and lockfile before redeploying |
| Build passes but the app will not start | Wrong start command, missing executable, import error, or failed initialization | Startup log, process command, artifact contents, and exit code | Run the exact production command locally or in a temporary environment |
| Port or process error | Hard-coded port, incorrect process type, or a server bound only to localhost | Assigned port, bind address, process list, and platform configuration | Use the platform-provided port and bind the web server to an externally reachable interface |
| Health check fails | Wrong path, slow startup, authentication requirement, bad status code, or insufficient timeout | Health-check URL, response code, startup duration, and probe logs | Use a lightweight readiness endpoint and separate startup readiness from deep dependency checks where appropriate |
| App crashes after release | Runtime exception, missing environment variable, incompatible configuration, or unhandled request path | Application logs grouped by release and instance | Compare the failing release with the last healthy release and reproduce the smallest failing path |
| Database connection fails | Incorrect secret, network access, TLS setting, connection limit, or migration state | Redacted connection configuration, database logs, and migration output | Test connectivity without exposing credentials; verify migration compatibility before retrying |
| Requests time out | Slow query, upstream latency, resource exhaustion, queue backlog, or an unsuitable timeout | Request traces, latency percentiles, resource metrics, and upstream timings | Find the slowest dependency first; avoid increasing timeouts without understanding the bottleneck |
| New release serves errors while old instances work | Release-specific code, configuration, schema incompatibility, or partial rollout | Traffic split, instance health, error rate by version, and schema changes | Pause or reverse the rollout when user impact is increasing |
Track configuration separately from code. Environment variables, secrets, service bindings, and permissions can change the behavior of an otherwise identical artifact. Use a documented, deployment-safe process for these values rather than copying them manually between environments; the environment variables and secrets guide covers the relevant controls.
For every release, retain at least the build result, deployment result, startup result, health status, request error rate, latency, restart count, and rollback decision. Do not store secrets in the troubleshooting record. Redact tokens, connection strings, personal data, and full authorization headers before sharing logs.
Cadence and checkpoints
Use checkpoints instead of waiting for a complete outage. A staged review makes it easier to stop a bad release before it reaches all users.
Before deployment
- Confirm the build command, start command, runtime version, and artifact contents.
- Check that required configuration exists in the target environment and that the application fails clearly when a critical value is absent.
- Review database migrations for backward compatibility if old and new application versions may run together.
- Verify the health endpoint, expected port, resource limits, and dependency access.
- Define the rollback target and the person authorized to stop the release.
During deployment
- Watch build output for the first error, not only the final summary line.
- Confirm that a new instance reaches readiness within the expected startup window.
- Compare error rate, latency, restarts, and saturation with the previous release.
- Increase traffic gradually when the platform supports staged or canary delivery.
After deployment
- Run a small set of smoke tests covering login, the primary user journey, writes, reads, and important integrations.
- Review logs by release identifier so unrelated background noise does not obscure the incident.
- Record whether the release was successful, partially successful, rolled back, or left in place with a mitigation.
Review this checklist monthly or quarterly, depending on release frequency and operational risk. A deployment process that worked for a small service may need new checks after adding workers, queues, databases, scheduled jobs, or container images. If you build and deploy Node.js or Python applications regularly, keep the runtime-specific commands in the runbook and update them when the project changes its package manager, build system, or base image.
How to interpret changes
Look for relationships rather than isolated numbers. A rise in restarts immediately after a dependency update points toward startup or runtime compatibility. A healthy startup followed by increasing request latency points more toward traffic, resource limits, database performance, or an external service. A successful application check with failing user transactions may indicate that the health endpoint is too shallow.
Compare the current release with the last known healthy release using the same time window where possible. Useful comparisons include:
- Build duration and failure stage
- Startup duration and exit codes
- Request error rate and latency by endpoint
- CPU, memory, storage, and connection utilization
- Database query latency, rejected connections, and migration status
- Queue depth, worker failures, and external dependency response times
Rollback when the release is causing material user impact, the cause is not quickly reversible, or the previous version is known to be safe. First check whether the rollback itself is compatible with database and message changes. A code rollback may not undo a schema migration, consumed message, or changed external resource. If rollback is unsafe, pause further rollout, apply a narrowly scoped mitigation, and preserve evidence for diagnosis. The backup and disaster recovery guide can help separate release rollback from data restoration decisions.
After stabilization, write a short incident record: impact, timeline, detection signal, root or contributing causes, recovery action, and one or two preventive changes. Avoid treating “redeployed successfully” as a root cause. The useful question is why the original process allowed the failure to reach production or made it difficult to detect.
When to revisit
Revisit this troubleshooting guide on a monthly or quarterly cadence and whenever a recurring data point changes. At minimum, update it after a failed deployment, a rollback, a new runtime or base image, a change in build or start commands, a database migration strategy change, or the introduction of a new platform health check.
Use the following practical review sequence:
- List the deployment failures and near misses since the last review.
- Group them by build, startup, health, runtime, data, network, and capacity causes.
- Identify signals that appeared late or were difficult to query.
- Turn repeated manual checks into automated tests, release gates, or alerts.
- Confirm that the documented rollback target and procedure still work.
- Test restoration and recovery procedures separately from application rollback.
Keep the runbook close to the deployment configuration and include the exact commands or dashboard queries your team uses. A provider-neutral workflow is durable, but the details of runtime behavior, buildpacks, container images, health-check defaults, and log terminology can change. Review those details whenever the app deployment platform or delivery pipeline changes. The goal is not to eliminate every failure; it is to detect failures earlier, limit their reach, and give the next responder enough evidence to act safely.
