- Joel Zamboni

Production Readiness Checklist for a Vibe-Coded App on AWS

A production readiness checklist for an AI-built app on AWS: secrets, IAM, backups, monitoring, CI/CD, rollback, rate limits, logging and cost alarms.

An app built quickly with AI assistance can work well in a demo and still be far from ready for paying customers. This production readiness checklist is for founders and CTOs who have a vibe-coded app running on AWS, or about to be, and want to know what has to be true before real users and real data arrive. Every item is concrete and checkable, and each links to the AWS or industry documentation behind it.

Why vibe-coded apps need a checklist

Andrej Karpathy coined the term in February 2025 to describe coding where you “fully give in to the vibes” and “forget that the code even exists”. That is a fine way to explore an idea. It becomes a problem when the result goes to production unchanged, because the parts of software that keep a business safe are the parts a demo never exercises: failure, abuse, recovery and cost.

AI coding tools tend to optimize for “it runs.” They will happily put an API key in a config file, open a database to the internet to fix a connection error, or skip authorization checks on an endpoint because the prompt did not mention them. None of these show up when you click through the app yourself. All of them show up eventually. Most vibe coding security problems are not exotic attacks; they are ordinary defaults that nobody reviewed.

The checklist below is grouped by area, and it works as a production readiness checklist template you can copy into your issue tracker. You do not need to finish every item before launch, but you should know which ones are open and why.

1. Secrets

  • No secrets in the repository, including old commits. Search the full Git history, not only the current files. GitHub secret scanning can flag known token formats.
  • Rotate every key that was ever committed, even if you deleted it later. Deleting a file does not remove it from history or from anyone who cloned the repository.
  • Store secrets in AWS Secrets Manager or SSM Parameter Store and load them at runtime.
  • No secrets in front-end code. Anything shipped to the browser is public. Calls to paid APIs, including LLM providers, go through your back end.

2. IAM and account access

  • Do not use the root user for daily work, and protect it with MFA. AWS’s IAM security best practices cover this and the items below.
  • Least privilege for the application. The app’s role should grant only the actions it uses on the resources it uses. A policy with "Action": "*" that an AI tool suggested to get past a permissions error is a common find.
  • No long-lived AWS keys in CI. Use OpenID Connect so GitHub Actions can access AWS without storing credentials as long-lived secrets.
  • Separate production from everything else, ideally in its own AWS account, so a mistake in a test environment cannot touch customer data.

3. Data exposure and application security

  • The database is not publicly accessible. It should live in private subnets, reachable only from the application.
  • S3 buckets block public access unless a bucket is deliberately public. AWS provides S3 Block Public Access settings at the account and bucket level.
  • Every endpoint checks authorization on the server. Broken access control tops the OWASP Top 10 list of web application risks. Test it directly: log in as user A and request user B’s records by ID.
  • Inputs are validated and database queries are parameterized.
  • Dependencies are scanned for known vulnerabilities, with automated update pull requests turned on.

4. Backups and recovery

  • Automated database backups are on, with point-in-time recovery and a retention period you chose on purpose. RDS automated backups let you restore to any point within that period.
  • You have tested a restore. A backup you have never restored is a hope, not a backup. Restore to a new instance, check the data and time how long it took.
  • Deletion protection is on for the production database.
  • User uploads in S3 are protected with versioning or a backup plan.

5. Monitoring and alerting

  • The app exposes a health check that confirms it can reach its dependencies, not only that the process is up.
  • Alarms exist for what customers feel: error rate, latency and availability, rather than only CPU.
  • Alarms reach a person. An alert that goes to an unread inbox is the same as no alert. Decide who gets paged at night and how.
  • Each alarm has a short runbook: what it means, how to check and what to do first.

6. Logging and audit

  • Logs are structured (JSON with request IDs) so you can follow one request through the system.
  • No secrets or sensitive personal data in logs. Check what your framework logs by default on errors.
  • Log retention is set. By default, CloudWatch Logs stores log data indefinitely, which grows cost and can conflict with privacy commitments.
  • AWS CloudTrail is on so you can see who changed what in the account.

7. CI/CD

  • Every production deploy goes through a pipeline, never from a laptop and never by editing resources in the console.
  • The pipeline runs tests and fails the deploy when they fail. Even a small suite on the critical paths (sign-up, login, payment) catches a lot of AI-introduced regressions.
  • Infrastructure is defined in code (Terraform, CloudFormation or CDK) so environments can be rebuilt and reviewed.
  • Changes are reviewed by a person before they reach production, including AI-generated changes.

8. Rollback

  • You can return to the previous version in minutes. Keep previous container images tagged and know the command or button.
  • Automatic rollback on failed deploys where your platform supports it. On ECS, the deployment circuit breaker can roll a failed deployment back to the last completed one.
  • Database migrations are backward compatible, so the old code still runs against the new schema while you roll back.

9. Rate limiting and abuse

  • Public endpoints are rate limited. AWS WAF rate-based rules count requests and limit sources that send them too fast.
  • Login, sign-up and password reset have stricter limits to slow credential stuffing and fake account creation.
  • Expensive endpoints are capped per user, especially anything that calls an LLM API, sends email or SMS, or starts heavy background jobs. These are the endpoints where abuse turns directly into your bill.

10. Cost alarms

  • An AWS Budget with alerts is set at a level that would surprise you. AWS Budgets can email or notify you when actual or forecast spend crosses a threshold.
  • Cost Anomaly Detection is on, so unusual spend in one service gets flagged even if the total is still under budget.
  • Third-party API spending limits are set, especially with LLM providers, where a loop or an abusive user can run up charges quickly.
  • Resources are tagged by environment so you can see what production costs versus everything else.

The short version

If you only have a day, do these first: rotate any committed secrets, close public database and bucket access, check authorization on every endpoint, turn on and test database backups, set a budget alert, and make sure one alarm on errors reaches a person. Those six cover the failures that are hardest to recover from: leaked data, lost data and a runaway bill.

Then work through the rest as part of your normal sprint. Production readiness is not a one-time gate. New features bring new endpoints, new secrets and new costs, so the checklist is worth repeating every time the app changes shape.

From checklist to production readiness review

Larger engineering organizations turn this kind of list into a formal step. At Google, SRE teams run a Production Readiness Review, described as “a process that identifies the reliability needs of a service based on its specific details,” before they take responsibility for a service in production. A small team does not need the ceremony, but the habit transfers well: before a major launch, walk through the checklist with whoever will carry the pager and agree on which open items are acceptable.

Frequently asked questions

What is a production readiness checklist?

A production readiness checklist is a list of concrete, checkable conditions an application should meet before it serves real users and real data. For an app on AWS that usually covers secrets, IAM, data exposure, backups, monitoring, logging, CI/CD, rollback, rate limiting and cost alarms. Its value is that every open item is known and has a reason, rather than discovered during an incident.

What is production readiness?

Production readiness means an application can handle the conditions a demo never exercises: failures, abuse, recovery and cost. A production-ready app keeps its secrets out of code, can be restored from a tested backup, alerts a person when users are affected, deploys and rolls back through a pipeline, and will not run up an unbounded bill. It is not a one-time gate, because every new feature changes the picture.

What is a production readiness review?

A production readiness review is a structured walkthrough of a service before launch, in which the team that will operate it checks its reliability needs against a list like the one above. Google’s SRE teams use one as a prerequisite before taking over a service. For a startup, a one-hour review with the checklist and the person on call covers most of the benefit.

Getting help with production readiness

If you want the app itself built and maintained by a team, that is what our sister brand Avanti Studio does.

If the app exists and what you need is the AWS side of this checklist done and kept up, that is what Webera does. Webera is a DevOps subscription for SaaS teams on AWS: senior engineers working alongside nine AI agents, including Guardian for security, Sentinel for observability, Conductor for CI/CD, Optimizer for FinOps and Forge for application readiness. The agents do the watching, routing and first analysis around the clock; the engineers make the calls and ship the fixes.

Pro is $4,999 a month, with unlimited requests (two active at a time), 24/7 monitoring and incident response, and an average 36-hour turnaround. Elite ($9,999) and Platform ($14,999) add a named engineer, compliance support and a team inside your sprints. Every plan has a three-month minimum, then continues month to month, and you own all the code. Not sure where you stand? Score your team first, or compare the plans on our pricing page.

Need DevOps expertise?

Our team of senior engineers can help you implement these practices.

Book a Discovery Call