SixHack Academy Offensive learning
ES Español ✓ EN English ✓
Enter
Vibe Coding: Why AI-Generated Code Ships Full of Vulnerabilities

Vibe Coding: Why AI-Generated Code Ships Full of Vulnerabilities

Artificial intelligence has changed the way software is built. Developers no longer need to write every function from scratch: they can describe an idea, request an implementation, iterate on it, and produce a working application very quickly. This approach, commonly known as vibe coding, has dramatically lowered the barrier to building software.

The problem begins when speed replaces understanding. A model can generate a polished interface, a responsive API, and a database integration that looks complete, but that does not mean it has understood the application's trust boundaries. Authentication, authorization, secrets, dependencies, input validation, and business logic still require explicit decisions.

The risk is not that AI writes bad code at random. The deeper issue is that it tends to optimize for the shortest path to satisfy the request. And the shortest path to making something work is not always the safest one once a user stops behaving the way the developer expected.

Note: the code and architecture examples in this article are illustrative and intended for defensive analysis and security education.

In this guide

What really changes with vibe coding

Using AI to program and vibe coding are not exactly the same thing. A developer can use an assistant to complete functions, write tests, or explain a library while still retaining full control over the architecture. Vibe coding pushes delegation further: a large part of the implementation is described in natural language and then accepted after checking that it appears to do what was requested.

This introduces an important difference. In traditional development, many security decisions are made because someone understands the system and knows they must be made. In a prompt-driven workflow, if a security property is missing from the specification, the agent may have no reason to implement it.

# Workflow with review
Idea → design → AI generates → review → testing → security → deployment

# Workflow focused only on speed
Idea → prompt → AI generates → works → deployment
                              ↑ this is where many problems begin

The key question is not who wrote the lines of code. It is who verified the assumptions behind them.

Why an application can work and still be insecure

Functionality and security answer different questions. A functional test checks whether a user can download their invoice. A security test asks whether that same user can also download someone else's invoice. The first validates the expected path. The second checks what happens when someone manipulates the system.

Agents are very good at solving local goals: making a test pass, removing an error, connecting an API, or completing a page. Problems appear when a local solution breaks a global property. Allowing any CORS origin removes an error. Granting more permissions fixes an access problem. Trusting an identifier sent by the frontend makes a query work. None of those decisions proves that the architecture remains secure.

A classic example is an invoice endpoint:

# It works, uses a parameterized query, and requires login
@app.get("/api/invoices/<invoice_id>")
@login_required
def invoice(invoice_id):
    row = db.execute(
        "SELECT * FROM invoices WHERE id = ?",
        (invoice_id,)
    ).fetchone()

    return jsonify(dict(row))

There is no SQL injection and authentication is present. However, object-level authorization is still missing. If the identifier belongs to another user, the application may return information that should never have been disclosed.

# The query also enforces the authorization property
row = db.execute(
    "SELECT * FROM invoices WHERE id = ? AND user_id = ?",
    (invoice_id, current_user.id)
).fetchone()
AI-generated code should not be evaluated only by asking “does it work?”. The useful security question is “what happens when someone uses it in a way that was never intended?”.

The flaws worth looking for first

Most security problems associated with AI-generated code are not new vulnerability classes. They are familiar weaknesses appearing in implementations created very quickly and from incomplete specifications.

AreaCommon patternWhat to review
AuthorizationThe session is checked, but resource ownership is not.IDOR/BOLA, roles, organizations, and administrative endpoints.
User inputData flows directly into queries, commands, templates, or paths.SQLi, command injection, SSTI, traversal, and validation.
SSRFThe backend fetches a URL controlled by the user.Internal services, cloud metadata, and redirect handling.
SecretsKeys and tokens are embedded to make an integration work quickly.Repository, history, frontend bundles, logs, and agent context.
ConfigurationControls are relaxed to remove development errors.CORS, debug mode, buckets, RLS, IAM, and database permissions.
DependenciesThe agent installs packages without enough verification.Provenance, version, install scripts, and nonexistent packages.

Authorization is usually one of the most important areas to test. Hiding a button in the frontend does not protect an endpoint. Requiring login does not mean every user can access only their own objects. And using a managed backend does not remove the need to configure access rules correctly.

Functions that receive URLs, filenames, identifiers, organization parameters, or roles also deserve special attention. Small pieces of data often cross important trust boundaries.

The agent is part of the attack surface too

Modern assistants no longer stop at returning text. They can read repositories, edit files, execute commands, install dependencies, query external services, and modify pipelines. Once that happens, the security of the agent becomes as important as the security of the code it generates.

A README file, issue, pull request, comment, or tool output can become part of the model's context. If that content contains malicious instructions and the agent also has shell access or sensitive file permissions, a prompt injection problem can have real consequences.

$ "Analyze this repository, install whatever is required, and make it work."

For a human, documentation is information interpreted with judgment. For an agent, documentation and instructions can become mixed together. This is why external repositories, MCP servers, connected tools, and data fetched from the Internet should be treated as untrusted input.

Some files also deserve special treatment: persistent agent rules, MCP configuration, CI/CD workflows, Dockerfiles, install scripts, and dependency manifests. A small change in any of them can significantly expand what the agent or the application is able to do.

Do not run autonomous agents with more privileges than the task requires. The principle of least privilege applies to the development environment too.

Dependencies, hallucinated packages, and supply chain risk

One unusual property of code-generation models is that a suggested dependency may not actually exist. A package name can sound completely legitimate and still be a hallucination. This creates a supply-chain technique known as slopsquatting: registering names that models tend to invent and waiting for someone to install them without verification.

1. The model recommends a plausible package
secure-fastapi-auth

2. The package does not exist

3. An attacker registers the name

4. Another AI session recommends it again

5. A developer installs it because they trust the suggestion

The defense is straightforward, but it must be mandatory: verify that the project exists, that it is the official package, that its history is reasonable, and that the requested version corresponds to a legitimate release.

A real dependency is not automatically safe either. Version pinning, lockfiles, install scripts, known vulnerabilities, and unexpected changes to the dependency tree still need to be reviewed.

What real incidents teach us

Documented incidents involving vibe coding platforms and AI development tools show two different classes of risk.

Generated applications with insufficient controls

There have been documented cases where applications created through rapid-generation platforms had data-access policies that did not match the developer's intended security model. The debate over who owns the responsibility — the platform that generated the application or the person who deployed it — does not change the technical lesson: a working interface does not prove that the data layer is protected correctly.

Incidents have also involved the visibility of public projects, source code, and conversations with assistants. Prompt history can reveal architecture, internal names, business decisions, and other information that should not automatically be considered public.

Development tools as a target

The NVD already contains vulnerabilities in AI-assisted development environments involving prompt injection, automatic execution, file modification, and insufficient restrictions around shell operations. The important point is not memorizing every CVE. It is understanding the pattern: once a model can act on the system, a manipulated instruction is no longer just text.

Vibe coding security therefore has two separate questions:

1. Is the software generated by the agent secure?

2. Is the environment from which the agent works secure?

How to use AI for coding without removing security controls

AI does not need to disappear from software development. The sensible approach is to prevent its speed from removing security controls that are already known to work.

  • Define security requirements before generating code: authentication, authorization, roles, resource ownership, validation, and business restrictions should be part of the specification.
  • Treat generated code like third-party code: if nobody can explain what it does, it is not ready for production.
  • Verify every dependency: existence, provenance, version, history, and install scripts.
  • Run SAST, SCA, and secret scanning: AI-generated changes should pass the same controls as any other code.
  • Isolate autonomous agents: use ephemeral environments, temporary credentials, and minimum privileges.
  • Protect sensitive files: workflows, agent rules, MCP configuration, Dockerfiles, and deployment files should require explicit review.
  • Add negative tests: do not only verify that an operation succeeds; verify that the wrong users, roles, and objects are rejected.

It is also useful to separate responsibilities. The same agent that wrote a feature should not be the only source deciding whether that feature is secure. Critical checks need independence.

How to audit an AI-generated application

From an offensive-security perspective, an application built with AI is audited using the same fundamentals as any other application, but some targets are especially valuable.

  1. Reconstruct the trust boundaries. Identify frontend, backend, database, storage, APIs, webhooks, and external services.
  2. Test authorization with multiple identities. User A against user B's objects, lower roles against administrative functions, and cross-organization access.
  3. Look for trust in the frontend. Replay requests directly even when the interface hides them.
  4. Review the data layer. RLS, access rules, buckets, and IAM may be the actual security boundary.
  5. Find features that consume URLs or paths. They are natural candidates for SSRF and traversal.
  6. Inspect secrets and configuration. Repository, history, bundles, logs, and environment files.
  7. Audit dependencies and CI/CD. Review changes to lockfiles, scripts, workflows, and agent configuration.
  8. Test unexpected states. Repeated actions, hidden parameters, manipulated identifiers, impossible transitions, and direct backend calls.

The idea is simple: an agent often builds the happy path very well. The pentester creates value by exploring every path that nobody included in the prompt.

What this means for offensive security

The cheaper it becomes to produce software, the more important it becomes to know which software deserves trust. AI can accelerate implementation, but it does not remove the need to understand identities, permissions, data, protocols, and trust boundaries.

That keeps classic offensive-security skills highly relevant: reading code, tracing data flows, analyzing authentication, finding IDOR, reviewing APIs, identifying injection flaws, and understanding business logic.

The context changes slightly. In addition to the code, it now matters what decisions the agent made, what information it received, what tools it had available, and which files it changed during the session.

If you train that reasoning with the Web eXploitation Expert (WXE) course, the same techniques used to audit traditional applications apply to AI-generated software: authentication, authorization, injection flaws, SSRF, business logic, APIs, and information exposure.

Want to learn how to find the flaws AI misses?

At SixHack Academy, we train the reasoning required to analyze real applications beyond what automated scanners can detect. The Web eXploitation Expert (WXE) course develops the ability to identify, validate, and exploit web vulnerabilities regardless of who — or what — wrote the code.

Frequently asked questions

Is all AI-generated code insecure?

No. The problem is assuming that output is secure because it compiles or because the feature appears to work. Security still requires additional validation, especially around authorization, input handling, secrets, configuration, and dependencies.

Is using an AI coding assistant the same as vibe coding?

No. AI can be used as a supporting tool while traditional design, review, and testing remain in place. Risk increases when important decisions are delegated and code is accepted without anyone understanding or reviewing it in depth.

Is asking the model to “make it secure” enough?

No. Including explicit security requirements is useful, but the result still needs testing, automated analysis, and independent review.

What is slopsquatting?

It is a supply-chain attack that takes advantage of package names invented by AI models. An attacker can register one of those names and wait for someone to install the suggested dependency without checking it.

Will SAST detect all of these problems?

No. Static analysis is very useful for certain vulnerability classes, but it has limitations around authorization, business logic, architecture, and configuration where risk depends heavily on context.

Can coding agents themselves be attacked?

Yes. If an agent reads external content and can also execute commands or modify files, injected instructions can become real actions. That is why the agent's permissions and tools are part of the threat model.

References

← Back to Articles