CWE-1426—Improper Validation of Generative AI Output

Weakness Name

Improper Validation of Generative AI Output

Description

The product invokes a generative AI/ML component whose behaviors and outputs cannot be directly controlled, but the product does not validate or insufficiently validates the outputs to ensure that they align with the intended security, content, or privacy policy.

Common Consequences

Scope:

Integrity

Impact:

Execute Unauthorized Code or Commands, Varies by Context

Notes:

In an agent-oriented setting, output could be used to cause unpredictable agent invocation, i.e., to control or influence agents that might be invoked from the output. The impact varies depending on the access that is granted to the tools, such as creating a database or writing files.

Related Weaknesses

• CWE-707Improper Neutralization