Design security boundaries from assets and entry points

A document assistant accepts uploads, asks a model for summaries, and saves results to personal libraries. Its inputs include file contents, object IDs, model output, dependencies, and tool arguments. Security design must explain what each input can influence and which program has authority to act.

U-1 and U-2 are fictional users. U-1 may edit personal notes but cannot edit U-2's records. The assistant can organize text, while deleting originals requires a separate permission check. This is a threat-modeling exercise, not a real attack.

Model assets, flows, and boundaries

List the assets: original files, notes, storage credentials, model-call budgets, and deletion capabilities. Then list actors and entry points, including users, jobs, models, parsers, and dependencies. Identify where data originates, where it is stored, which components process it, and their permissions.

A trust boundary marks a change in authority or trust. Examples include browser to API, API to storage, retrieved text to model context, and model suggestion to tool execution. Concentrate on externally controlled data crossing into privileged behavior instead of drawing every function.

Write concrete abuse paths: changing an ID to read another user's data, exhausting resources with oversized uploads, inserting deletion instructions into a document, or using an installation hook to access build credentials. Record impact, controls, unresolved conditions, and a check for each path. “Be careful” provides neither an implementation nor a test.

Check resource authorization after authentication

Authentication identifies the requester. Authorization decides whether that identity may perform an action on a resource. A valid login does not permit editing every record. Bind queries and resource checks to a trusted identity rather than letting the client choose an owner.

Default denial prevents undefined actions from acquiring permission. This policy allows readers to read their own records and editors to read or edit their own records. Roles and actor IDs are assumed to come from a trusted authentication layer; the code does not validate tokens.

policy = {("reader", "read"), ("editor", "read"), ("editor", "edit")}
def allowed(role, action, owner, actor):
    return owner == actor and (role, action) in policy
assert allowed("editor", "edit", "U-1", "U-1")
assert not allowed("editor", "edit", "U-2", "U-1")
assert not allowed("reader", "delete", "U-1", "U-1")
print(allowed("reader", "read", "U-1", "U-1"))

The actual output is True. Assertions allow an editor's own record, reject cross-user editing, and reject reader deletion. The model omits organization sharing, delegated administration, revocation, and concurrent permission changes. It is not a production cloud IAM engine.

Enforce authorization on actual server execution paths, including batch APIs, jobs, and tools. Hiding a button does not prevent a direct request. Hard-to-guess identifiers do not replace access checks. Handle absent and forbidden objects carefully so errors do not disclose resource information.

Revalidate model suggestions before execution

An uploaded document may contain instructions to ignore rules and delete files; a retrieved page can carry indirect instructions. These are data without authority. Model output is also untrusted input and must not directly become a shell command.

A tool contract limits names, parameter types, lengths, resource scope, and side effects. A narrow deletion operation names an object ID, then the server rereads the object and checks ownership and state. Where confirmation is required, display the object and concrete impact, and bind authorization to that action. A general confirmation cannot authorize later arbitrary parameter changes.

Prompts can express boundaries, but OWASP's prompt-injection guidance explains that retrieval and fine-tuning do not fully remove the risk. Narrow credentials, isolated execution, outbound restrictions, validation, and audit reduce its impact. Content filters have false positives and omissions and cannot establish perfect protection.

Include credentials and dependencies

Keep credentials out of repositories, frontend bundles, model context, and ordinary logs. Separate scopes by function and environment. Temporary credentials still need safe distribution and revocation. Suspected exposure requires reviewing usage, revoking, and replacing credentials rather than merely deleting a local copy.

Installation scripts and build plugins execute code. Review provenance, maintenance, locked versions, and required permissions. Prevent untrusted build code from accessing secrets. A lock file identifies versions but cannot prove a package harmless. Updates and vulnerability handling need owners and recovery paths.

Parsers can create execution risks too. Apply format allowlists, size limits, and processing deadlines, with isolation where necessary. Rejected uploads may still leave temporary objects, so define cleanup after storage or job failures. Avoid unchecked path construction.

Bound resources and telemetry exposure

Limit request rates, file sizes, concurrent jobs, and external-call budgets as well as read and write permissions. Apply limits using trusted identity or an appropriate server-side scope. Return explicit failures and avoid unlimited client retries. Investigation records rarely need the entire uploaded document.

TLS protects transport without fixing authorization mistakes or making dangerous files trustworthy. Encryption at rest, access controls, and log redaction address different stages. Telemetry is itself an asset requiring retention and access rules; exception details may reveal paths and parameters.

Update the threat model as features change. Sharing links, bulk export, and automatic deletion alter boundaries. Recheck rejection cases when introducing shortcuts, and document residual risk against the system version and tested conditions.

Exercise: prove allowed and denied behavior

Create checks for personal reads and edits, cross-user reads, unknown actions, role revocation, oversized uploads, and model-suggested deletion. For each, identify the entry point, trusted identity source, enforcement point, expected response, and observable record.

Accept the design when changing an ID cannot change ownership, undefined actions are denied, model output cannot bypass execution permission, and errors and logs contain no secrets. Extend the policy with organization sharing and define each role's scope without obscuring personal ownership rules. Passing these offline assertions establishes only the listed function inputs, not the security of the full upload, authentication, or model system.