# Project: a teaching-illustration job queue and asset library

Canonical page: https://yishto.com/en/learn/project-labs/lab-illustration-job-queue/

Author: [Yishto](https://yishto.com/en/about/)

Design owned illustration jobs with budgets, a quota ledger, R2 assets, and human review, then handle cancellation and repeated execution.

Published: 2026-11-10
Updated: 2026-11-10

A teaching-illustration project turns image requests into jobs with budgets, ownership, and review records. A user submits a brief, the system accepts and executes the job, saves candidate assets, and lets the user choose what to approve. Generation success is separate from approval, and failures do not necessarily make retries free.

Start with one illustration use case and one model adapter. This example uses synthetic jobs and internal quota rather than actual pricing, payments, or provider refunds. Complete the request-to-asset path before adding quota purchases or public galleries.

## Give jobs and assets separate identities

A job records user, input version, target format, budget, and state. An attempt records a particular execution's provider reference, duration, and error. Assets record storage location, ownership, originating job, dimensions, content type, and review state. One job may produce several candidates.

Waiting, running, awaiting review, approved, failed, and cancelled states must correspond to real behavior. Provider output still needs validation and storage before download becomes available. A cancellation request does not establish that the provider stopped or charged nothing.

Bound input size, formats, and use. Users need appropriate rights for reference material, and results require suitability checks. A public library reads approved versions rather than exposing every candidate by default.

## Account for quota by logical job

Accepting a request creates a unique logical job and reserves internal quota in a transaction. A retry by the same user with the same operation key returns the existing job. The same key with different input conflicts. User scope and payload identity form part of the deduplication contract.

The ledger distinguishes reservation, settlement, and compensation with reasons and job references. Restoring internal quota after failure is a product policy, not evidence of a provider refund. A timed-out provider request may have completed; repeatedly creating paid generations is not a safe default.

Conditional state transitions make settlement or compensation occur once. Separately reading and writing balances across services can race and needs transactions, constraints, and concurrency control. A frontend balance display does not authorize job acceptance.

## Bound queues and execution

A queue separates acceptance from completion. Return a job identity and queryable state rather than requiring the page to remain connected. A transactional outbox preserves message intent; publishers still retry and consumers still deduplicate.

Limit concurrency, input size, time, and attempts. Retry transient failures within budget and stop on permanent invalid inputs. Providers may return an image directly or an asynchronous task reference. Adapters map their behavior to business states without assuming identical API flows.

A worker lease coordinates execution, but an expired worker can still return a result. Check current attempt and state before accepting it. External deduplication depends on provider support; an internal queue cannot promise exactly-once external generation.

## Coordinate R2 objects and records

Save output under controlled object keys and store ownership and status in the database. Object upload and database commit are separate operations. Stage, upload, verify, and confirm, then reconcile unconfirmed objects.

R2 object consistency does not synchronize database records or eliminate CDN caches. A successful upload does not establish a user's download permission. Authorize access using ownership, review state, and policy.

Presigned URLs grant temporary access to a particular object and operation. Their holders can use them while valid; the link does not perform application membership checks or revoke downloaded content. Authorize before signing and keep credentials, private briefs, and access links out of public logs.

## Check quota reservation and compensation offline

The teaching policy reserves one internal unit on acceptance and restores it once on failure. Existing jobs cannot reserve again, and failed jobs cannot compensate twice.

```python
balance = 3
jobs = {}
def submit(job_id):
    global balance
    if job_id in jobs or balance < 1:
        return False
    balance -= 1
    jobs[job_id] = "reserved"
    return True
def fail(job_id):
    global balance
    if jobs.get(job_id) != "reserved":
        return False
    jobs[job_id] = "failed"
    balance += 1
    return True
assert submit("J-1") and not submit("J-1")
assert balance == 2
assert fail("J-1") and not fail("J-1")
assert balance == 3
print(balance, jobs)
```

The output is `3 {'J-1': 'failed'}`. This single-process model implements no actual ledger, payment, queue, R2 access, or concurrency. It has no success-settlement function either. The service still needs success, cancellation, unknown external results, and reconciliation paths.

## Build the interface using fixed results first

Use fixed assets and controlled failures to implement submission, status, and ownership checks, then storage. Explain job stages, failure categories, and retry effects. Image-loading failure is different from generation failure.

Verify asset confirmation and quota transitions before adding a provider. Then introduce model attempts, timeouts, refusals, review, download, and cancellation. Record real platform checks separately; Python output does not verify them.

For quota purchases, test signed events, duplicates, and entitlement changes in a payment test environment. A browser success redirect does not increase balance directly. Test records are not actual revenue.

Asset confirmation checks file signatures and decodable formats rather than trusting an extension. Bound dimensions and decoding resources, and preserve ownership and job references in thumbnails. Authorize public listings and private downloads separately and choose caches by visibility. Making an asset private requires checking existing public caches. Define asset deletion independently from job retention so removing an object does not erase the quota ledger's explanation.

## Exercise: fail between upload and confirmation

Test duplicate operation keys, insufficient quota, generation timeout, database failure after upload, and repeated compensation. Balance changes occur once. Unconfirmed objects stay unavailable, and another owner cannot download them.

Then return a successful old-worker result after cancellation. It must not blindly overwrite cancelled state. Define verification or isolated-candidate handling along with internal quota and external costs. Record feedback, actual state, and reconciliation for each branch.

## Sources & further reading

- [Image generation APIs](https://developers.openai.com/api/docs/guides/image-generation)
- [Object storage concepts](https://developers.cloudflare.com/r2/)
- [Synchronized queue classes](https://docs.python.org/3/library/queue.html)
- [Test payments and integrations](https://docs.stripe.com/testing)

Sources checked on 2026-11-10. Examples are teaching data, not production measurements.

## Related guides

- [Handle load with jobs, caches, and backpressure](https://yishto.com/en/learn/service-data/jobs-caches-and-backpressure/)
- [Handle images, speech, and video inside an application](https://yishto.com/en/learn/ai-foundations/multimodal-image-and-speech/)