Problem Awareness

The advantage of AI-generated content is its novelty every time. However, repeatedly calling the API for the same input can lead to high costs. The tarot card service faces a similar issue. Although we use the OpenAI API for card readings, calling the API for every request isn't practical due to cost and response speed concerns.

However, a simple cache using only the card and orientation as keys is insufficient. Users expect different readings even if they draw the same card. To bridge this gap, we introduced a**caching strategy using a bucket system**.

Buckets: Balancing Diversity and Efficiency

The core idea is simple. We combine 78 cards, upright/reverse orientations, and 10 buckets to create**1,560 unique cache keys**and store AI-generated readings for each key.

78 cards × 2 orientations × 10 buckets = 1,560 unique combinations

Each time a user requests, one of these combinations is randomly selected. The cache key is in the form of`tarot:read:{card}:{direction}:{bucket}`, and subsequent requests with the same key return immediately from Valkey. The OpenAI API is only called when not in the cache.

We added another mechanism. The server randomly selects 4 keywords for each request and delivers them to the AI as context. This means different readings can emerge based on the keywords, even with the same card, orientation, and bucket.

flowchart LR
    subgraph RandomSelect["Random Selection"]
        Card[78 cards]
        Dir[Upright/Reverse]
        Bucket[Bucket 1~10]
        Keywords[4 Keywords]
    end

    subgraph CacheKey["Cache Key"]
        Key["tarot:read:{card}:{dir}:{bucket}"]
    end

    Card --> Key
    Dir --> Key
    Bucket --> Key

    Key --> Valkey[(Valkey)]
    Key -.->|Cache Miss| OpenAI[OpenAI API]
    Keywords -.->|Reading Direction| OpenAI

The entire flow is shown in the sequence diagram below.

sequenceDiagram
    autonumber

    actor Client as Client
    participant Service as TarotService
    participant Cache as Valkey
    participant AI as OpenAI

    Client->>Service: Request Tarot Reading
    Note over Service: Random selection of card / direction / bucket

    Service->>Cache: Cache lookup (`GET`)

    alt Cache Hit
        Cache-->>Service: Return stored result
    else Cache Miss
        Service->>AI: Call OpenAI API
        AI-->>Service: Return reading result ({advice})
        Note over Service: Merge data<br/>(card.name / card.nameKR / keywords)
        Service->>Cache: Store result (`SET`)
    end

    Service-->>Client: Return Final Reading Result

Deployment

The frontend runs on Vercel, whereas the backend operates on a home Kubernetes cluster.

flowchart TD
    subgraph Front["Vercel"]
        Vercel[Tarot Card Next.js App]
    end

    subgraph CICD["CI / CD"]
        GH[GitHub]
        Actions[GitHub Actions]
        Harbor[(Harbor)]
        ArgoCD[ArgoCD]
        S3[(S3)]
    end

    subgraph K8s["Home K8s Cluster"]
        GW[Gateway API]
        Service[Tarot Card Service]
        HPA[HPA: 2~10 replicas]
    end

    %% Deployment pipeline flow
    GH -->|Pushes Git version tag v*.*.*| Actions
    Actions -->|Build/Push images| Harbor
    Harbor -->|Store images| S3
    Harbor -->|Reference images| ArgoCD
    ArgoCD -->|GitOps Deployment| Service

    %% Frontend pipeline flow
    Actions -->|Frontend build/deploy| Vercel

    %% Traffic flow
    Vercel --> |External traffic| GW
    GW -->|Routing| Service
    Service -->|Auto-scale| HPA

The deployment pipeline starts automaticallyonce a new version tag (`v*.*.*`) is pushed to GitHub. GitHub Actions builds the images based on that tag and pushes them to Harbor, our in-house container registry. ArgoCD detects changes through GitOps configuration and automatically synchronizes the cluster state. Additionally, the frontend is directly built and deployed to Vercel via GitHub Actions.Additionally, the backend auto-scales from 2 to 10 replicas based on traffic via HPA, and external user requests are securely routed to the internal Gateway API.

Additionally, the backend auto-scales from 2 to 10 replicas based on traffic via HPA, and external user requests are securely routed to the internal Gateway API.

Future Improvements

Currently, it's a simple service accessible by anyone without login, but we are planning to add login functionality to save user-specific reading histories and provide personalized experiences.

Conclusion

Although the tarot card service is a small project, it embodies the process of addressing practical issues of balancing AI generation and caching. The caching strategy employing a bucket system offers a realistic compromise between cost and variety, with plenty of room for future improvement.