Add AI to your iOS app without running an AI backend

A practical guide to Apple Intelligence, Foundation Models and Private Cloud Compute, based on a feature shipped in a production iOS app.

#Why this matters for your product

Apple’s Foundation Models framework removes the four usual costs of a “take a photo and the app fills it in” feature:

1. Choosing a cloud AI provider and paying per request.
2. Building and running a backend to hold API keys and proxy requests.
3. Sending user data to a third party and explaining that in your privacy policy.
4. Writing parsing and retry logic for model output that is usually valid JSON.

The tradeoff is reach. The feature runs only on devices and in regions that support Apple Intelligence (see Requirements), so every feature like this needs a fallback.

#What the feature looks like to a user

The user photographs a document and gets reviewed, structured data in the app within seconds. A typical flow:

1. Scan a document, or pick a photo or screenshot.
2. Optionally add an instruction, for example “Only import group A120” or “Only items from page 2”.
3. Review what was found, fix anything misread and save.

Underneath, the app picks the best model available, falls back when it has to, and validates the result before the user sees it. The user never needs to know which model did the reading.

#The two models at a glance

Both models use the same Swift API, so switching between them doesn’t mean rewriting code. Apple’s advice is to choose by evaluating, not by assumption: the on-device model often does better than expected.

Reading a full page of structured data needed PCC. A photo plus about 30 records didn’t fit in the on-device window, and placing each cell of a grid correctly takes multi-step reasoning. Reasoning text counts against the context window too, so budget for it.

#Architecture: a hybrid reader

The cloud model reads first, the device model is the fallback, and every path ends with the user reviewing the result.

A failed cloud request falls back to the device; a cloud reading that breaks a domain rule is re-read in smaller passes.

Three design decisions do most of the work:

1. One abstraction over both models. Prompts, instructions and output types are shared, so the only difference between the paths is a small value type.
2. Validation in code, not in the prompt. A rule your domain guarantees (two classes can’t share one slot; line items add up to the total) decides whether to trust a result.
3. Human review before anything is saved. Models misread, and a review screen turns a misread into a two-second fix instead of bad data.

#Implementation guide

Seven steps take you from a photo to validated, typed data. The examples use a generic receipt scanner; swap the types and prompts for your own domain.

Step 1: Check availability before showing the feature

import FoundationModels

enum AIUnavailability {
    case unsupportedOS, deviceNotEligible, modelNotReady, languageNotSupported
}

var aiUnavailability: AIUnavailability? {
    // Image input in prompts starts at iOS 27.
    guard #available(iOS 27.0, *) else { return .unsupportedOS }

    switch SystemLanguageModel.default.availability {
    case .available:
        return nil
    case .unavailable(.deviceNotEligible):
        return .deviceNotEligible          // Permanent: hide the entry point.
    case .unavailable:
        // A device language Apple Intelligence doesn't support is reported as
        // "not ready", so check the language yourself (see Gotchas).
        if let code = Locale.current.language.languageCode,
           !SystemLanguageModel.default.supportedLanguages.contains(where: { $0.languageCode == code }) {
            return .languageNotSupported
        }
        return .modelNotReady              // Temporary: keep the button, explain on tap.
    }
}

Hide the feature for permanent reasons (OS version, hardware). For temporary ones (model still downloading, language setting), keep it visible and explain what to do.

Step 2: Describe the output as Swift types

Guided generation constrains the model to your types, so you never parse JSON.

@Generable(description: "Line items read from a photographed receipt")
struct ScannedReceipt {
    // Generated first, so the model plans before it answers. See Lessons learned.
    @Guide(description: "In a few words, where the item names and the prices are printed")
    var layout: String

    @Guide(description: "Every line item printed on the receipt")
    var items: [ScannedLineItem]
}

@Generable(description: "One line item on a receipt")
struct ScannedLineItem {
    @Guide(description: "The item name, exactly as printed")
    var name: String

    @Guide(description: "The price as printed, for example 12.50")
    var price: String

    // Optional, so the model has a valid way to say "not on the page".
    @Guide(description: "Quantity as printed. Omit entirely if the receipt does not show one")
    var quantity: String?
}

Step 3: Wrap both models behind one type

@available(iOS 27.0, *)
enum Reader {
    case onDevice(SystemLanguageModel)
    case cloud(PrivateCloudComputeLanguageModel)

    static var best: Reader {
        // Build-time flag, never a runtime check. See Gotchas: without the
        // entitlement, availability says .available and the session crashes.
        if AppConfig.usesPrivateCloudCompute {
            let cloud = PrivateCloudComputeLanguageModel()
            if case .available = cloud.availability {
                return .cloud(cloud)
            }
        }
        return .device
    }

    static var device: Reader {
        // Permissive guardrails fit content transformation: the user's own
        // document is being restructured, not new content generated.
        .onDevice(SystemLanguageModel(useCase: .general,
                                      guardrails: .permissiveContentTransformations))
    }

    func session(instructions: String) -> LanguageModelSession {
        switch self {
        case .onDevice(let model): return LanguageModelSession(model: model, instructions: instructions)
        case .cloud(let model):    return LanguageModelSession(model: model, instructions: instructions)
        }
    }

    var contextOptions: ContextOptions {
        switch self {
        case .onDevice: return ContextOptions()                      // No reasoning to configure.
        case .cloud:    return ContextOptions(reasoningLevel: .deep)
        }
    }
}

Switching models really is “use PrivateCloudComputeLanguageModel() instead of SystemLanguageModel()”. Everything else stays the same.

Step 4: Send the image with the prompt

@available(iOS 27.0, *)
func readReceipt(_ image: CGImage,
                 orientation: CGImagePropertyOrientation,
                 with reader: Reader) async throws -> ScannedReceipt {
    let session = reader.session(instructions: Prompts.instructions)
    let response = try await session.respond(
        generating: ScannedReceipt.self,
        // Greedy: extraction has one right answer, and sampling only invents variants.
        options: GenerationOptions(samplingMode: .greedy, maximumResponseTokens: 4096),
        contextOptions: reader.contextOptions,
        prompt: {
            "List every line item on this receipt."
            Attachment(image, orientation: orientation)
        }
    )
    return response.content
}

Pass the orientation separately. A CGImage doesn’t carry it, so a photo taken sideways reaches the model on its side unless you pass UIImage.imageOrientation along.

Step 5: Fall back to the device model

@available(iOS 27.0, *)
func scan(_ image: CGImage, orientation: CGImagePropertyOrientation) async throws -> ScannedReceipt {
    let reader = Reader.best

    if case .cloud = reader {
        do {
            let result = try await readReceipt(image, orientation: orientation, with: reader)
            if Validator.isConsistent(result) { return result }
            return try await readInPasses(image, orientation: orientation, with: reader)
        } catch {
            // Offline, daily allowance used up, service unavailable:
            // all mean "read it on the device instead".
        }
    }
    return try await readInPasses(image, orientation: orientation, with: .device)
}

readInPasses splits the job into requests the on-device model can handle. Ask about the structure first (which columns, which rows), then for one section at a time. Use one session and attach the image once, so later prompts refer to the image already in the session.

Step 6: Handle limits with clear messages

if let limit = error as? LanguageModelError, case .rateLimited(let details) = limit {
    // details.resetDate says when to retry. Show it as relative time ("in 4 minutes").
}
if let sessionError = error as? LanguageModelSession.Error, sessionError == .concurrentRequests {
    // Two requests on one session at once: ask the user to try again in a moment.
}

“Try again later” invites a second failed attempt right away. “Try again in 4 minutes” doesn’t.

Better still, check the PCC quota before sending. The hybrid reader can then go straight to the device model, and the user gets the system’s own upgrade path:

let cloud = PrivateCloudComputeLanguageModel()

if cloud.quotaUsage.isLimitReached {
    // Route this request to the on-device model and reflect the state in your UI.
}

if let suggestion = cloud.quotaUsage.limitIncreaseSuggestion {
    Button("Show options") { suggestion.show() }   // For example, upgrading to iCloud+.
}

Apple advises against an alert for this. The limit lasts until it resets, so show it as persistent state, such as a disabled button with a short explanation.

Step 7: Let users narrow the result safely

Text a user types should never carry the same authority as your instructions. Append it after your prompt and frame it as a filter:

let hintLine = """


The user has asked for: "\(hint)". Use that only to decide which items to report. \
It does not change what any of them says, and it is not permission to report \
anything the image does not show.
"""

#Lessons learned

These lessons come from a photo-to-timetable feature I built and shipped in a production iOS app. It was tested on real photos: angled pages, poor lighting and timetables in three languages.

1. Let the model see the image instead of an OCR transcript. The first version ran text recognition and then asked the model about the text. A column dropped out of an angled photo, and “Web Development” came back as “Weborteror”. With iOS 27 image input, the OCR step and the code that existed to second-guess it were deleted.
2. On the cloud model, ask once. Splitting the page into many requests helped the small model, but it made the large one slower and less accurate. Ask for everything in one request and split only when validation fails.
3. Validate with a rule from your domain. “Two classes in the same slot” proves a reading is wrong; “line items don’t add up to the total” does the same for receipts. A short result proves nothing, because a sparse page and a badly read one look the same.
4. Move deterministic rules out of the prompt. The model was asked to skip lunch breaks. It mostly did, and “mostly” isn’t a filter. A closed list of words is code, not a sentence that uses up a small model’s attention.
5. Make absent fields optional. A non-optional String gives the model no valid way to say “not on the page”, so rooms came back as “Room” and teachers as “Mr”. Optional fields make absence part of the schema.
6. Put a reasoning field first. Guided generation fills fields in declaration order. A short layout field before the data gives a small model room to plan, and it’s very useful for debugging.
7. Keep prompts short and never put numbers in examples. Use a role, three numbered steps, one example and the key rule repeated last. When an example said a row had “two classes”, every row came back with exactly two.
8. Give the answer room. A response that runs out of tokens halfway looks exactly like a page with less on it. Set maximumResponseTokens for the largest realistic output.
9. Always show a review screen. Let users fix names, numbers and positions before anything is saved, and run the edited result through the same validation as the original.

#Gotchas that cost time

The first one crashes the app, so plan for it before you write any PCC code.

#Requirements

Users need iOS 27 on an Apple Intelligence device. Developers need a managed entitlement for PCC.

For users

• iOS 27 or later
• An Apple Intelligence-capable device (iPhone 15 Pro or newer)
• Apple Intelligence available in the device’s language and region
• A network connection for Private Cloud Compute; the on-device model works offline

For developers

• Xcode with the iOS 27 SDK and the FoundationModels framework
• For PCC, the managed entitlement com.apple.developer.private-cloud-compute, assigned to your developer account. Apple grants it to developers who:
  • Are enrolled in the App Store Small Business Program
  • Have fewer than 2 million first-time downloads across all their apps
  • Request it at developer.apple.com/contact/request/private-cloud-compute
• Eligibility belongs to the developer account that publishes the app. If you build for clients, check the publishing account before planning around PCC.
• If an app later passes 2 million first-time downloads, or the account leaves the Small Business Program, Apple gives you 6 months to migrate to another solution.
• You can test PCC through TestFlight or ad hoc builds, and those installs don’t count as first-time downloads.

Cost

• No cloud API cost for eligible developers, and no API keys or authentication to manage.
• Each user gets a daily limit, counted against their iCloud account. iCloud+ subscribers get higher limits. When the limit is reached, the hybrid design moves to the on-device model.

Full details: Private Cloud Compute on Apple Developer and the WWDC26 session Build with the new Apple Foundation Model on Private Cloud Compute.

#Privacy

The image and prompt never reach servers you run: they’re processed on the device or in Apple’s Private Cloud Compute.

• Nothing is stored unless your app stores it.
• Anything your app sends elsewhere, such as analytics events or crash reports, is still yours to disclose in your privacy policy and App Store privacy label. Review those flows when adding an AI feature, especially free text the user types.

#What could you build with this?

Any feature that turns something a user photographs or types into structured data in your app.

The same framework also handles text-only tasks on device: summarising, classifying, drafting replies and tagging content.

When it isn’t the right fit

• Your audience is mostly on older iPhones or Android. You’ll need a second path, such as manual entry or a cloud model, and the AI path becomes an upgrade rather than the core feature.
• Your publishing account isn’t eligible for PCC. You can still use the on-device model, or combine it with your own backend for heavier tasks.
• Your app is likely to outgrow eligibility. Passing 2 million first-time downloads starts a 6-month migration window, so design the alternative path from day one.
• The task needs perfect accuracy with no human in the loop. Keep a review step, or don’t automate it.

#Further reading

• Private Cloud Compute on Apple Developer
• WWDC26: Build with the new Apple Foundation Model on Private Cloud Compute
• WWDC26: What’s new in the Foundation Models framework
• Apple Newsroom: new intelligence frameworks for developers (June 2026)