Frontier Agent Interaction on iMessage: Tech Overview

Preface

At Photon, we keep asking ourselves a simple question: what should AI and agents actually look like in the future?

Will we really open a browser, type in a URL, and use agents as if they were just another SaaS tool? Will we download yet another app just to “chat with an AI boyfriend or girlfriend”? This feels unexciting and it certainly does not match the futures we see in sci-fi scenes.

We believe that in the world we are heading toward, AI should not appear as a “feature” or a “tool.” It should feel like a type form of life, deeply woven into our social structures. Our kids will not be surprised when they see AI as we were. They will not treat it as a cold, external program as we treat our macs and phones. They will look at AI the same way they look at friends and classmates.

In that world, AI becomes a first‑class citizen in our society.

With that in mind, we started asking what we can build today that moves us in that direction. One answer we kept coming back to was iMessage.

In the United States, almost everyone uses iMessage every day. Millions of messages flow through it constantly. It might be the most natural and native interaction surface for agents in this era: the agent shows up in your conversation list like a friend, and even joins your group chats.

So we decided to turn this idea into reality and build infrastructure that lets AI exist in a truly “human” way inside iMessage.

That is how we arrived at imessage-kit - an open‑source TypeScript SDK for controlling iMessage. It lets developers send, receive, and orchestrate iMessage messages through code.

Along the way, we had to work through a lot of technical constraints and ended up re‑imagining how AI can communicate with people.

This blog focuses on the technical side of that journey: how iMessage works under the hood, what it takes to build a reliable SDK around it, and how that unlocks new interaction patterns for agents. We will save more speculative interaction design and UX experiments for future posts.

1. Understanding iMessage’s Data Architecture

1.1 Where the database lives and how it is structured

All iMessage messages are stored in a single SQLite database located here:

This database has existed since the very first versions of iMessage. If you have been using iMessage for years, your chat.db file may already be hundreds of megabytes, or even over 1GB.

Core tables:

These tables together describe who is talking to whom, what was sent, and which files are attached.

1.2 Timestamps: the Mac epoch

The timestamps in this database are not Unix timestamps. For example:

If you do something like new Date(408978598) directly, the result will be wrong.

Key point: macOS uses its own epoch starting from 2001‑01‑01, instead of Unix’s 1970‑01‑01.

Here is the correct conversion pattern:

const MAC_EPOCH = new Date('2001-01-01T00:00:00Z').getTime()

function convertMacTimestamp(timestamp: number): Date {
    // macOS stores timestamps in nanoseconds, so divide by 1,000,000
    return new Date(MAC_EPOCH + timestamp / 1000000)
}

This “Mac epoch” style is standard across macOS and iOS (Core Data timestamps). Once you get used to it, it becomes straightforward.

1.3 Message body encoding: NSAttributedString

Text in iMessage is stored across two fields:

• message.text: plain text

• message.attributedBody: rich text, stored as a binary plist representing an NSAttributedString

In imessage-kit, we use two strategies to parse attributedBody.

Strategy 1: Direct string extraction (fast but a bit rough)

const bufferStr = buffer.toString('utf8')
// Match readable characters (ASCII + Chinese)
const readableMatches = bufferStr.match(/[\\x20-\\x7E\\u4e00-\\u9fff]{5,}/g)

We use a regex to pull out human‑readable text from the binary, and then filter out plist keywords such as NSAttributedString, NSDictionary, and so on.

Strategy 2: Use plutil (precise but slower)

plutil -convert xml1 -o - "temp.plist"

macOS ships with a command‑line tool plutil that converts binary plists into XML. From there, we can parse out the contents of <string> tags.

Each approach has tradeoffs:

• Strategy 1 is fast but may capture some noisy or broken strings.

• Strategy 2 is very accurate but requires temporary files and shelling out to a system command.

In practice, imessage-kit uses a hybrid approach: try Strategy 1 first, and if it fails, fall back to Strategy 2. This gives a good balance between speed and correctness.

2. Working Around macOS Security

2.1 Full Disk Access

Starting from macOS Mojave (10.14), Apple tightened privacy controls. The ~/Library/Messages folder is now protected. Programs without explicit permission cannot access it.

Typical error message when you try to access:

$ sqlite3 ~/Library/Messages/chat.db
Error: unable to open database "chat.db"

How to fix it:

1. Open System Settings → Privacy & Security → Full Disk Access.

2. Click the “+” button and add your terminal and/or IDE (Terminal, iTerm, VS Code, Cursor, etc.).

3. Restart those apps. This is critical. Permissions do not take effect until you restart.

We recommend adding both your main editor and terminal to avoid permission inconsistency in different environments.

2.2 SQLite WAL mode behavior

The iMessage database uses SQLite’s WAL (Write‑Ahead Logging) mode, so you will see three files:

Important behavior: when new messages arrive, chat.db-wal is updated immediately, but the main chat.db file may take several seconds or minutes to catch up (until a checkpoint is triggered).

This matters a lot for “real‑time” monitoring. If you only watch changes in chat.db, your view of new messages will lag behind.

A better approach is:

1. Use periodic polling instead of file‑system “change” events.

2. Open the database in read‑only mode and let SQLite handle the WAL file for you.

// Correct pattern
const db = new Database(path, { readonly: true })

2.3 Concurrency considerations

SQLite allows concurrent readers but serializes writes. imessage-kit always opens the database in read‑only mode (readonly: true) in order to avoid disturbing Messages.app’s own operations.

3. Sending Messages: AppleScript’s Art and Limitations

3.1 Why AppleScript?

Apple does not provide an official API for iMessage. For automation, the one official option we get is AppleScript - a scripting language introduced in 1993.

A minimal example:

In practice, there are many details hiding behind this simple script.

3.2 String escaping

AppleScript is very picky about special characters. You must escape them properly.

// Incorrect
const text = 'He said "Hello"'
const script = `send "${text}" to targetBuddy`
// This will cause a syntax error.

// Correct
function escapeAppleScriptString(str: string): string {
    return str
        .replace(/\\\\/g, '\\\\\\\\')  // backslash
        .replace(/"/g, '\\\\"')      // double quote
        .replace(/\\n/g, '\\\\n')      // newline
        .replace(/\\r/g, '\\\\r')      // carriage return
        .replace(/\\t/g, '\\\\t')      // tab
}

With this function, you can safely interpolate text into AppleScript templates.

3.3 Working around sandbox constraints

Sending file attachments introduces another challenge: macOS sandboxing.

Problem: Messages.app runs in a sandbox and can only access a limited set of folders (Documents, Downloads, Pictures, etc.). If your file lives elsewhere, the send operation will fail silently or with an error.

Workaround: copy the file temporarily into ~/Pictures.

In imessage-kit, we encapsulate this pattern in a dedicated TempFileManager, which automatically scans for and cleans up files that match the imsg_temp_* pattern in ~/Pictures.

3.4 Choosing the right delay for attachments

Different file sizes need different delays to reliably upload to iCloud:

function calculateFileDelay(filePath: string): number {
    const sizeInMB = getFileSizeInMB(filePath)
    if (sizeInMB < 1) return 2      // < 1 MB: 2s
    if (sizeInMB < 10) return 3     // 1–10 MB: 3s
    return 5                        // > 10 MB: 5s
}

This is implemented in imessage-kit and tuned from real‑world usage.

3.5 Handling chat IDs for group messages

Sending to a group chat is more complex than sending to an individual contact, because you need to address the chat by its chatId:

How do you get chatId?

You can query it from the chat table:

SELECT
    chat.guid AS chat_id,
    chat.display_name AS name,
    (SELECT COUNT(*) FROM chat_handle_join
     WHERE chat_id = chat.ROWID) > 1 AS is_group
FROM chat
WHERE is_group = 1

About chatId formats:

• Group chats use GUIDs like chat45e2b868ce1e43da....

• 1‑on‑1 chats may look like iMessage;+1234567890 or just +1234567890.

• AppleScript may expect forms such as iMessage;+;chat45e2b868..., so you need normalization.

imessage-kit ships with built‑in normalization logic that abstracts away these differences for developers.

4. Real‑Time Monitoring: Polling and Performance

Unlike one‑off automation scripts, an iMessage agent needs to receive messages promptly and respond using an LLM or a tool‑driven policy.

Human‑computer interaction research suggests that a delay of around 500ms is already noticeable and can feel “off.” In practice, we found many pitfalls around real‑time monitoring and ended up relying on polling plus incremental queries.

4.1 Why polling instead of file‑system events?

A common first question is: why not use fs.watch or a similar mechanism to watch for new messages?

There are several reasons:

1. WAL mode introduces lag between the .wal file and chat.db.

2. File‑system events fire on many internal operations and are noisy.

3. Polling with timestamp‑based queries is simpler and more reliable.

A good default polling interval:

// Too fast: wastes CPU
pollInterval: 500

// Too slow: visible latency
pollInterval: 10000

// Sweet spot: 2 seconds
pollInterval: 2000  // default

In imessage-kit, 2 seconds turned out to be a good balance between responsiveness and system overhead.

4.2 Incremental queries and de‑duplication

On each poll, we only fetch messages created after the last check:

// Overlap window: min(1s, poll interval)
const overlapMs = Math.min(1000, this.pollInterval)
const since = new Date(lastCheckTime.getTime() - overlapMs)

const { messages } = await db.getMessages({
    since,
    excludeOwnMessages: false,  // initially include self‑sent messages
})

// De‑duplicate using a Map
const seenMessageIds = new Map<string, number>()
const newMessages = messages.filter(msg => !seenMessageIds.has([msg.id](<http://msg.id>)))

Why have an overlap window at all?

Because clocks and commit orders are not perfect. A small overlap helps make sure messages near the boundary are not missed.

5. Cross‑Runtime Support: Bun vs Node.js

5.1 Choosing database drivers

One fun design choice in imessage-kit is native support for both Bun and Node.js:

async function initDatabase() {
    if (typeof Bun !== 'undefined') {
        // Bun runtime – use built‑in SQLite
        const bunSqlite = await import('bun:sqlite')
        Database = bunSqlite.Database
    } else {
        // Node.js runtime – use better-sqlite3
        const BetterSqlite3 = await import('better-sqlite3')
        Database = BetterSqlite3.default
    }
}

Each runtime has its own strengths:

• Bun (bun:sqlite): built‑in, zero external dependencies, fast startup, smaller memory footprint.

• Node.js (better-sqlite3): mature, battle‑tested, large ecosystem.

5.2 The zero‑dependency benefit of Bun

Using Bun lets you avoid adding an extra database dependency:

// Node.js
"dependencies": {
    "better-sqlite3": "^11.0.0"
}

// Bun
"dependencies": {}

For lightweight tooling or CLIs, this can be a very nice property.

6. Real‑World Examples and Use Cases

After building imessage-kit, the Photon team immediately started dogfooding it. We even used it to take over a real iMessage account and introduce Photon to investors.

Along the way, we added more features to make it feel natural to developers: a more expressive syntax, chainable APIs, and constructs that fit agent workflows.

Here are a few concrete scenarios you can build.

6.1 An auto‑reply bot

You can wire up message content to automated responses, or integrate a full agent for intelligent handling:

// Start watching
await sdk.startWatching({
    onDirectMessage: async (message) => {
        await sdk.message(message)
            .ifFromOthers()
            .matchText(/urgent/i)
            .replyText('Got it — I will take care of this ASAP.')
            .execute()
    }
})

6.2 Message analytics

Using the SDK also makes it easy to analyze your message history:

const result = await sdk.getMessages({
    since: new Date('2024-01-01'),
    limit: 10000,
})

// Compute top senders
const senderCounts = new Map<string, number>()

for (const msg of result.messages) {
    senderCounts.set(
        msg.sender,
        (senderCounts.get(msg.sender) || 0) + 1,
    )
}

const sorted = Array.from(senderCounts.entries())
    .sort((a, b) => b[1] - a[1])
    .slice(0, 10)

console.log('Top 10 most active senders:', sorted)

From here, you can layer on time‑of‑day patterns, group chat activity, attachment statistics, and more.

6.3 Webhook integrations

You can forward iMessage events into other systems such as Slack or Discord:

const sdk = new IMessageSDK({
    webhook: {
        url: '<YOUR_WEBHOOK_URL>',
    },
})

await sdk.startWatching()

This is especially useful for teams that want to centralize critical alerts or customer messages.

6.4 Tracking sent messages

imessage-kit implements an OutgoingMessageManager to keep track of sent messages. Once the watcher is running, you can get the fully populated message object after sending:

// Start the watcher
await sdk.startWatching()

// Send a message and await confirmation
const result = await sdk.send('+1234567890', 'Hello!')

if (result.message) {
    console.log('Message ID:', [result.message.id](<http://result.message.id>))
}

How it works under the hood:

1. Before sending, the SDK creates a MessagePromise with the timestamp, content, and chatId.

2. AppleScript executes the send.

3. The watcher’s polling loop detects the new message in the database.

4. The manager matches it by time and content and resolves the corresponding promise.

The matching logic also normalizes chat IDs (for example, between iMessage;-;recipient and recipient) so you do not have to deal with those edge cases yourself.

6.5 Automatic cleanup for temporary files

To bypass sandboxing, attachments are temporarily copied into ~/Pictures. TempFileManager is responsible for cleaning them up.

Lifecycle:

1. Naming convention: all temp files are prefixed with imsg_temp_.

2. Startup cleanup: on SDK init, any leftover temp files are removed.

3. Periodic cleanup: every 5 minutes, the manager scans and deletes files older than 10 minutes.

4. Shutdown cleanup: on SDK shutdown, all temp files are removed.

const DEFAULT_CONFIG = {
    maxAge: 10 * 60 * 1000,          // keep files for 10 minutes
    cleanupInterval: 5 * 60 * 1000,  // scan every 5 minutes
}

Even if the process crashes, the next run will reclaim old files.

6.6 De‑duping incoming messages

The watcher uses a Map<string, number> to track processed message IDs and avoid double‑handling:

private seenMessageIds = new Map<string, number>()

// On each poll
const newMessages = messages.filter(msg => !this.seenMessageIds.has([msg.id](<http://msg.id>)))

for (const msg of newMessages) {
    this.seenMessageIds.set([msg.id](<http://msg.id>), [Date.now](<http://Date.now>)())
}

// Periodically trim (keep roughly 1 hour of history)
if (this.seenMessageIds.size > 10000) {
    const hourAgo = [Date.now](<http://Date.now>)() - 3600000

    for (const [id, timestamp] of this.seenMessageIds.entries()) {
        if (timestamp < hourAgo) {
            this.seenMessageIds.delete(id)
        }
    }
}

Key points:

• Use Map (with timestamps) instead of Set to support time‑based cleanup.

• Clean only when the map exceeds a threshold.

• Keep about an hour of history to cap memory usage.

• Combine with a 1‑second overlap window in polling to avoid edge‑case misses.

7. Pitfalls and How We Solved Them

7.1 Getting the message object right after sending

Problem: After calling send(), you cannot immediately get the full message object. The return value can be undefined.

Reason: AppleScript sends messages asynchronously, and it takes some time for the message to land in the database.

Solution:

await sdk.startWatching()

const result = await sdk.send('+1234567890', 'Hello!')

if (result.message) {
    console.log('Message ID:', [result.message.id](<http://result.message.id>))
}

imessage-kit’s OutgoingMessageManager uses timestamps and content to link sent messages back to database entries.

7.2 ~ in attachment paths

Attachment paths inside the database often include a leading ~:

Solution:

import { homedir } from 'os'

const fullPath = rawPath.startsWith('~')
    ? rawPath.replace(/^~/, homedir())
    : rawPath

This ensures your script can correctly locate files on disk.

8. Performance Tuning

8.1 Query optimization

A naïve query can be very slow:

-- Bad: full table scan
SELECT * FROM message WHERE text LIKE '%keyword%'

A better pattern is:

-- Better: use a time range plus limit
SELECT * FROM message
WHERE date >= ? AND text LIKE '%keyword%'
ORDER BY date DESC
LIMIT 100

Restricting by time, ordering by recency, and limiting rows are usually enough for interactive tooling.

8.2 Controlling send concurrency

When sending many messages concurrently, a semaphore helps protect Messages.app from being overloaded:

class Semaphore {
    private running = 0
    private waiting: Array<() => void> = []

    constructor(private readonly limit: number) {}

    async acquire(): Promise<() => void> {
        while (this.running >= this.limit) {
            await new Promise<void>(resolve => this.waiting.push(resolve))
        }
        this.running++

        // Release function
        return () => {
            this.running--
            const next = this.waiting.shift()
            if (next) next()
        }
    }

    async run<T>(fn: () => Promise<T>): Promise<T> {
        const release = await this.acquire()
        try {
            return await fn()
        } finally {
            release()
        }
    }
}

// Usage
const sem = new Semaphore(5)  // cap concurrency at 5
await [sem.run](<http://sem.run>)(() => sendMessage(...))

imessage-kit includes built‑in concurrency limits (default concurrency 5) so you do not have to hand‑roll this every time.

8.3 Long‑running memory management

For long‑running watcher processes, the map of processed message IDs can grow indefinitely. imessage-kit uses an auto‑cleanup strategy:

if (this.seenMessageIds.size > 10000) {
    const hourAgo = [Date.now](<http://Date.now>)() - 3600000

    for (const [id, timestamp] of this.seenMessageIds.entries()) {
        if (timestamp < hourAgo) {
            this.seenMessageIds.delete(id)
        }
    }
}

This keeps memory bounded while still preventing duplicate handling.

9. Limits of the Current Approach

9.1 Known limitations

imessage-kit is powered by AppleScript and local database reads. As an open‑source SDK, it already covers most core operations, but because Apple has not exposed official APIs for iMessage, there are limitations that are hard to break through today, such as:

1. Editing messages — cannot edit messages after sending.

2. Message recalls — cannot recall messages within 2 minutes.

3. Tapbacks — can read reactions (heart, thumbs up, “ha ha”, etc.) but cannot send them.

4. Typing indicators — cannot send or receive typing state.

5. Message effects — cannot send special effects (fireworks, confetti, balloons, etc.).

6. Read receipts — cannot toggle or simulate read/unread status.

7. Stickers — cannot send iMessage stickers.

8. Voice messages — cannot send the waveform‑style voice messages.

On top of this, AppleScript itself has stability and concurrency limits. The whole system also depends on a user’s iCloud account and needs to run on a specific Mac, which constrains scalability.

9.2 Advanced iMessage Kit: a next‑generation foundation

After digging deeply into these limitations, we built Advanced iMessage Kit.

It uses a different system architecture to get rid of AppleScript’s constraints, unlocks nearly the full iMessage features, and runs on a dedicated infrastructure layer designed for higher concurrency and more stable service.

We have open‑sourced parts of Advanced iMessage Kit and would love to see more people bring agents into iMessage.

Conclusion

iMessage automation is a challenging but worth exploring domain. From low‑level database structure and AppleScript quirks, to macOS security and long‑running performance, every layer demands a deep understanding of how the platform works.

To make this accessible, we wrapped everything above into an open‑source, free TypeScript SDK: https://github.com/photon-hq/imessage-kit.

It turns complex iMessage operations into straightforward APIs, so you can build production‑ready workflows in minutes instead of days.

For developers who need more capabilities but do not want to maintain their own infrastructure, we also provide Advanced iMessage Kit. It unlocks more system‑level functionality while dramatically reducing deployment and configuration overhead.

We are continuing to explore what an amazing agent experiences on iMessage should feel like - from pacing and tone, to when an agent should stay quiet, to how long responses should be.

Internally, we are experimenting with even more ambitious interaction patterns. Once they are ready, we will share them with the community.

If you find any issues in https://github.com/photon-hq/imessage-kit, please feel free to open an issue or PR on GitHub. And if this project helps you, a single star is more than enough motivation for us to keep pushing this work forward.