Initial diagnosis

The initial incident report identified WindowServer as the likely cause. On the affected machine, WindowServer had grown to 1.79 GiB, used 96–99% of one CPU core, missed watchdog check-ins for 40 seconds, and was terminated along with all GUI sessions.

We tested this hypothesis on a staging Mac. We repeatedly connected and disconnected Screen Sharing clients, ran mutiple concurrent clients, transferred 15 GB of full-frame traffic, generated pointer movement, cycled display sleep 30 times, and changed screensavers across six user sessions. WindowServer reached 711 MiB and returned to 667 MiB after each test.

Because WindowServer returned to its baseline, it was not the source of the leak. Its high resource use was a symptom of system-wide memory pressure.

Kernel memory analysis

We then examined kernel allocation zones using zprint as root:

The growing zone was data.kalloc.1024, which stores 1024-byte pointer-free kernel buffers. It used 359 MB on a recently rebooted machine and 1,539 MB on a machine with seven days of uptime. No other zone differed by more than 21 MB.

Sampling every 30 seconds showed a steady increase of 249 KiB per minute with no decreases. A reboot reset the zone to near zero, after which growth resumed. This rate is about 358 MB per day and can exhaust a 16 GB Mac in approximately 40 days.

The memory is wired, so it cannot be paged out, compressed, or reclaimed under pressure. Terminating the process that triggers the leak does not release existing allocations. In one test, the zone remained at exactly 18,352 K for 12 minutes after we stopped the responsible process. A reboot is required to release the memory.

GUI session count

We next tested whether the number of logged-in GUI sessions affected the leak. The affected machines had multiple sessions, while healthier machines generally had fewer sessions. We rebooted one machine with a single session and measured the allocation rate.

The leak rate was effectively unchanged. Session count was not a factor.

Across all tested conditions—including connection churn, process creation, display sleep, session count, and boot-time load—the rate remained close to 249 KiB per minute. This indicated that a periodic process, rather than workload, was triggering the allocation.

Identifying the triggering process

Because the rate was constant, we investigated periodic background work. We stopped ten Photon daemons at the same time. The leak stopped.

The zone remained at 15,472 K for 24 consecutive samples while the rest of the system continued running. This showed that one of our daemons was triggering the kernel leak.

Three rounds of bisection identified macrocosm-astrolabe, our configuration-management daemon. Restarting it resumed the leak within one minute. Stopping it reduced the rate to zero:

These transitions were measured on one machine during one boot, with only the daemon state changed.

Inspecting leaked allocations

Identifying the triggering process did not identify the kernel allocation site. We attempted to use Apple’s Kernel Debug Kit, but Apple silicon does not support installing the instrumented kernel variants required for backtrace-based zone logging. Zone-logging boot arguments and a macOS virtual machine with an unsealed system volume also did not provide usable backtraces.

The showallocatedzoneelement command can list live element addresses without kernel instrumentation. We forced a panic on the affected Mac, attached LLDB from a second Mac over the local network, and inspected 600 leaked buffers.

The buffers contained pathnames. This is consistent with the allocation size because MAXPATHLEN is 1024 bytes. Every sampled path referred to a shell script.

The same workload executed /usr/bin/tr 891 times, xcrun 495 times, and xcode-select 495 times. None of the sampled buffers contained those paths. All three are compiled binaries. The leak appeared to apply only to scripts.

Reducing the trigger

Astrolabe ran brew list --formula for every managed package once every five seconds to verify installation. Homebrew and several of its helper programs are scripts, and each check created approximately 30 processes. We measured the leak associated with several commands:

Ruby did not trigger the leak. The git shim, xcrun, git, sudo, execution as another user, and scripts with shebang arguments also produced no measurable leak when tested separately. The leak occurred only when scripts launched other scripts.

We reproduced the execution chain with a minimal test. Removing xcrun, git, or the privileged-shebang flag did not change the result. Replacing two nested scripts with a single script stopped the leak.

The following script reproduces the issue:

printf '#!/bin/sh\nexit 0\n'        > /tmp/leaf.sh
printf '#!/bin/sh\n/tmp/leaf.sh\n'  > /tmp/top.sh
chmod +x /tmp/leaf.sh /tmp/top.sh

# as root, before and after:
#   zprint data.kalloc.1024 | awk '/^data.kalloc.1024 /{print $3}'
i=0; while [ $i -lt 1000 ]; do /tmp/top.sh; i=$((i+1)); done

printf '#!/bin/sh\nexit 0\n'        > /tmp/leaf.sh
printf '#!/bin/sh\n/tmp/leaf.sh\n'  > /tmp/top.sh
chmod +x /tmp/leaf.sh /tmp/top.sh

# as root, before and after:
#   zprint data.kalloc.1024 | awk '/^data.kalloc.1024 /{print $3}'
i=0; while [ $i -lt 1000 ]; do /tmp/top.sh; i=$((i+1)); done

printf '#!/bin/sh\nexit 0\n'        > /tmp/leaf.sh
printf '#!/bin/sh\n/tmp/leaf.sh\n'  > /tmp/top.sh
chmod +x /tmp/leaf.sh /tmp/top.sh

# as root, before and after:
#   zprint data.kalloc.1024 | awk '/^data.kalloc.1024 /{print $3}'
i=0; while [ $i -lt 1000 ]; do /tmp/top.sh; i=$((i+1)); done

One thousand executions increase the zone by approximately 1,000 KiB. The allocation remains until reboot.

The leak scales with script nesting depth:

Each additional script launched from a script leaks approximately 1 KiB. A script launching a compiled binary does not produce the same result.

We could not identify the responsible kernel function because the required backtrace tooling is unavailable on this hardware. Based on the buffer contents, the likely sequence is that the kernel resolves a #! interpreter, allocates a MAXPATHLEN buffer for its path, and does not release the buffer when the calling program was itself launched from a script. This is an inference from the observed allocations.

Affected versions and allocation zones

We reproduced the issue at the same rate on macOS 26.5.2 and 26.5.1. On macOS 26.3, data.kalloc.1024 remained nearly empty, but data_shared.kalloc.1024 grew instead.

A macOS 26.3 machine accumulated 12.3 GB in data_shared.kalloc.1024 over 38 days, or approximately 324 MB per day. This is close to the 358 MB per day measured on macOS 26.5.2 under a comparable workload. The evidence indicates that the same defect uses different allocation zones across releases.

Both data.kalloc.1024 and data_shared.kalloc.1024 should therefore be checked when testing affected systems or verifying a fix.

We also reproduced the leak from a standard account without administrative privileges. Any user who can execute shell scripts can consume wired kernel memory, and the memory is not released until reboot.

Mitigation

Our immediate fix was to stop invoking Homebrew when checking whether a formula was installed. Checking the corresponding Cellar directory provides the required result without creating approximately 30 processes:

# thirty-ish processes, ~4 KiB of wired kernel memory
brew list --formula htop

# one syscall, nothing leaked
test -d

# thirty-ish processes, ~4 KiB of wired kernel memory
brew list --formula htop

# one syscall, nothing leaked
test -d

# thirty-ish processes, ~4 KiB of wired kernel memory
brew list --formula htop

# one syscall, nothing leaked
test -d

Setting HOMEBREW_NO_ANALYTICS=1 reduces the remaining leak by about half because Homebrew telemetry launches an additional script-based curl chain.

Homebrew is not the source of the defect. It uses a normal pattern in which scripts invoke other scripts. The kernel should release the associated pathname allocation.

We also added monitoring for Pages wired down. Monitoring WindowServer memory did not provide sufficient warning because it remained near normal until the system was close to failure.

Recovery behavior

We rebooted the fleet. Before we rebooted the original unresponsive machine, it became responsive again without restarting.

Its uptime was 38 days, and the load average had fallen from 250 to 2. The 12 GB leak was still wired, and only 60 MB of memory was free.

The machine recovered because stopping the leak stopped further reduction of available memory. While the leak was active, newly reclaimed pages were immediately consumed by wired allocations, preventing compression and swap from stabilizing the system. After the leak stopped, the compressor reduced 3.5 GB of user memory to 924 MB, swap stored 592 MB, and the system stabilized within its remaining 2.4 GB.

The machine was responsive but remained in an unsafe state. It had 6% free memory and a compression ratio close to 4:1. We rebooted it to release the wired allocations.

For long-running Mac infrastructure, responsiveness alone is not evidence of normal memory state. This issue produced no direct error or crash log and did not identify a responsible user-space process. It accumulated approximately 1 KiB at a time over several weeks, while the first visible symptom incorrectly suggested a WindowServer problem.

About Photon

Photon is a unified API that brings agents to interfaces like iMessage, WhatsApp, Telegram, Slack, and more. It is free to start, and it scales as your agent grows.

Ready to build? Get started at photon.codes · Explore on GitHub · Talk to us about enterprise