Privacy and architecture

Memsearch keeps its vector index and thumbnail cache on the Mac. It is not an offline-only system: supported content and every search query are sent to Google Gemini to create embeddings.

Component data flow

Electron desktop ─┐
Raycast extension ├── HTTP ──> FastAPI on 127.0.0.1:7242
CLI / curl ───────┘                 │
                                    ├── local files and PDF/image processors
                                    ├── PhotoKit + read-only Photos.sqlite
                                    ├── Google Gemini embedding API
                                    └── local ChromaDB + thumbnail cache

The Electron app manages a backend child process after onboarding. Raycast and other clients use the same HTTP API. The CLI can either access storage directly for its commands or start the HTTP server.

Indexing flow

1. The backend scans an allowed folder or enumerates image assets through PhotoKit.
2. A local processor extracts text or prepares JPEG image content.
3. The prepared content is sent to Gemini with the RETRIEVAL_DOCUMENT task.
4. Gemini returns a vector with the configured dimensions.
5. ChromaDB stores the vector plus local metadata such as path/asset ID, filename, modality, summary, size, time, thumbnail filename, and optional people names.
6. Derived 256 × 256 JPEG thumbnails are saved to the local thumbnail directory.

Search flow

1. FastAPI removes @person mentions and retains them for local filtering.
2. The remaining query—or photo for a people-only query—is sent to Gemini with RETRIEVAL_QUERY.
3. ChromaDB performs cosine nearest-neighbor search, optionally filtering by modality.
4. Person filters are applied locally against stored names.
5. The client receives result metadata and fetches local thumbnails as needed.

What is sent to Gemini

The embedding calls do not intentionally include the filesystem path or filename as a separate argument. Result summaries and metadata stay in ChromaDB. For Photos, person names are appended to local metadata after the image embedding is returned.

Data locations

All ~ paths resolve under the current macOS user account. Storage locations can be overridden with the corresponding MEMSEARCH_* environment settings.

ChromaDB contents

Each record uses a SHA-256 hash of the file path or photos:// identifier as its Chroma ID. Stored metadata can include:

file_path / photos:// identifier
modality
filename or generated photo display name
text_summary (up to 1,000 characters in storage)
file_size
indexed_at
thumbnail_filename
person_names

Text passed as the Chroma document field is the same stored summary or filename fallback—not the entire original file. The embedding itself can nevertheless encode information from the larger prepared payload sent to Gemini.

Photos access

PhotoKit permission gates asset enumeration and image requests. Network access is enabled so iCloud-only assets can be downloaded. The people feature opens Photos.sqlite in read-only/query-only mode and does not modify the Photos library database.

Opening a result runs AppleScript against Photos using the local asset identifier. Memsearch does not export the photo to a separate permanent source-file location.

API-key handling

The desktop app currently stores the Gemini key in Electron Store under apiKey as plaintext. On backend launch it copies the key into the child process environment as MEMSEARCH_GEMINI_API_KEY.

Legacy builds may have stored a safeStorage-encrypted base64 value. Current code attempts to decrypt such a value once, validates that it resembles a Gemini key, and rewrites it as plaintext. An undecryptable legacy value is discarded.

The backend GET /config response does not expose the key.

Logs

When the desktop app manages the backend, stdout and stderr are appended to:

~/Library/Application Support/MemSearch/logs/backend.log

The log records backend starts, dependency/install progress, process exits, file/photo progress, and error messages. Indexing output can contain filenames, file paths, photo labels/dates, and service errors. Review or redact the file before sharing it.

The backend does not implement log rotation in the current source; the Electron process opens the file in append mode.

Network surfaces

The FastAPI service itself defaults to loopback, but it has no authentication and enables wildcard CORS. See Local API before changing its host.

Clearing and retention

DELETE /index, the desktop Clear Index action, and CLI clear remove the Chroma collection and cached thumbnails. They do not remove:

• original files or Photos assets;
• backend/Electron settings;
• the stored Gemini key;
• the Python environment; or
• backend logs.

Use the reset procedures in Troubleshooting when you need to remove more than search data.