Bridging Browser and Desktop: Cross-App Orchestration

Browser AI extensions are confined to the browser. They can read pages, click buttons, and fill forms, but the moment you need to do something outside the browser — run a script, call an API, create a file, or interact with a desktop application — they hit a wall. The browser sandbox is both a security feature and a limitation. It keeps extensions from doing dangerous things, but it also keeps them from doing useful things.

Tensor's Cross-App Orchestration system breaks through this wall. Through a companion native messaging host and our MCP (Model Context Protocol) relay architecture, Tensor orchestrations can seamlessly bridge browser actions with terminal commands, REST API calls, file system operations, and local application control. A single orchestration can scrape data from a website, run a Python script to process it, save the results to a CSV file, and POST a summary to a Slack webhook — all from one natural language instruction.

The MCP Relay Architecture

Chrome extensions communicate with the outside world through a mechanism called Native Messaging. The extension sends JSON messages to a native host application running outside the browser sandbox, and the host can do anything a regular application can do: read files, execute processes, make network requests, and interact with the operating system.

Tensor's native host is a lightweight process we call the MCP Relay. It acts as a bridge between the extension and the local system. When an orchestration step requires a non-browser action, the extension serializes the step as a JSON message, sends it to the MCP Relay, and the relay executes it. The relay supports several step types:

• Shell steps — execute terminal commands and capture stdout/stderr
• HTTP steps — make arbitrary HTTP/HTTPS requests with custom headers, bodies, and authentication
• File steps — read, write, append, or delete files on the local file system
• Script steps — run Python, Node.js, or shell scripts with argument passing
• Browser steps — navigate, click, extract, or fill within browser tabs
• AI steps — send a prompt to the AI model and use the response in subsequent steps

Each step type has a well-defined input/output contract. The output of one step can be piped into the input of the next step using variable interpolation, creating powerful data pipelines that span the browser-desktop boundary.

Variable Interpolation

The glue that holds orchestration steps together is variable interpolation. Every step can produce named outputs, and subsequent steps can reference those outputs using {{stepName.output}} syntax. This lets you build pipelines where data flows naturally from one step to the next.

// Example: Scrape prices, process with Python, save results

Step 1 (Browser): "extract_prices"
  Navigate to amazon.com/s?k=mechanical+keyboard
  Extract: product names and prices
  Output: {{extract_prices.data}}  // JSON array

Step 2 (Script): "analyze"
  Run: python analyze_prices.py
  Stdin: {{extract_prices.data}}
  Output: {{analyze.result}}  // processed analysis

Step 3 (File): "save_report"
  Write: {{analyze.result}}
  Path: ~/reports/keyboards-{{date}}.csv

Step 4 (HTTP): "notify"
  POST: https://hooks.slack.com/services/xxx
  Body: {
    "text": "Keyboard report ready. Best deal:
             {{analyze.result.best_deal}}"
  }

Variable interpolation supports dot notation for accessing nested properties, array indexing, and basic string formatting. The interpolation engine runs in the MCP Relay, so variables can reference data from any step type — browser extractions, script outputs, file contents, or API responses.

Building Orchestrations

You can build orchestrations in two ways: through natural language conversation or through a visual step editor. Most users start with natural language and refine with the editor.

The conversational approach works like this: you describe your desired pipeline in plain English, and Tensor's planner breaks it down into steps, identifies the appropriate step types, and constructs the orchestration. You review the plan, make adjustments, and execute it.

You: "Every morning, check Hacker News for posts about
      browser AI, summarize the top 5, and save a markdown
      report to my Desktop."

Tensor: "I'll create an orchestration with these steps:

  1. [Browser] Navigate to news.ycombinator.com
  2. [Browser] Extract posts mentioning 'browser AI',
     'browser agent', or 'browser extension AI'
  3. [AI] Summarize the top 5 posts into a brief report
  4. [File] Save the summary as
     ~/Desktop/hn-browser-ai-{{date}}.md

  Shall I set this up as a daily scheduled orchestration?"

The visual editor lets you drag and drop step blocks, configure inputs and outputs, set up conditional branching, and test individual steps in isolation. It is particularly useful for complex orchestrations with many steps or conditional logic.

Step Types in Detail

Shell Steps execute terminal commands in a sandboxed subprocess. You can run any command available in your default shell — bash on Linux and macOS, PowerShell on Windows. Shell steps capture both stdout and stderr, and you can access the exit code for conditional branching. For security, shell steps require explicit user approval before the first execution of any new command, and the MCP Relay maintains a whitelist of approved commands.

HTTP Steps make HTTP requests from outside the browser, which means they are not subject to CORS restrictions. This is crucial for calling APIs that do not support browser-origin requests. You can set custom headers, request bodies, authentication tokens, and timeouts. Response bodies are parsed as JSON when possible, making them easy to work with in subsequent steps.

File Steps interact with the local file system. Read operations return file contents as strings or binary buffers. Write operations create or overwrite files. Append operations add to existing files. File steps respect the MCP Relay's security boundaries — by default, they can only access files within your home directory unless you explicitly expand the allowed paths in settings.

Script Steps are an enhanced version of shell steps optimized for running scripts. You specify a runtime (Python 3, Node.js, bash), a script file or inline code, and optional arguments. The MCP Relay handles runtime detection, environment setup, and argument passing. Script steps are ideal for data transformation tasks that are too complex for simple shell commands but do not warrant a full application.

Security Model

Cross-app orchestration is powerful, which means it needs strong security guardrails. The MCP Relay implements a defense-in-depth security model with multiple layers of protection.

User approval. Every new orchestration requires explicit user approval before its first run. The approval dialog shows every step, what it will do, and what system resources it will access. Orchestrations from the marketplace must also be approved, even if they were created by trusted community members.

Command whitelisting. Shell steps can only execute commands that the user has explicitly approved. The first time an orchestration tries to run a new command, Tensor prompts the user to approve or deny it. Approved commands are stored locally and remembered for future runs.

Path restrictions. File steps are restricted to a configurable set of allowed directories. By default, this is the user's home directory. Users can add or remove directories through settings. Attempts to access files outside allowed paths are blocked and logged.

Network restrictions. HTTP steps can be restricted to a list of allowed domains. This prevents orchestrations from making unexpected requests to unknown servers. By default, all domains are allowed, but security-conscious users can enable domain whitelisting.

Execution isolation. Each orchestration runs in its own process context with its own environment variables. Orchestrations cannot read each other's variables, outputs, or state. The MCP Relay enforces strict process isolation to prevent cross-orchestration information leakage.

Real Pipeline Examples

Here are some orchestration pipelines that our users have built and shared through the marketplace:

Daily News Digest. Scrapes five news sites, extracts headlines and summaries, runs a deduplication script to remove overlapping stories, uses the AI to produce a personalized digest based on your interests (from Personal Context), and saves it as a markdown file. Runs every morning at 7 AM via the scheduling system.

Invoice Processor. Watches a Gmail inbox for new invoice emails, downloads PDF attachments, extracts line items using a Python script with tabula-py, enters the data into a QuickBooks web form via browser automation, and sends a Slack notification with the total. Triggered by persistent agent monitoring the inbox.

Repository Health Check. Pulls the latest changes from a Git repository using a shell step, runs the test suite, parses the results with a Node.js script, generates a health report, and posts it to a team Discord channel via webhook. Runs on every push via a GitHub webhook received by the MCP Relay.

The Future of Orchestration

Cross-app orchestration transforms Tensor from a browser AI tool into a general-purpose automation platform. The browser becomes one node in a larger system that spans your entire digital workspace. We are continuing to expand the set of step types, with upcoming support for database queries, desktop application control via accessibility APIs, and inter-device orchestration that lets pipelines span your laptop, phone, and cloud services.

The line between browser and desktop is already blurring. With Tensor's orchestration system, it disappears entirely.