Getting started

Portals runs code written in another language inside supervised OS processes ("workers") and lets Elixir call it like a function. Each worker is a normal program in its own language that imports a small Portals SDK; the BEAM launches it, talks to it over a private Unix-domain socket, and restarts or fails it like any other OTP process.

This page covers what is the same for every language. Each language has its own page with a complete walkthrough: Python, Ruby, JavaScript, PHP, Java and Go.

For a working app that uses all six languages at once, see examples/hello_live, a Phoenix LiveView page that calls a hello-world function in each.

Packages

Core portals contains the Elixir side only: pools, the connection state machine, the wire protocol and the conformance kit. Each language is a separate package that bundles its worker SDK, so you install only the languages you use:

Every package is published to Hex only. The SDKs are never installed from PyPI, RubyGems, npm, Packagist, Maven or the Go module proxy: they live in the package's priv/ directory and the language module puts them on the worker's import path for you. All packages are released in lockstep, at the same version as portals.

def deps do
  [
    {:portals_python, "~> 0.1.0"},
    {:portals_javascript, "~> 0.1.0"}
  ]
end

A language package depends on portals, so you don't list portals itself.

The shape of every integration

1. Handlers: ordinary functions in the other language.
2. An entry script (or main class or binary) that creates a Worker and calls run(). Portals passes it the socket path as its last command-line argument; the SDK reads it from there.
3. A pool in your supervision tree, configured with the language module's worker_opts/1.
4. Calls from Elixir: Portals.Pool.call(pool, module, function, args).

children = [
  {Portals.Pool,
   Portals.Python.worker_opts(
     script: Application.app_dir(:my_app, "priv/python/worker.py"),
     name: MyApp.Python,
     size: 4
   )}
]

{:ok, 2} = Portals.Pool.call(MyApp.Python, "text_tools", "word_count", ["hello world"])

module and function are strings in the worker's naming scheme, for example a Python module path or a name registered with the worker. The arguments are a list of values; see "Values" below.

Pools

Portals.Pool keeps size workers running and can start up to max_overflow temporary ones under load. Each worker serves many calls at once, up to the concurrency it advertises (max_concurrency in its SDK). The pool sends each call to the least-busy worker.

Per-call options: :timeout (how long the caller waits, default 5_000) and :deadline_ms (a deadline passed to the worker, which handlers can read).

For a single unpooled worker, use Portals.start_worker/1 with the same options and Portals.call/5. Streams currently need a single worker (see "Streaming").

Errors

Every failure is {:error, %Portals.Error{}}, never a raise (except from the call! variants). kind tells you what happened:

Portals.cancel/1 is cooperative: it tells the worker the call was cancelled, and the handler decides what to return (each SDK exposes a "cancelled?" check). The caller still receives that terminal result.

Callbacks and messages

A handler can call back into Elixir while it runs, for example to report progress or to fetch data:

portals.callback("Elixir.MyApp.Progress", "report", [step])

and can send a message to any PID it was given:

portals.send_message(pid, {"done": True})

which arrives as {:portals_message, worker_pid, value} (or the bare value with message_envelope: :raw).

Streaming

Portals.open_stream/5 opens a bidirectional stream on a single worker. Both sides send chunks; each direction has a byte-credit window, so a slow consumer slows the producer down instead of growing memory without bound.

{:ok, worker} = Portals.start_worker(Portals.Python.worker_opts(script: "priv/python/worker.py"))
{:ok, stream} = Portals.open_stream(worker, "text_tools", "upcase_stream", [])
:ok = Portals.Stream.send_enumerable(stream, ["a", "b"])
["A", "B"] = stream |> Portals.Stream.to_enumerable() |> Enum.to_list()
{:ok, 2} = Portals.Stream.await(stream)

Values

Values are encoded with MessagePack plus Erlang extensions, so most Elixir terms survive a round trip:

Gotchas (all languages)

• Wrap raw bytes in %Portals.Codec.Bin{}. Elixir binaries are sent as text. A binary that is not valid UTF-8 is corrupted or rejected depending on the language. In Python it closes the worker's connection, failing every call in flight on it.
• Only return atoms the BEAM already knows. A worker that sends an atom which does not exist on the BEAM (to stop atom-table exhaustion, Portals never creates atoms from worker input) breaks the protocol: the worker's connection is closed and every call in flight on it fails. Send strings for open-ended values.
• Callback targets need the Elixir. prefix and must exist. Name the module as "Elixir.MyApp.Progress", not "MyApp.Progress". The module and function atoms must already exist on the BEAM, so the module must be loaded (it is in a release; in tests, call Code.ensure_loaded!/1).
• :timeout only stops the caller waiting. The handler keeps running, the call keeps its slot in the worker's concurrency budget until it returns, and its late reply is dropped into the caller's mailbox as a {:portals_result, ...} message. Keep handlers bounded, honor :deadline_ms inside long handlers, and for work you may abandon use Portals.async/5, then Portals.cancel/1 with a handler that checks its SDK's cancelled flag.
• Worker stdout and stderr are discarded under the default Unix-socket transport. A print or a crash traceback written there never reaches your logs. Log to a file (PORTALS_LOG_FILE in the Ruby, JavaScript, PHP, Java and Go SDKs; the logging module in Python).
• Streams are per worker. Portals.Pool has no streaming API; open streams on a dedicated Portals.start_worker/1 worker.
• Give each pool its own child id. Two {Portals.Pool, ...} children in one supervisor need Supervisor.child_spec({Portals.Pool, opts}, id: MyApp.Python).
• The working directory is the BEAM's. Use absolute paths (Application.app_dir/2) for scripts and data files, or pass :cd.
• Unix socket paths are limited to about 100 bytes. Sockets live under $PORTALS_RUNTIME_DIR (default System.tmp_dir!()/portals). If your temp directory has a long path, set PORTALS_RUNTIME_DIR to something short.
• Portals owns only the direct worker process. Processes your handler spawns are not killed with the worker. Use a container or process group if you need that.
• Workers are trusted. Process isolation is not a sandbox. Never let user input choose the module/function names: in Python, for example, any importable module, os included, is callable.

Best practices (all languages)

• Build worker_opts/1 with Application.app_dir(:my_app, "priv/...") paths so they work identically in development and in releases.
• Size pools for CPU parallelism and max_concurrency for I/O concurrency. Most runtimes have a global lock or a single event loop, so one worker rarely uses more than one core.
• Set :callback_allowlist in production so a worker can only call the functions it is meant to.
• Pass plain data (maps, lists, strings, numbers) across the boundary and keep language objects on their own side.
• Treat the worker like a microservice: keep it stateless where possible so a restart loses nothing, and make module and function names constants in your Elixir code.