On-Prem Worker

BigPanda's L1 Agent On-Prem Worker is a lightweight containerized agent that runs on your infrastructure to execute runbook automation steps. It connects to BigPanda using mutual TLS (mTLS) authentication and executes pre-registered workflows on demand.

The worker is required for L1 Agent runbook automation. It is not required for AI Detection and Response (ADR).AI Detection and Response

Key features

Secure by design: All communication is encrypted with per-organization certificates. The worker only makes outbound connections.
Data in place: The worker reads data from your systems at the source and returns structured diagnostic results to BigPanda. It does not copy, replicate, or move data between systems or environments.
Customer-held credentials: Every credential the worker uses is an environment variable you set, typically injected from your secret store. Credentials never go in the worker's integration configuration, and BigPanda does not store them.
Customer-controlled execution: All runbooks are authored, reviewed, and owned by your team. The worker only runs pre-registered workflows, and only against the APIs, paths, methods and SSH commands you allow in its configuration.

Supported versions

These instructions apply to worker v13.9.0 and later. New installs use the latest version; earlier versions can no longer be installed.

To check the version you are running:

docker images | grep adr-temporal-worker

For install steps, see Install the On-Prem Worker.

On-Prem Worker architecture

The worker connects outbound to BigPanda and waits for workflow instructions. When BigPanda's triage agent detects a device-related incident, it triggers the health check workflow for your worker. Device health checks are one of the worker's capabilities: depending on your configuration, it also calls HTTP integrations, runs commands on SSH targets (v13.10.0 and later), and triggers Itential automations.

The BigPanda triage agent detects an incident that requires information from your environment.
BigPanda routes a workflow task to your on-premises worker.
The worker executes the workflow (for example, checking device health against your monitoring platform).
The worker returns the result to BigPanda, where it feeds into the L1 Agent's reasoning.

BigPanda Cloud                        Your Network
──────────────                        ────────────

Triage Agent detects                  Worker (this service)
"device unreachable"                      │
        │                                 │ Connects to BigPanda
        ▼                                 │ via mTLS (outbound only)
Temporal Server                           │
        │                                 │
        └── Routes workflow ─────────────►│
            to your worker                │
                                          ▼
                                   Calls your monitoring API
                                   GET {healthCheckUrl}/{deviceName}
                                   Auth: Bearer <DEVICE_API_KEY>
                                          │
                                          ▼
                                   Returns result to BigPanda
                                   (UP / DOWN / NOT_SUPPORTED)

The worker receives the details each workflow needs, such as which monitoring endpoints to call and which devices to check, as input at execution time. You don't need to schedule workflows: BigPanda routes them to the worker when your runbooks call for them. The worker just needs to be running and connected.

Communication model

Direction	Destination	Protocol	Purpose
Outbound	BigPanda cloud services	gRPC + mTLS	Workflow task retrieval and result delivery
Outbound	Your systems: monitoring platform, HTTP integration targets, SSH targets, Itential, Jira and Confluence	HTTPS or SSH	Runbook steps (for example, device health checks, API calls and SSH commands)
Inbound	None required	—	The worker only makes outbound connections

For hosts and ports, see Network requirements in Install the On-Prem Worker.

Security

The On-Prem Worker is designed for data-in-place information gathering, and you can configure it to be read-only. The following sections describe the controls that support that posture.

Identity and access management

The worker has no access to your systems of its own. Every credential it uses is an environment variable you set on the worker: the variables named by secret_env, client_secret_env and private_key_env in its integration configuration, Itential credentials, and JIRA_AUTH.

Credentials as environment variables: The worker's configuration names an environment variable for each credential; the credential itself never goes in the configuration. Set these variables through your runtime's secret mechanism, for example a Kubernetes Secret.
Least privilege: Scope each credential (API token, service account or SSH key) to only the systems and data your runbooks need. BigPanda provides guidance on recommended permission scoping.
Credentials read at startup: The worker reads its credentials from its environment when it starts. OAuth 2.0 access tokens are cached in memory and renewed before they expire. Secrets passed in a query string are masked in debug output.
Customer-controlled credential rotation: You control credential rotation according to your internal security requirements. To rotate a credential, update its environment variable and restart the worker. BigPanda never stores or manages your infrastructure credentials.
Certificate-derived org identity: Your organization identity is derived from the mTLS certificate. No manual namespace configuration is needed.

Network security

The worker operates entirely within your network perimeter. All communications between the worker and your systems are subject to your network policies, firewall rules, and segmentation controls.

Encrypted communications: All communication between BigPanda and the worker uses TLS 1.2 or higher, with strong cipher suites. The worker validates certificates to prevent man-in-the-middle attacks.
Outbound-only connectivity: The worker initiates outbound mTLS connections to BigPanda. No inbound ports need to be opened in your firewall.
Network isolation: The worker can be deployed in an isolated network segment with access limited to only the systems required by your runbooks.

Data protection

Data in place: The worker reads and processes data at its origin. It does not copy, replicate, or relocate data between systems or environments. Diagnostic results returned to BigPanda are structured summaries, not raw data exports.
Data in transit: Execution requests and results are transmitted over mutually authenticated, TLS 1.2+ encrypted channels.
Data at rest: The worker does not persist data at rest. Execution context and gathered information exist only in memory and are released on completion or failure. Credentials come from the worker's environment variables.
Data minimization: The worker collects only the information specified by your runbook steps. It does not perform broad discovery, scanning, or enumeration. You control what data is gathered by controlling runbook content.

Controlled execution

The worker only executes pre-registered workflows authored and approved by your team. Within those workflows, it can call only the API paths and methods you allow (allowed_path_prefixes, allowed_methods) and run only the SSH commands you allow (allowed_commands, allowed_path_prefixes). A command outside those lists is refused before any connection opens.

Compliance and trust posture

BigPanda undergoes annual SOC 2 Type II audits covering security, availability, and confidentiality trust service criteria. The On-Prem Worker development and release processes are within the audit scope.

For the SOC 2 Type II report, the L1 Agent On-Premises Worker security and trust brief, and additional security documentation, visit the BigPanda Trust Center.

Shared responsibilities

Security of the On-Prem Worker deployment operates under a shared responsibility model. BigPanda is responsible for the security of the worker software itself. You are responsible for the security of the environment in which it runs and the access it is granted.

BigPanda responsibilities

Worker software security: Secure development lifecycle (threat modeling, secure coding, static and dynamic analysis, dependency scanning, peer code review, and penetration testing). Continuous monitoring for known vulnerabilities in the worker codebase and dependencies, with severity-based remediation SLAs.
Secure communication protocol: Maintenance of the encrypted communication channel between BigPanda and the worker, including certificate management for the BigPanda-side endpoints.
Security updates and advisories: Release of security patches and advisories for the worker, with notifications through standard communication channels.
Audit trail: Platform-side logging of the end-to-end process for review and troubleshooting.

Customer responsibilities

Runbook authoring and review: Create, review, and maintain runbooks. Runbooks define what systems the worker connects to and what data it gathers.
Incident-runbook associations: Configure which runbooks are triggered for which incident types.
Credential provisioning and scoping: Create the credentials the worker uses (API tokens, service accounts, SSH keys) and scope each one to least privilege.
Secret management: Store the worker's credentials in your secret store, inject them as environment variables, and rotate them according to your security policies.
Network security: Manage the network environment in which the worker operates, including firewall rules, segmentation, DNS, and traffic monitoring.
Host and container security: Manage the compute environment running the worker container, including host OS patching, container runtime security, and resource isolation.
Worker updates: Apply BigPanda-released worker updates in line with your patch management SLA.
Monitoring and alerting: Monitor the worker's operational behavior, including resource consumption, network connections, and execution anomalies.

Operational security guidance

The following recommendations supplement the install instructions and are aligned with the controls described in the L1 Agent On-Premises Worker security and trust brief on the BigPanda Trust Center.

Deployment best practices

Image verification: Before you load the image, check the downloaded file size against the value BigPanda provides. After loading, confirm the image with docker images | grep adr-temporal-worker.
Dedicated service account: Create a dedicated service identity for the worker. Do not reuse identities shared with other services or human users.
Network segmentation: Deploy the worker in an isolated network segment with access limited to only the systems required by active runbooks.
Resource limits: Configure CPU and memory limits on the worker container to prevent resource exhaustion and contain unexpected behavior. For recommended values, see How big should the container be? in the FAQs.
Update cadence: Establish a process to apply BigPanda-released worker updates within your organization's patch management SLA.

Credential scoping recommendations

Start minimal: Begin with the narrowest permissions possible and expand only as specific runbooks require additional access.
Read-only by default: The worker's intended function is information gathering. Configure HTTP integrations as read-only with "allowed_methods": ["GET"], limit SSH targets to read commands in allowed_commands, and scope credentials to read-only permissions unless a runbook explicitly requires write access for a documented, approved purpose.
Regular rotation: Rotate the worker's credentials on a regular schedule. Update the environment variable and restart the worker to apply a new credential.
Source restrictions: Where your systems support it, restrict the worker's credentials to the worker host's network location, for example with source IP allowlists.

Runbook governance

Peer review: Require peer review for all runbook changes before deployment, applying the same rigor used for infrastructure-as-code.
Version control: Maintain runbooks in version control with a complete audit trail of changes, approvals, and deployments.
Testing: Test runbooks in a non-production environment before associating them with production incident types.
Periodic review: Review active runbooks on a regular cadence to confirm they remain aligned with current infrastructure, access policies, and operational requirements.

FAQs

Does BigPanda have access to my systems?

No. The worker runs in your environment using credentials you provision. BigPanda does not have direct access to your infrastructure.

Can the worker modify my systems?

It can, if you allow it. HTTP integrations allow GET, POST, PUT, PATCH and DELETE unless you set "allowed_methods": ["GET"], and SSH targets and Itential can make changes. The worker only acts as your runbooks instruct. BigPanda recommends configuring integrations as read-only and scoping credentials to read-only permissions.

Where does my data go?

Data is processed in place within your environment. Only structured diagnostic results specified by your runbook are returned to BigPanda over an encrypted channel.

How do I control what the worker can access?

You control access through the credentials you provide and the worker's configuration. The worker can only reach the systems you configure, using the credentials you set, within the paths, methods and commands you allow. Apply least-privilege principles to each credential.

Does BigPanda write or modify my runbooks?

No. All runbooks are authored, maintained, and owned by you. BigPanda provides the execution engine but does not create, modify, or inject runbook logic.

How are worker updates delivered?

BigPanda releases updated container images. Download and deploy updates according to your organization's change management process.

How big should the container be?

The container is lightweight:

Deployment	Recommended resources
Docker	A small VM: `2 vCPU`, `2 GB` RAM
Kubernetes requests	`256Mi` memory, `100m` CPU
Kubernetes limits	`512Mi` memory, `500m` CPU

Get help

If you run into any issues or need help with your deployment, contact BigPanda support at support@bigpanda.io with:

Your worker version (docker images | grep adr-temporal-worker)
Relevant logs (docker logs --tail 200 adr-temporal-worker)
Steps to reproduce the issue

For broader L1 Agent capabilities, see the BigPanda L1 Agent overview.

Install the On-prem worker

The worker ships as a Docker container image. BigPanda provides a presigned download URL during onboarding.

Before you start

Make sure you have the following ready:

Docker installed on the target machine
mTLS certificates from BigPanda (provided during onboarding)
An API key for your monitoring platform with read access to the relevant health or status endpoints

Deploy the worker with Docker

Download and load the image

BigPanda provides a pre-signed URL for the Docker image. Download and load it:

curl -L '<PRESIGNED-URL>' -o adr-temporal-worker.tar.gz
docker load < adr-temporal-worker.tar.gz

Caution

The image is delivered as a gzipped Docker-save archive (.tar.gz). docker load decompresses it automatically. If you use crane, skopeo, or another registry tool that expects an uncompressed archive, run gunzip adr-temporal-worker.tar.gz first.

Install certificates

BigPanda provides your organization's mTLS certificates during onboarding. Place them in a certs/ directory:

Example file structure:

certs/
├── client.pem       # Client certificate (identifies your organization)
├── client-key.pem   # Client private key
└── ca.cert          # BigPanda CA certificate

The filenames above are examples. The worker uses whatever paths you set in TEMPORAL_TLS_CERT_PATH, TEMPORAL_TLS_KEY_PATH, and TEMPORAL_TLS_CA_PATH. The names on disk do not matter as long as those variables point to the right files.

Certificate security

Keep the client certificate file secure. It authenticates your worker to BigPanda. If compromised, contact BigPanda immediately to rotate the certificate.

(Optional) Stop the existing worker

If you are upgrading an existing worker, stop and remove the current container:

docker stop adr-temporal-worker && docker rm adr-temporal-worker

Create a configuration file

Create an .env file with the following variables:

# --- Connection (provided by BigPanda) ---
TEMPORAL_ADDRESS=temporal.bigpanda.io:7233
TEMPORAL_TLS_CERT_PATH=/certs/client.pem
TEMPORAL_TLS_KEY_PATH=/certs/client-key.pem
TEMPORAL_TLS_CA_PATH=/certs/ca.cert

# --- Secrets (configured by you) ---
DEVICE_API_KEY=<your-monitoring-platform-api-key>

Monitoring platform authentication

DEVICE_API_KEY must have read access to device health status on your monitoring platform. The worker sends it as a Bearer token in the Authorization header. If your monitoring platform does not require authentication, you can omit this variable.

Run the worker

docker run -d \
  --name adr-temporal-worker \
  --restart unless-stopped \
  --log-opt max-size=50m \
  --log-opt max-file=3 \
  -p 8080:8080 \
  -v $(pwd)/certs:/certs:ro \
  --env-file .env \
  adr-temporal-worker:latest

Verify the deployment

Check the worker logs to confirm a successful start:

docker logs -f adr-temporal-worker

You should see output similar to:

Starting Temporal Worker
Worker state changed { state: 'RUNNING' }
Worker started
Health server listening

Then verify the health endpoints:

curl http://localhost:8080/health    # Returns: {"status":"ok","startedAt":"..."}
curl http://localhost:8080/ready     # Returns: {"status":"ready"}

Deploy the worker with Kubernetes

If you are deploying to Kubernetes, use the following manifests as a starting point.

Create certificates and monitoring API key

apiVersion: v1
kind: Secret
metadata:
  name: adr-temporal-worker-secrets
  namespace: bigpanda
type: Opaque
stringData:
  DEVICE_API_KEY: "<your-monitoring-platform-api-key>"
---
apiVersion: v1
kind: Secret
metadata:
  name: adr-temporal-worker-certs
  namespace: bigpanda
type: Opaque
data:
  client.pem: <base64-encoded-cert>
  client-key.pem: <base64-encoded-key>
  ca.cert: <base64-encoded-ca>

Create the deployment

apiVersion: apps/v1
kind: Deployment
metadata:
  name: adr-temporal-worker
  namespace: bigpanda
spec:
  replicas: 1
  selector:
    matchLabels:
      app: adr-temporal-worker
  template:
    metadata:
      labels:
        app: adr-temporal-worker
    spec:
      containers:
        - name: worker
          image: adr-temporal-worker:latest
          ports:
            - containerPort: 8080
          envFrom:
            - secretRef:
                name: adr-temporal-worker-secrets
          env:
            - name: TEMPORAL_ADDRESS
              value: "temporal.bigpanda.io:7233"
            - name: TEMPORAL_TLS_CERT_PATH
              value: "/certs/client.pem"
            - name: TEMPORAL_TLS_KEY_PATH
              value: "/certs/client-key.pem"
            - name: TEMPORAL_TLS_CA_PATH
              value: "/certs/ca.cert"
          volumeMounts:
            - name: certs
              mountPath: /certs
              readOnly: true
          resources:
            requests:
              memory: "256Mi"
              cpu: "100m"
            limits:
              memory: "512Mi"
              cpu: "500m"
          livenessProbe:
            httpGet:
              path: /health
              port: 8080
            initialDelaySeconds: 30
          readinessProbe:
            httpGet:
              path: /ready
              port: 8080
            initialDelaySeconds: 15
      volumes:
        - name: certs
          secret:
            secretName: adr-temporal-worker-certs

Configure the worker

Connection configuration

Primary connection configuration is provided by the Bigpanda team.

Variable	Description	Example
`TEMPORAL_ADDRESS`	BigPanda server address	`temporal.bigpanda.io:7233`
`TEMPORAL_TLS_CERT_PATH`	Path to mTLS client certificate	`/certs/client.pem`
`TEMPORAL_TLS_KEY_PATH`	Path to mTLS client private key	`/certs/client-key.pem`
`TEMPORAL_TLS_CA_PATH`	Path to BigPanda CA certificate	`/certs/ca.cert`

Secrets

Secrets are configured by you, following the procedure for the connected monitoring platform.

Variable	Description	Required
`DEVICE_API_KEY`	API key for your monitoring platform. Sent as a `Bearer` token in the `Authorization` header.	Only if your monitoring platform requires authentication

Optional configuration

Variable	Default	Description
`METRICS_PORT`	`8080`	Port for health endpoints
`LOG_LEVEL`	`info`	Log level (`debug`,`info`,`warn`,`error`)

Health and Monitoring endpoints

Endpoint	Response	Purpose
`GET /health`	`{"status":"ok","startedAt":"..."}`	Health check — returns 200 if the worker is running
`GET /ready`	`{"status":"ready"}`	Readiness check — returns 200 if the worker is initialized

Use these endpoints for container orchestration health probes such as Docker healthcheck or Kubernetes liveness and readiness probes.

Network Requirements

Direction	Destination	Port	Protocol	Purpose
Outbound	`temporal.bigpanda.io`	`7233`	gRPC + mTLS	BigPanda server connection
Outbound	Your monitoring platform	Varies	HTTPS	Device health checks
Inbound	None required	—	—	—

Firewall configuration

No inbound ports need to be opened. The worker only makes outbound connections.

Troubleshooting

Symptom	Likely Cause	Solution
Worker fails to start with an ENOENT error	Certificate files not found	Verify that cert files exist at the paths specified in `TEMPORAL_TLS_CERT_PATH`, `TEMPORAL_TLS_KEY_PATH`, and `TEMPORAL_TLS_CA_PATH`.
TLS handshake failed in logs	Invalid, expired, or incorrectly signed certificates	Contact BigPanda to verify or rotate your certificates.
Worker crashes with DEADLINE_EXCEEDED or a connection error	Cannot reach BigPanda	Verify that your firewall or proxy allows outbound traffic to the address specified in `TEMPORAL_ADDRESS`.
Health endpoint not responding	Port not exposed or blocked	Verify that port 8080 is exposed (`-p 8080:8080`) and not blocked by a local firewall.
Device check returns DOWN for all devices	Invalid or missing monitoring API key	Verify that `DEVICE_API_KEY` has read access to device health status. Test the API manually: `curl -H "Authorization: Bearer <key>" https://your-monitor/api/device-health/<device>`
Device check times out (60 seconds)	Monitoring platform unreachable from worker	Verify that the worker can reach your monitoring platform. Check DNS resolution and firewall rules.
Container exits immediately	Missing required environment variables	Verify that all `TEMPORAL_*` variables are set in your `.env` file. Run `docker logs adr-temporal-worker` to identify the specific missing variable.
Container restarts repeatedly	Persistent connection or authentication failure	Run `docker logs adr-temporal-worker` to identify the error, then refer to the relevant row above.

Useful Commands

docker logs -f adr-temporal-worker          # View live logs
docker logs --tail 100 adr-temporal-worker  # View last 100 lines
docker restart adr-temporal-worker          # Restart the worker
docker stop adr-temporal-worker             # Stop the worker
docker rm adr-temporal-worker               # Remove the worker
curl http://localhost:8080/health           # Check health status
curl http://localhost:8080/ready            # Check readiness status

In this section: