Workers
BP’s Asynchronous Design #
BuildPiper is asynchronous by design — job execution happens in the background through dedicated workers instead of blocking the main application flow. This enables high scalability, better reliability, and efficient workload distribution.
Core Components #
Four key components work together to deliver BuildPiper’s asynchronous job execution model.
Entry point where the user interacts with BuildPiper. Acts as the visualization layer for async execution.
Controller is the brain of BuildPiper, coordinating all async operations. Ensures loose coupling between UI and execution layer.
Persistent storage for the job lifecycle. Acts as the source of truth for all job state.
Worker performs the actual execution of the job independently from the user request lifecycle. Handles compute-intensive operations.
BP Workers #
There are 3 categories of BP Workers, based on the BP container component on which their workers run.
These workers execute the core CI/CD pipeline operations inside BuildPiper. They handle build, deploy, environment cloning, insights, and parallel execution.
| Worker | Description |
|---|---|
bp_agent_build@bpagent |
Executes application build steps — compiling code, packaging artifacts, running build scripts, and uploading deployable artifacts for both v2 and v3 jobs. |
bp_agent_canary_analysis@bpagent |
Performs automated analysis for canary deployments by comparing performance metrics between old and new versions to ensure safe rollout. |
bp_agent_clone_env@bpagent |
Creates a replica of an existing environment configuration to speed up new environment provisioning. |
bp_agent_deploy@bpagent |
Executes deployment workflows — Helm releases, Kubernetes manifest application, config-map deployment, secret deployment, JIRA, SNOW, attach documents and rollout validation. |
bp_agent_insight@bpagent |
Pushes required data to BuildPiper Maturity Insights to power dashboards such as DORA Metrics, Deployment Insights, and other analytics. |
bp_agent_parallel_build@bpagent |
Runs multiple build jobs in parallel to reduce pipeline execution time for multi-architecture execution. |
bp_agent_parallel_build_decision@bpagent |
Decides build execution strategy based on multi-architecture configuration feature flag. |
bp_android_build@bpagent |
Handles Android build steps such as Gradle compilation and APK/AAB generation. |
bp_android_deploy@bpagent |
Manages deployment of Android builds to distribution platforms such as Play Store or artifact repositories. |
These workers manage time-based or event-based automation tasks.
| Worker | Description |
|---|---|
admin_scheduler@scheduledapi |
Handles administrative scheduled tasks such as maintenance jobs, cleanup operations, and system-level cron executions. |
scheduler@scheduledapi |
Executes and schedules pipeline triggers such as periodic scans, manual triggers, and async jobs. |
scheduler_batch_job@scheduledapi |
Critical monitoring queue that keeps track of all queues. |
scheduler_clone_env@scheduledapi |
Automates environment cloning, useful when a user triggers a complete clone of an existing environment. |
versioning_api@scheduledapi |
Manages application version updates when a user edits a specific workflow of BP Resources such as CI, CD, pipeline, etc. |
These workers manage deployment orchestration and API-driven deployment requests.
| Worker | Description |
|---|---|
admin_deploy_api@deployapi |
Handles privileged deployment operations such as manual overrides, rollbacks, and admin-triggered deployments. |
deploy_api@deployapi |
Core deployment API worker responsible for processing deployment requests from UI or pipeline triggers. |
Jobs in BuildPiper #
BuildPiper native jobs provided out-of-the-box by BP.
Custom created jobs in the platform for specific custom needs.
Job to Worker Mapping #
The table below lists each job type and the worker responsible for executing the corresponding job.
Static (v2) Jobs — Worker Mapping #
| Job Type | Operation | Worker Responsible |
|---|---|---|
| v2 | Build job | bp_agent_build |
| v2 | Build cron job | bp_agent_build |
| v2 | Deploy job | bp_agent_deploy |
| v2 | Deploy cron job | bp_agent_deploy |
| v2 | Promote job | bp_agent_deploy |
| v2 | Promote cron job | bp_agent_deploy |
| v2 | JIRA — create / read / update jobs | bp_agent_deploy |
| v2 | SNOW — create / read / update jobs | bp_agent_deploy |
| v2 | Delay job | scheduledapi |
| v2 | Canary Analysis | bp_agent_canary_analysis |
| v2 | Android build | bp_android_build |
| v2 | Android deployments | bp_android_deploy |
| v2 | Environment Clone | bp_agent_clone_env |
Dynamic (v3) Jobs — Worker Mapping #
| Job Type | Operation | Worker Responsible |
|---|---|---|
| v3 | All v3 operations | bp_agent_build |
Job States #
Job received by the Controller and queued for processing.
Worker has picked the job and started processing it.
Job has completed successfully.
Job that has failed due to some error.
Job no longer required and marked as revoked.
How to Troubleshoot a Job Stuck In-Queue #
When a job remains in In-Queue status, it means the job has been created and stored in the database but the corresponding worker has not yet picked it up. This is generally related to one of the following causes.
All configured workers are already occupied processing existing jobs. Newly triggered jobs remain In-Queue until a worker becomes available.
The worker responsible for processing jobs is not in a running or healthy state. Queued jobs will not be picked for execution.
The Scheduler component manages adding tasks to the appropriate queue and updating execution state. If not functioning correctly, jobs may remain In-Queue even when workers are healthy.
If a worker does not appear on the Health Status page, it may indicate missing or incorrectly defined configuration with no active worker available to process queued jobs.
📘 BuildPiper Documentation · Asynchronous Design
Last updated: March 2026