☎️ Analogy
A call center has a fixed number of agents and one queue of waiting callers. No matter how long the queue gets, only as many calls are handled at once as there are agents. Add callers and they wait; the staffing — your concurrency — stays bounded and predictable.
The problem
The easy move — go process(job) for every job — is unbounded. With a flood of jobs you spawn a flood of goroutines, each maybe opening a DB connection or hitting an API, until something falls over. A worker pool fixes the count: N workers, one queue.
Structure
graph LR
P["producer"] -->|"jobs chan"| Q(("jobs"))
Q --> W1["worker 1"]
Q --> W2["worker 2"]
Q --> W3["worker 3"]
W1 -->|"results chan"| R(("results"))
W2 --> R
W3 --> RIdiomatic Go
A fixed set of workers range the jobs channel; a WaitGroup closes results once they’ve all finished. Edit and Run:
package main
import (
"fmt"
"sync"
)
func worker(id int, jobs <-chan int, results chan<- int, wg *sync.WaitGroup) {
defer wg.Done()
for j := range jobs { // pulls jobs until the channel is closed
results <- j * j
}
}
func main() {
const numWorkers = 3
jobs := make(chan int)
results := make(chan int)
// start a fixed pool of workers
var wg sync.WaitGroup
for i := 1; i <= numWorkers; i++ {
wg.Add(1)
go worker(i, jobs, results, &wg)
}
// close results once every worker has finished
go func() { wg.Wait(); close(results) }()
// feed jobs, then signal "no more" by closing the channel
go func() {
defer close(jobs)
for j := 1; j <= 9; j++ {
jobs <- j
}
}()
sum := 0
for r := range results {
sum += r
}
fmt.Println("sum of squares 1..9 =", sum) // 285
}
🐹 The shape to memorize
Close jobs to tell workers the work is done. WaitGroup-then-close results so the consumer’s range terminates. This same skeleton — bounded workers over a channel — is how you turn the unbounded Fan-out into something safe for production. For a one-liner bound, errgroup.Group.SetLimit(n) gives you a pool with built-in error propagation.
Bounded concurrency: three tools
Go gives you several ways to cap how many things run at once — pick by job shape and whether failures matter:
| Tool | Shape | Reach for it when |
|---|---|---|
| Worker pool | N long-lived workers ranging a queue | a steady stream of jobs; reuse amortizes setup |
| Semaphore | goroutine per job, gated by N tokens | occasional jobs; no standing pool wanted |
errgroup + SetLimit(n) | bounded group with error propagation | jobs can fail and you want first-error + cancel |
All three bound concurrency at N; they differ in lifecycle and error handling. The concurrency track covers the goroutine/channel machinery underneath them in depth.
In practice
- A fixed pool processing an HTTP request queue or a batch of files.
- Capping concurrent outbound calls so you don’t overwhelm a downstream service.
golang.org/x/sync/errgroupwithSetLimitwhen jobs can fail.
Pitfalls
⚠️ Deadlock from closing in the wrong place
If you close results right after sending the last job (instead of after the workers finish), a worker still writing will panic on a closed channel — or the consumer ranges a channel that never closes and the program hangs. The ordering is the whole trick: close jobs from the producer; close results only after wg.Wait().
When to use it — and when not
✅ Reach for it when
- You have a stream/queue of jobs and want to cap how many run at once.
- Unbounded 'one goroutine per job' would exhaust memory, connections, or a rate limit.
- Workers can be reused across many jobs.
⛔ Think twice when
- Work is trivial and unbounded concurrency is fine — just launch goroutines.
- You need per-job error handling and cancellation — reach for errgroup.
Related patterns
Distribute work across multiple goroutines (fan-out) and merge their results back into one stream (fan-in).
concurrencySemaphoreLimit how many goroutines may run a section of code (or hold a resource) at the same time.
concurrencyerrgroupRun a group of goroutines, wait for them all, capture the first error, and cancel the rest automatically.
concurrencyPipelineProcess a stream of data through a series of stages connected by channels, where each stage is a goroutine.
Check your understanding
Score: 0 / 51. What does a worker pool bound?
A fixed set of N workers pull from a shared jobs channel, so at most N jobs run at once regardless of how many are queued — predictable resource use.
2. How do workers know there are no more jobs?
Closing the jobs channel makes every worker's `for j := range jobs` loop finish naturally — the idiomatic 'no more work' signal.
3. When is the results channel safe to close?
Multiple workers send to results, so it must be closed exactly once, after all of them are done — a `wg.Wait()`-then-`close` goroutine does that.
4. Worker pool vs semaphore — what's the difference in approach?
Both cap concurrency at N. A pool reuses a fixed set of workers (great for steady streams, amortizes goroutine setup). A [semaphore](/patterns/semaphore/) is per-job — acquire a token, go, release — simpler when jobs are occasional and you don't want a standing pool.
5. How should workers stop early when the caller cancels?
Thread a context through the workers; each does `select { case j := <-jobs: ...; case <-ctx.Done(): return }`. On cancellation workers drain and exit instead of finishing the whole queue — see [context cancellation](/patterns/context-cancellation/).
Comments
Sign in with GitHub to join the discussion.