Memory-Mapped Files

What memory mapping is

mmap asks the kernel to map a file (or an anonymous region) directly into your process’s virtual address space. After that, the file’s bytes are memory: you read and write them like a []byte, and the kernel pages the actual data in from disk on demand and flushes writes back lazily.

graph LR
F["file on disk"] -->|"mmap"| V["region of virtual memory"]
V -->|"access page → page fault"| K["kernel pages it in (lazy)"]
V -. "[]byte view, no read() per access" .-> APP["your Go code"]

The two payoffs: zero-copy (no read() into a user buffer) and laziness (only the pages you touch are loaded). For random access over a multi-gigabyte file — a database file, a search index, a column store — that beats read/seek, which would copy and often over-read.

A real mmap, runnable

The simplest mapping is anonymous — memory not backed by any file. It’s a real mmap(2)/munmap(2) and runs right here: the returned region is a []byte you read and write as ordinary memory, with no per-access syscall:

package main

import (
"fmt"
"syscall"
)

func main() {
const size = 4096 // one page

// Anonymous mmap: ask the kernel for memory directly (fd = -1, no file).
// MAP_SHARED means a forked child would see writes too — true shared memory.
region, err := syscall.Mmap(-1, 0, size,
	syscall.PROT_READ|syscall.PROT_WRITE,
	syscall.MAP_ANON|syscall.MAP_SHARED)
if err != nil {
	fmt.Println("mmap failed:", err)
	return
}
defer syscall.Munmap(region) // outside the GC heap — you must free it

// The region IS a []byte. Write to it like ordinary memory...
msg := "written straight into mapped memory"
copy(region, msg)
// ...and read it back — no read()/write() syscall per access.
fmt.Printf("read back: %q\n", region[:len(msg)])
fmt.Println("region length:", len(region), "bytes")
}

File-backed mappings are the same call with a real fd instead of -1 — that’s the next section.

Mapping a file in Go

There’s no standard os.Mmap; use golang.org/x/sys/unix (or syscall). The result is a []byte aliasing the file:

//go:build unix

package main

import (
	"fmt"
	"os"

	"golang.org/x/sys/unix"
)

func main() {
	f, _ := os.Open("big.dat")
	defer f.Close()

	info, _ := f.Stat()
	size := int(info.Size())

	// Map the whole file read-only into memory.
	data, err := unix.Mmap(int(f.Fd()), 0, size,
		unix.PROT_READ, unix.MAP_SHARED)
	if err != nil {
		panic(err)
	}
	defer unix.Munmap(data) // outside the Go heap — must free it yourself

	// data is a []byte view of the file; no per-access read syscall.
	fmt.Printf("first 4 bytes: %v\n", data[:4])
	fmt.Printf("byte at 1<<20: %d\n", data[1<<20]) // pages in just that block
}

For writable mappings, add PROT_WRITE and MAP_SHARED, mutate the slice, and call unix.Msync(data, unix.MS_SYNC) to force changes to disk.

Shared memory between processes

Two processes that mmap the same file with MAP_SHARED see the same physical pages — the fastest IPC possible, because nothing is copied at all:

// Process A and Process B both:
data, _ := unix.Mmap(fd, 0, size, unix.PROT_READ|unix.PROT_WRITE, unix.MAP_SHARED)
// A writes data[i] = x  →  B sees x at data[i]

The catch: the kernel gives you shared bytes but no synchronization. You must coordinate access yourself — a mutex in a shared header, atomic operations, or a separate signaling channel — or you’ll have data races across processes.

When mmap wins — and when it doesn’t

See also

• Unix sockets — the other fast local-IPC mechanism.
• files & directories — the portable file API to prefer by default.
• the memory allocator (internals) — why mmap’d memory sits outside the Go heap.
• system calls — mmap/munmap/msync are syscalls behind x/sys.

Next: the foundations these primitives run on — the internals track.

Related topics