Files & os

Whole files vs. streaming

For small files, read or write everything in one call. os.ReadFile returns a []byte; os.WriteFile takes the bytes plus a permission mode (0644 = owner read/write, everyone else read):

package main

import (
"fmt"
"os"
"path/filepath"
)

func main() {
// Build a portable path under the OS temp dir.
path := filepath.Join(os.TempDir(), "greeting.txt")

// Write the whole file at once with 0644 permissions.
if err := os.WriteFile(path, []byte("hello\nfrom os\n"), 0644); err != nil {
	fmt.Println("write error:", err)
	return
}

// Read it all back into memory.
b, err := os.ReadFile(path)
if err != nil {
	fmt.Println("read error:", err)
	return
}

fmt.Printf("base: %s\n", filepath.Base(path)) // greeting.txt
fmt.Printf("ext:  %s\n", filepath.Ext(path))  // .txt
fmt.Printf("bytes read: %d\n", len(b))
fmt.Print(string(b))

os.Remove(path) // clean up
}

When a file is large — or you process it one line at a time — don’t load it all. os.Open returns an *os.File (an io.Reader); always defer f.Close() so the handle is released even on an early return. Wrap it in a bufio.Scanner to walk it line by line:

graph LR
F["os.Open(path)"] --> FILE["*os.File<br/>(io.Reader)"]
FILE --> SC["bufio.Scanner"]
SC -->|"Scan() / Text()"| L["one line at a time"]
F -.->|"defer"| C["f.Close()"]

The same bufio.Scanner pattern reads from any io.Reader — a file, os.Stdin, or a strings.Reader. Here it scans an in-memory reader so the output is fully deterministic:

package main

import (
"bufio"
"fmt"
"strings"
)

func main() {
// In a real program this would be a *os.File from os.Open;
// any io.Reader works the same way.
data := "alpha\nbeta\ngamma"
scanner := bufio.NewScanner(strings.NewReader(data))

lineNo := 0
for scanner.Scan() {
	lineNo++
	fmt.Printf("%d: %s\n", lineNo, scanner.Text())
}
if err := scanner.Err(); err != nil {
	fmt.Println("scan error:", err)
}
fmt.Println("total lines:", lineNo)
}

Open flags and permission bits

os.ReadFile/WriteFile cover the common cases, but os.OpenFile is the full control panel: a bitmask of flags plus a permission mode. The flags are OR-ed together — O_CREATE (make if missing), O_WRONLY/O_RDWR, O_APPEND (write at the end), O_TRUNC (empty first), O_EXCL (fail if it exists). Appending to a log file is the canonical example:

package main

import (
"fmt"
"os"
"path/filepath"
)

func main() {
path := filepath.Join(os.TempDir(), "app.log")
defer os.Remove(path)

// Open for appending, creating the file if it doesn't exist yet.
f, err := os.OpenFile(path, os.O_CREATE|os.O_WRONLY|os.O_APPEND, 0644)
if err != nil {
	fmt.Println("open error:", err)
	return
}
// *os.File is an io.Writer — fmt.Fprintf writes straight to it.
fmt.Fprintf(f, "first entry\n")
fmt.Fprintf(f, "second entry\n")
f.Close()

b, _ := os.ReadFile(path)
fmt.Print(string(b)) // both lines, in order
}

The permission mode is a Unix octal: 0644 is rw-r--r--, 0600 keeps a secret owner-only, 0755 is an executable or directory. (On Windows only the owner-write bit is honored.)

Directories and file info

os.Stat returns an fs.FileInfo — size, mode, modified time, and IsDir. A missing file yields an error you match with errors.Is(err, os.ErrNotExist) (the idiomatic “does it exist?”). os.MkdirAll makes a directory tree, and os.ReadDir lists a directory’s entries:

package main

import (
"errors"
"fmt"
"os"
"path/filepath"
)

func main() {
root := filepath.Join(os.TempDir(), "demo")
defer os.RemoveAll(root) // remove the whole tree

// Make a nested directory and a couple of files.
os.MkdirAll(filepath.Join(root, "sub"), 0755)
os.WriteFile(filepath.Join(root, "a.txt"), []byte("hi"), 0644)
os.WriteFile(filepath.Join(root, "sub", "b.txt"), []byte("there"), 0644)

// Stat: size + kind.
info, _ := os.Stat(filepath.Join(root, "a.txt"))
fmt.Printf("a.txt: %d bytes, dir=%v\n", info.Size(), info.IsDir())

// Existence check via errors.Is.
_, err := os.Stat(filepath.Join(root, "missing.txt"))
fmt.Println("missing.txt is absent:", errors.Is(err, os.ErrNotExist)) // true

// List a directory.
entries, _ := os.ReadDir(root)
for _, e := range entries {
	fmt.Printf("entry: %-6s dir=%v\n", e.Name(), e.IsDir())
}
}

To walk a whole tree recursively, use filepath.WalkDir(root, fn) — it visits every file and subdirectory, calling your function with the path and entry. It’s the efficient (Go 1.16+) replacement for the older filepath.Walk.

Portable paths with filepath

Never build paths by gluing strings with "/" — that breaks on Windows. The path/filepath package handles the OS-correct separator and gives you the pieces of a path:

p := filepath.Join("data", "logs", "app.log") // data/logs/app.log (or \ on Windows)
filepath.Base(p)   // "app.log"
filepath.Dir(p)    // "data/logs"
filepath.Ext(p)    // ".log"
filepath.Clean("a/../b/./c") // "b/c"
abs, _ := filepath.Abs(p)    // absolute path from the cwd
matches, _ := filepath.Glob("*.go") // shell-style matching

The os doorway

Beyond files, os is how a program reaches its environment and controls its own process:

• os.Args — the command-line arguments (os.Args[0] is the program name); parse them with flag.
• os.Getenv("KEY") / os.Setenv / os.LookupEnv — environment variables (LookupEnv returns an ok bool so you can tell unset from empty).
• os.Stdin / os.Stdout / os.Stderr — the three standard streams, each an *os.File (so an io.Reader/io.Writer). That’s why fmt.Fprintln(os.Stderr, …) writes errors to the right place.
• os.Exit(code) — exit immediately with a status code. It does not run deferred functions — reserve it for main, after cleanup.

fmt.Fprintln(os.Stderr, "warning: config not found")
home, ok := os.LookupEnv("HOME")
_ = ok

Reference

See also

• fmt & io — *os.File, os.Stdout, and friends are all io.Reader/io.Writer.
• CLI Tools with flag — turn os.Args into typed options.
• os/exec — run external programs and wire up their stdin/stdout.
• errors — errors.Is(err, os.ErrNotExist) and sentinel errors.

Next: turn os.Args into real options — CLI Tools with flag.

Related topics