Files & Directories

The portable filesystem API

Go’s filesystem story spans a few packages: os (open, create, read, write, stat, remove), io/fs (the abstract fs.FS and fs.DirEntry types), and path/filepath (OS-aware path joining and tree walking). For everyday reading and writing of whole files see files & the os package; this page is about working with directories and metadata.

Two calls do most of the work:

• os.ReadDir(dir) — the immediate entries of one directory, as []fs.DirEntry.
• filepath.WalkDir(root, fn) — a depth-first walk of the whole tree, calling fn(path, d, err) for every entry.

Walk a tree: size and duplicate detection

This runs on the playground (it builds a small tree in a temp dir first), and shows the two everyday jobs — total size and finding duplicates by content hash:

package main

import (
"crypto/sha256"
"encoding/hex"
"fmt"
"io"
"io/fs"
"os"
"path/filepath"
"sort"
)

func main() {
// Build a sample tree in a temp directory (works on the playground FS).
root, _ := os.MkdirTemp("", "tree")
defer os.RemoveAll(root)
os.MkdirAll(filepath.Join(root, "sub"), 0o755)
os.WriteFile(filepath.Join(root, "a.txt"), []byte("hello"), 0o644)
os.WriteFile(filepath.Join(root, "b.txt"), []byte("world!!"), 0o644)
os.WriteFile(filepath.Join(root, "sub", "c.txt"), []byte("hello"), 0o644) // dup of a.txt

// Walk the tree: sum sizes and group by content hash.
var total int64
byHash := map[string][]string{}
filepath.WalkDir(root, func(path string, d fs.DirEntry, err error) error {
	if err != nil || d.IsDir() {
		return err
	}
	info, _ := d.Info() // lazy: only stat the files we keep
	total += info.Size()
	byHash[hashFile(path)] = append(byHash[hashFile(path)], filepath.Base(path))
	return nil
})

fmt.Printf("total size: %d bytes\n", total)
for h, files := range byHash {
	if len(files) > 1 {
		sort.Strings(files)
		fmt.Printf("duplicates (%s…): %v\n", h[:8], files)
	}
}
}

func hashFile(path string) string {
f, _ := os.Open(path)
defer f.Close()
h := sha256.New()
io.Copy(h, f) // stream, so huge files don't blow up memory
return hex.EncodeToString(h.Sum(nil))
}

a.txt and sub/c.txt hash identically, so they’re reported as duplicates even though the names differ. Note d.Info() is called lazily — that’s the WalkDir win over the older Walk.

Permissions and metadata

Every entry carries a mode: type bits plus Unix permission bits.

graph LR
M["0o644"] --> O["owner: rw- (6)"]
M --> G["group: r-- (4)"]
M --> W["other: r-- (4)"]

This runs here — create a file, read its mode, flip the permission bits, and read them back:

package main

import (
"fmt"
"os"
"path/filepath"
)

func main() {
dir, _ := os.MkdirTemp("", "perms")
defer os.RemoveAll(dir)
path := filepath.Join(dir, "script.sh")
os.WriteFile(path, []byte("#!/bin/sh\necho hi\n"), 0o644)

info, _ := os.Stat(path)
fmt.Printf("before: mode=%v size=%d isDir=%v\n",
	info.Mode(), info.Size(), info.Mode().IsDir())

os.Chmod(path, 0o755) // make it executable

info, _ = os.Stat(path)
fmt.Printf("after:  mode=%v  (owner can execute: %v)\n",
	info.Mode(), info.Mode().Perm()&0o100 != 0)
}

// The mode also encodes the file TYPE, queryable with the helper methods:
info.Mode().IsDir()              // directory?
info.Mode()&os.ModeSymlink != 0  // symlink?
info.Mode().Perm()               // just the 0o777 permission bits

Symlinks

A symlink is a file whose contents are a path to another file. The key distinction is Stat follows it, Lstat doesn’t:

os.Symlink("real.txt", "link.txt")      // create link.txt -> real.txt

target, _ := os.Readlink("link.txt")    // "real.txt"
st, _ := os.Stat("link.txt")            // info about real.txt (followed)
ls, _ := os.Lstat("link.txt")           // info about the link itself
fmt.Println(ls.Mode()&os.ModeSymlink != 0) // true

When walking trees, use Lstat semantics (which WalkDir gives you via DirEntry) so you don’t follow a symlink into a cycle or outside your root.

See also

• temp files & atomic writes — writing files safely (the other half of this).
• files & the os package (stdlib) — reading/writing whole files, the io interfaces.
• system calls — the open/read/stat calls underneath.
• compile & link (internals) — //go:embed for bundling files into the binary.

Next: writing files without corrupting them — temp files & atomic writes.

Related topics