Floodlight Labs runs its AI workloads on local hardware. The chat assistant on this site, the ranking pipelines behind our discovery products, and day-to-day experiments all run through Ollama on our own AMD GPU rather than cloud APIs.

Local inference has one operational cost that cloud users never think about: the GPU is always there, plugged in, drawing power — whether it’s generating tokens or doing absolutely nothing.

That tension produced rocm-powerd, a small open-source daemon that automatically switches an AMD GPU between power-management modes based on what it’s actually doing. It’s MIT licensed and available on GitHub.

The problem

ROCm gives you good manual control over an AMD GPU’s behavior. With rocm-smi you can pin performance levels, cap the shader and memory clocks, and meaningfully reduce how much power inference draws.

On our Radeon RX 7900 XTX, inference-tuned clock settings reduced power draw during LLM inference at the cost of roughly 10–15% throughput — a trade we’ll happily take on a box that runs unattended around the clock.

But those settings come with a catch: once you apply manual clock tuning, the GPU no longer drops into its deepest idle state. A card that would happily sit under 50 W with automatic power management now idles noticeably higher, forever, because you optimized it for a workload that only runs some of the time.

So you can have great idle power or tuned inference power — but not both, unless something switches the settings for you at the right moments.

That’s the entire job of rocm-powerd.

How it works

rocm-powerd is deliberately boring: a single bash script, a TOML config file, and a systemd unit. No runtime dependencies beyond rocm-smi itself.

The daemon polls the GPU’s power draw every few seconds and treats wattage as a proxy for activity:

  • Power at or above a busy threshold (default 120 W) means a real workload just started — switch to AI mode.
  • Power at or below an idle threshold (default 80 W) means the workload is gone — but only switch back to idle mode after the GPU has stayed quiet for a configurable cooldown (default 5 minutes).

The asymmetry is intentional. Switching into AI mode happens on the first busy poll, because a model is already loading and you want tuned clocks immediately. Switching out is slow and cautious, because chat-style workloads are bursty — tokens stream for ten seconds, pause for thirty, resume. Flapping between power states on every pause would be worse than either state.

AI mode applies the inference tuning:

cpupower frequency-set -u 3000MHz
rocm-smi --setperflevel manual
rocm-smi --setsclk 2
rocm-smi --setmclk 3

Idle mode hands control back to the driver so the card can reach its lowest power states:

rocm-smi --resetclocks
rocm-smi --setperflevel auto
rocm-smi --resetprofile

Thresholds, timings, clock levels, and CPU frequency caps are all configurable in /etc/rocm-powerd/rocm-powerd.toml, and both modes can be replaced entirely with your own scripts — the daemon just decides when, and your scripts decide what.

Small tool, small decisions

A few implementation choices say more about the philosophy than the feature list does.

It’s bash on purpose. The whole daemon is a few hundred lines. It reads its TOML config with a small awk function instead of pulling in a parser, logs through logger to syslog, and runs as a plain systemd service with Restart=on-failure. There is nothing to compile, no runtime to install, and nothing to break when ROCm versions move.

Power is the signal, not process detection. The daemon doesn’t try to detect Ollama, watch process lists, or integrate with inference servers. Wattage is a universal signal: any workload that makes the GPU work — Ollama, llama.cpp, Stable Diffusion, whatever comes next — trips the same threshold. If rocm-smi output ever fails to parse, it falls back to reading the kernel’s hwmon power sensors directly from sysfs.

Hysteresis over cleverness. Two thresholds, a trigger count, and a cooldown timer. No moving averages, no prediction, no machine learning to manage the machine learning. The failure modes are easy to reason about, and --status prints one line telling you the current wattage and mode when you want to check on it.

Why this matters for local AI

Individually, the watts here are small. But self-hosted inference only makes economic sense if the always-on costs stay low — that’s the whole pitch against per-token cloud pricing.

A GPU that idles high because it’s tuned for inference quietly eats into that math every hour of every day. A daemon that removes the tension — deep idle when quiet, tuned clocks under load, no human in the loop — makes the local-first approach easier to defend on cost.

It’s the same philosophy that runs through everything Floodlight Labs builds: small, dependency-light systems that solve one real operational problem and then stay out of the way.

The code, config reference, and install scripts are on GitHub: github.com/dev-dude/rocm-powerd.