PowerScripting All articles
Opinion & Analysis

Run It Twice, Trust It Once: Engineering Scripts That Survive Repeated Execution

PowerScripting
Run It Twice, Trust It Once: Engineering Scripts That Survive Repeated Execution

Photo by Photo by Tyler on Unsplash on Unsplash

Every automation engineer eventually learns this lesson the hard way: the script that worked flawlessly in testing will, at some point, run twice. A cron job overlaps with a manual run. A CI pipeline retries a flaky step. An on-call engineer, staring at a half-finished deployment at 2 a.m., hits Enter again because they cannot tell whether the first attempt succeeded.

What happens next separates two very different tiers of script author.

The Cost of Assuming a Clean Slate

Consider a provisioning script that appends a line to /etc/hosts, creates a service account, and starts a daemon. Run once, it works. Run twice, and you now have a duplicate hosts entry, a failed useradd (or worse, a silently recreated account with a fresh UID), and a daemon that refuses to bind a port already in use.

None of these failures are exotic. They are the ordinary consequence of writing scripts that mutate state without first inspecting state. In a production environment — where retries, concurrency, and human intervention are facts of life — a non-idempotent script is not a script. It is a liability with a shebang.

The principle is simple to state and demanding to practice: running the script again after a previous run should produce no additional changes.

Patterns That Make Bash Re-Runnable

Idempotency in shell scripting mostly comes down to replacing blind actions with checked ones.

Check before you create. Rather than:

useradd -m svc-deployer

write:

id -u svc-deployer &>/dev/null || useradd -m svc-deployer

Prefer declarative writes over appends. Appending configuration lines (echo 'export EDITOR=vim' >> ~/.bashrc) is the classic idempotency killer. Instead, match first:

grep -q 'EDITOR=' ~/.bashrc || echo 'export EDITOR=vim' >> ~/.bashrc

Use tools that are idempotent by design. mkdir -p, chmod (setting, not adding), systemctl enable, and package managers' install commands are safe to re-run. Learn which of your everyday commands share that property and which do not.

PowerShell Makes It Easier — If You Let It

PowerShell's cmdlets lean heavily toward idempotent semantics. Set-Item overwrites rather than duplicates. New-LocalUser throws if the account exists, which forces the check:

if (-not (Get-LocalUser -Name 'svc-deployer' -ErrorAction SilentlyContinue)) {
    New-LocalUser -Name 'svc-deployer' -NoPassword
}

The broader pattern generalizes: Get-* to inspect, compare against desired state, then Set-* or New-* only when the diff demands it. This is, not coincidentally, exactly the model that made DSC and Ansible successful. You do not need a configuration management platform to borrow its philosophy.

Testing for Idempotency Before the Scheduler Does

You can verify idempotency cheaply, and you should:

  1. Snapshot state before the first run — file hashes, user lists, installed packages, whatever the script touches.
  2. Run the script twice back to back.
  3. Diff the snapshots. Any difference after the second run is a bug.

Automate this in CI with a container or a disposable VM, and it becomes a five-second gate. Pester tests in PowerShell or Bats in Bash can encode the assertions permanently: run the setup, run it again, assert that the second run changed nothing and exited zero.

The teams that treat re-runnability as a testable property — rather than a hope — are the ones whose 2 a.m. retries end quietly instead of in a postmortem. The scheduler will eventually run your script twice. The only question is whether you find out first.

All Articles

Related Articles

The Script Nobody Can Read: Undocumented Automation as Institutional Risk

The Script Nobody Can Read: Undocumented Automation as Institutional Risk

Why Your Automation Scripts Get Slower Every Month (And What You Can Actually Do About It)

Why Your Automation Scripts Get Slower Every Month (And What You Can Actually Do About It)

Stop Reinventing the Script: A Practical Guide to Building Automation Libraries Your Team Will Keep Using

Stop Reinventing the Script: A Practical Guide to Building Automation Libraries Your Team Will Keep Using