Joulie

Core Concepts

Mon, 01 Jan 0001 00:00:00 +0000

Before installing Joulie, understand the control model.

What Joulie is

Joulie is a Kubernetes-native energy management system that uses per-node digital twins to optimize data center power consumption.

It continuously ingests telemetry from every node (CPU/GPU power draw via RAPL and NVML/DCGM, read from Prometheus as the OpenTelemetry hardware metrics hw.energy and hw.power or as Kepler energy counters, and per-pod resource utilization via cAdvisor) to maintain an up-to-date model of each node’s thermal and power state.

Quickstart

Mon, 01 Jan 0001 00:00:00 +0000

This page is the fastest path to run Joulie. For conceptual context first, read Core Concepts.

Prerequisites

Kubernetes cluster with worker nodes
Node Feature Discovery (NFD) deployed
Optional for real enforcement: nodes exposing writable power interfaces
- RAPL power limit files, or
- cpufreq sysfs interfaces
Prometheus, for measured node power. The chart reads node power from Prometheus by default (NODE_POWER_SOURCE=prometheus): it expects Prometheus at http://prometheus-operated.monitoring:9090 with its OTLP receiver on (--web.enable-otlp-receiver), receiving hw.energy or hw.power for each node. Without that data every node reads 0 W, with source prometheus-no-data.
- values/prometheus-grafana.yaml configures kube-prometheus-stack this way (Prometheus settings).
- examples/11-otel-collector-bridge converts Kepler, dcgm-exporter and AMD device-metrics-exporter metrics into those names.
- To run without it, set controllerManager.env.NODE_POWER_SOURCE=static (0 W for every node).

Install from release (recommended)

Install directly from OCI chart release:

Pod Compatibility for Joulie

Mon, 01 Jan 0001 00:00:00 +0000

Joulie uses a single pod annotation to express workload placement intent:

joulie.io/workload-class: performance | standard

The scheduler extender reads this annotation and steers pods accordingly. No node affinity rules are needed.

Workload classes

Class	Behavior
`performance`	Must run on full-power nodes. The extender hard-rejects eco nodes.
`standard`	Default. Can run on any node. Adaptive scoring steers toward eco when performance nodes are congested.

If no annotation is present, the extender treats it as standard.

Agent Runtime Modes

Mon, 01 Jan 0001 00:00:00 +0000

The chart runs the agent in one of three modes, or not at all (agent.mode):

`agent.mode`	Pods	Use
`daemonset` (default)	one pod per managed node (`charts/joulie/templates/agent-daemonset.yaml`)	real hardware
`pool`	a StatefulSet whose pods each run many logical per-node agents (`agent-statefulset.yaml`)	KWOK experiments over the simulator’s HTTP API
`emulated`	one Deployment per node of the simulated cluster (`agent-emulated.yaml`), reading that node’s emulated `/sys` and GPU tools	the simulated cluster
`none`	no agent

DaemonSet mode (real hardware)

Required runtime settings

securityContext.privileged: true
Host mount:
- host path /sys -> container path /host-sys
Env:
- NODE_NAME from spec.nodeName
- AGENT_MODE=daemonset (default)
- optional RECONCILE_INTERVAL (default 20s)
- optional SIMULATE_ONLY=true (skip host writes, log requested actions)
- optional METRICS_ADDR (default :8080)

Pool mode (KWOK / simulation)

Pool mode preserves per-node semantics but shards logical node controllers across replicas.

CPU Support and Power Capping

Mon, 01 Jan 0001 00:00:00 +0000

Joulie supports node-level CPU power capping through NodeTwin intents enforced by the agent.

Contract model

CPU intent is defined in NodeTwin.spec.cpu:

packagePowerCapWatts (optional absolute cap)
packagePowerCapPctOfMax (optional normalized profile intent)

Precedence:

packagePowerCapWatts if present
otherwise packagePowerCapPctOfMax

Policy behavior

Controller manager profile assignment remains performance vs eco. CPU cap values are generated per profile and written into NodeTwin.spec:

performance profile typically maps to a higher cap (often 100%)
eco profile maps to a lower cap

For heterogeneous nodes, percentage-based intent remains useful because each node resolves normalized intent using node-local capabilities. If percentage intent cannot be converted to watts (for example missing RAPL range), the agent applies a DVFS percent fallback path when possible.

CRD and Policy Model

Mon, 01 Jan 0001 00:00:00 +0000

This page defines Joulie’s core contract:

demand comes from pod scheduling constraints,
supply is exposed by node power-profile labels,
discovered hardware is published through NodeHardware,
desired state is published through NodeTwin.

APIs

Group/version:

joulie.io/v1alpha1

CRDs:

NodeHardware (nodehardwares, cluster-scoped)
NodeTwin (nodetwins, cluster-scoped)

CRD definitions live in:

config/crd/bases/joulie.io_nodehardwares.yaml
config/crd/bases/joulie.io_nodetwins.yaml

Demand model (workloads)

Workload class is determined from the joulie.io/workload-class pod annotation:

performance demand: pod carries joulie.io/workload-class: performance.
standard demand (default): no annotation, or joulie.io/workload-class: standard. Can run on any node; adaptive scoring steers toward eco when performance nodes are congested.

Supply model (nodes)

Node supply is represented by:

GPU Support (NVIDIA, AMD, Intel)

Mon, 01 Jan 0001 00:00:00 +0000

Joulie supports node-level GPU power-cap intents for NVIDIA, AMD and Intel.

Validation status

GPU support has been validated in simulator mode only (no bare-metal GPU access yet). The host code paths are designed to work on bare metal (NVIDIA, AMD, Intel) when GPU nodes are available.

Contract model

NodeTwin.spec.gpu.powerCap defines a per-GPU cap intent:

scope: perGpu
capWattsPerGpu (absolute, optional)
capPctOfMax (percentage, optional)

Precedence:

capWattsPerGpu if present
otherwise capPctOfMax

The same cap is applied uniformly to all GPUs on the node.

Simulated Cluster

Mon, 01 Jan 0001 00:00:00 +0000

The simulated cluster runs the real Joulie components on emulated bare-metal nodes, on one machine with Docker. The agent discovers and caps emulated hardware through the same files and tools it uses on a real node, the controller manager reads node power from the simulator’s OpenTelemetry metrics in Prometheus, and the scheduler extender scores real pods. The scripts in scripts/simcluster drive it, and tests/simcluster checks it end to end.

Use it to see a change to the agent, the controller manager, the extender, the chart or the simulator work across components. A change to discovery or capping alone is checked faster by the hwemu tests in cmd/agent (go test ./cmd/agent/ -run Hwemu).

Workload Scenarios

Mon, 01 Jan 0001 00:00:00 +0000

A workload scenario runs a workload through the simulated cluster and checks what happened: whether the jobs completed, where they were placed, how long they took, what energy they used, and how Joulie’s settings compare with each other or with no Joulie at all. The jobs are real Kubernetes Jobs, scheduled by the real scheduler and the Joulie extender onto KWOK Nodes, and the simulator’s v1 workload engine drives them on the emulated hardware, so a cap the agent writes slows the jobs on that Node.

Installation

Mon, 01 Jan 0001 00:00:00 +0000

This page installs the simulator into a Kubernetes cluster you already have, or runs it from a workstation. To try Joulie on emulated nodes with nothing but Docker, use the simulated cluster instead: it creates the cluster and installs everything.

Every command below uses the namespace joulie-sim-demo and the release joulie-simulator, the defaults of make simulator-install.

Prerequisites

A Kubernetes cluster (real, or kind for local work), with kubectl pointed at it.
helm v3 or later.

Install with Helm

The simulator chart is published next to the Joulie chart:

Emulate Mode

Mon, 01 Jan 0001 00:00:00 +0000

Emulate mode emulates the bare-metal nodes of a cluster spec at the boundary the agent sees: the files it reads and writes, and the GPU tools it runs. An unmodified agent then runs its real discovery and capping code against each node, while Kubernetes scheduling stays real. The simulated cluster runs it; to run it yourself, see Installation. Nodes the spec does not list keep the label-based model of KWOK and standalone modes, in the same simulator.

KWOK and Standalone Modes

Mon, 01 Jan 0001 00:00:00 +0000

Outside emulate mode the simulator models each node from its labels: it keeps per-node power, cap and thermal state in memory and serves it over HTTP, with no files and no GPU tools. It runs in one of two ways:

KWOK mode: in a Kubernetes cluster whose API server and scheduler are real and whose worker nodes are KWOK fake nodes. The real controller manager plans, the real scheduler extender scores, and the agent in pool mode reads telemetry from and sends caps to the simulator’s HTTP API. The benchmark experiments run this way.
Standalone mode (SIM_STANDALONE=true): no Kubernetes at all. The simulator places the jobs of a trace and runs the policy baselines itself, in simulated time, as fast as the CPU allows.

How KWOK mode works

Pod placement and pod lifetime stay real in Kubernetes, the one source of truth for where work runs. The simulator only derives each node’s hardware state from what runs there:

Joulie Controller Manager

Mon, 01 Jan 0001 00:00:00 +0000

The controller manager is Joulie’s cluster-level decision engine.

It does not write host power interfaces directly. Instead, it decides desired node states and publishes them through Kubernetes objects and labels.

In practice, the controller manager answers one question over and over: which nodes should currently supply performance capacity, and which can safely supply eco capacity?

Responsibilities

At each reconcile tick, the controller manager:

selects eligible managed nodes,
reads NodeHardware when available and falls back to node labels when it is not,
resolves hardware identity against the shared inventory,
classifies workload demand from pod scheduling constraints,
runs a policy algorithm (pkg/controller/policy/) to compute a plan,
applies transition guards for safe downgrades,
writes desired node targets (NodeTwin.spec: caps and, with governor control on, governor presets) and the joulie.io/power-profile node label.

The agent then enforces those targets node-by-node.

Workload Generation

Mon, 01 Jan 0001 00:00:00 +0000

workloadgen (simulator/cmd/workloadgen) writes synthetic traces of AI cluster workloads for the simulator. This page says what a trace contains and how the generator samples it; Workload Simulator says how the simulator runs it.

go run ./simulator/cmd/workloadgen --out trace.jsonl --jobs 200 --seed 7

The generator models AI-oriented Kubernetes clusters: CPU and GPU jobs, memory-bound jobs, and workloads made of several pods, such as distributed training and hyperparameter searches. It does not model network bandwidth, disk throughput or framework communication protocols: their cost is folded into the utilisation and bottleneck signals of each workload (gpuUtilization, memoryIntensity, cpuFeedIntensityGpu).

Workload Simulator

Mon, 01 Jan 0001 00:00:00 +0000

This page says how the simulator runs a workload trace: which Kubernetes objects it creates, how each job progresses under caps, and how jobs load the hardware model. Workload Generation describes the traces; Power Simulator the power side of a node.

Two engines: legacy and v1

SIM_WORKLOAD_MODEL selects the engine that runs a trace.

	legacy (outside emulate mode)	v1 (emulate mode)
Started by	`SIM_WORKLOAD_TRACE_PATH`, replayed from startup	the run API, or `SIM_WORKLOAD_TRACE_PATH` as an implicit run
Kubernetes objects	one bare pod per job	one `batch/v1` Job per group (`workloadId`, `podRole`, submit offset), Indexed when larger than one, `backoffLimit: 0`, deleted `SIM_WORKLOAD_JOB_TTL` after it finishes
Class	inferred from the pod’s affinity	declared by the annotation `joulie.io/workload-class` (`SIM_WORKLOAD_INTENT`)
Placement	`type=kwok` and the KWOK toleration	the trace’s placement, plus the selector `sim.joulie.io/cluster=<spec>` and the KWOK toleration
Completion	the pod is deleted	the pod is annotated `sim.joulie.io/complete: "true"` and KWOK’s `pod-complete` Stage moves it to `Succeeded`
A job that cannot progress	completed by force after `SIM_WORKLOAD_STALL_FORCE_COMPLETE` (120 s)	reported as stalled (`joulie.sim.job.stalled`, `joulie.sim.run.stalled`), never completed
Creates per tick	20	no cap
Speed	the node model’s frequency scale and GPU cap ratio	the throughput of the emulated hardware the job runs on, under the limits the agent wrote
Energy	node and cluster totals at `/debug/energy`	each node’s energy ledger delta, attributed per job and class by the jobs’ requests; the unrequested share of idle package power is class `idle`
Results	`joulie_sim_job_*` counters at `/metrics`	OTLP JSON Lines in the run directory, plus the run API’s status

In a v1 run a job is submitted at the first tick after its create call returns, scheduled at the first tick that sees its pod bound, and done at the tick its work runs out, so its times are exact to one SIM_EMULATE_TICK. A finish with cleanup deletes the run’s Job objects or bare pods by the run’s label, one call per namespace (Run API).

Power Simulator

Mon, 01 Jan 0001 00:00:00 +0000

This page says how the simulator advances a modelled node (any node outside the emulate-mode cluster spec): its control state, its physics step, and the telemetry it serves. The physical model itself, its parameters and their sources, is in Hardware Modeling; how jobs progress on a node is in Workload Simulator.

The power simulator:

keeps per-node control state (CPU cap, DVFS throttle, GPU cap),
applies control actions from /control/{node},
advances caps, frequency, power and temperature with their settling lags,
serves power telemetry on /otlp/telemetry/{node} and /telemetry/{node},
integrates energy over time (/debug/energy).

The physics step

A modelled node advances only in a physics step, in simulated time, never when it is read:

Hardware Calibration

Mon, 01 Jan 0001 00:00:00 +0000

hwcalib fit (simulator/cmd/hwcalib, package simulator/pkg/calib) fits the physics of a hardware profile to a sweep measured on one of your nodes, and writes the result as a local profile the simulator loads from SIM_PROFILES_DIR. The builtin profiles keep vendor figures and placeholders; a calibrated profile replaces them with what your machine draws. hwcalib sweep records such a sweep on the node itself (see Recording on the node).

Recording a sweep

A sweep holds the node at a series of setpoints and records, at every sample, the node’s readings and the setpoints it held. Each setpoint should last long enough to settle: the fit drops the start of every setpoint (see Segments and steady windows) and needs at least 5 samples after that, so with a 2 s sample period a setpoint of 22 s or more is enough.

Simulator Metrics

Mon, 01 Jan 0001 00:00:00 +0000

The simulator publishes three kinds of telemetry:

Prometheus metrics about itself at /metrics (joulie_sim_*), for Prometheus to scrape.
OpenTelemetry metrics and events about the emulated nodes and the facility, in emulate mode only. Metrics are pushed as OTLP JSON over HTTP; events are written to events.jsonl and pushed when a logs endpoint is set.
Compatibility exporters, in emulate mode only: the Prometheus formats of real exporters (Kepler, dcgm-exporter, the ROCm device-metrics-exporter) at /exporters/<name>/metrics, opt-in.

The OpenTelemetry names are spelled once, in pkg/otel/semconv, and the controller manager reads the same names back from Prometheus. Their Prometheus spelling is computed by pkg/otel/promname.

Hardware Modeling and Physical Power Model

Mon, 01 Jan 0001 00:00:00 +0000

This page documents how Joulie models CPUs and GPUs across the project using a mix of:

official vendor specifications and management APIs,
public measured power curves, and
explicit proxy models where public exact curves are not yet available.

It serves two closely related purposes:

for the agent, it describes the hardware assumptions used to resolve caps, interpret device limits, and reason about how throttling affects attainable performance
for the simulator, it describes the physical model used to turn utilization and control actions into simulated power and slowdown

Quick summary

If you want the short version before the details:

Joulie Agent

Mon, 01 Jan 0001 00:00:00 +0000

The agent is Joulie’s node-side enforcement component.

It consumes desired state and applies node-local controls through configured backends.

If the controller manager decides “this node should now behave like eco” or “this node should stay performance”, the agent is the component that turns that intent into concrete control actions.

Responsibilities

At each reconcile tick, the agent:

identifies its node scope (single node in daemonset mode, sharded set in pool mode),
discovers local CPU/GPU hardware and runtime control capability,
publishes NodeHardware for each owned node,
reads desired target (NodeTwin.spec) for each owned node,
resolves telemetry/control backend from environment variables (default: host),
applies controls (host or HTTP),
exports metrics and status.

Inputs and outputs

Inputs:

Digital Twin

Mon, 01 Jan 0001 00:00:00 +0000

The digital twin is Joulie’s core predictive engine. It is a lightweight O(1) parametric model that predicts the impact of scheduling and power-cap decisions on node thermal and power state, without running a full simulation for each scheduling decision.

What the digital twin computes

For each managed node, the twin produces three scores stored in NodeTwin.status:

Signal	Range	Meaning
Power headroom	up to 100, can go negative	Fraction of the node’s capped power budget still unused. Higher is better for new workload placement. Negative means the node already draws more than its budget.
CoolingStress	0-100	Measured node power as a fraction of node TDP, scaled by an ambient temperature multiplier. High values indicate the node is near its thermal limit.
PSUStress	0-100	Rack power draw (or cluster power draw when no rack is known) as a fraction of a reference rack capacity. Computed and published, but not read by the scheduler today.

The twin also computes:

Policy Algorithms

Mon, 01 Jan 0001 00:00:00 +0000

This page documents the controller policy algorithms implemented in pkg/controller/policy/.

Use this page after:

Classification Input

Policy demand classification is derived from the joulie.io/workload-class pod annotation:

performance: pod carries joulie.io/workload-class: performance.
standard (default): no annotation or joulie.io/workload-class: standard.

Shared Reconcile Flow

Each reconcile tick:

Select eligible nodes from NODE_SELECTOR, excluding reserved and unschedulable nodes.
Build a hardware view from NodeHardware when available, otherwise from node labels/inventory fallback.
Build a desired plan with the selected policy.
Apply downgrade guard (sets NodeTwin.status.schedulableClass to draining while blocking pods still run).
Write NodeTwin.spec and update the joulie.io/power-profile node label.

In other words, static_partition and queue_aware_v1 decide how many nodes stay in performance, and the per-family split below decides which nodes those are.

Scheduler Extender

Mon, 01 Jan 0001 00:00:00 +0000

Joulie ships a scheduler extender that steers workloads toward appropriate nodes based on power profile, thermal stress, and hardware capabilities.

How a pod gets scheduled (end-to-end)

When a new pod is created in the cluster, the following sequence occurs:

1. Pod created (e.g., kubectl apply, Job controller, Deployment rollout)
 |
2. kube-scheduler picks up the unscheduled pod
 |
3. kube-scheduler runs its default filters (resource fits, taints, affinity)
 |
4. kube-scheduler calls Joulie's /filter endpoint
 | - Sends: pod spec + candidate node list
 | - Joulie reads pod annotation joulie.io/workload-class
 | - Performance pods: reject nodes with schedulableClass = eco or draining
 | - Standard pods: pass all nodes
 | - Returns: filtered node list + rejection reasons
 |
5. kube-scheduler calls Joulie's /prioritize endpoint
 | - Sends: pod spec + surviving node list
 | - Joulie reads NodeTwin CRs (cached, 30s TTL) for power state
 | - Joulie reads NodeHardware CRs (cached, 30s TTL) for hardware specs
 | - Joulie extracts pod CPU/GPU requests for marginal power estimation
 | - Joulie scores each node 0-100 using the scoring formula
 | - Returns: list of (node, score) pairs
 |
6. kube-scheduler combines Joulie scores with its own plugin scores
 |
7. Pod is bound to the highest-scoring node

The extender participates in steps 4 and 5 only. It does not replace the Kubernetes scheduler; it extends it with energy-aware filter and scoring logic.

Energy-Aware Scheduling

Mon, 01 Jan 0001 00:00:00 +0000

Joulie’s scheduler extender makes placement decisions informed by real-time energy telemetry, workload characteristics, and facility-level power conditions. This page describes the full pipeline from metrics collection through scoring and optional rescheduling.

End-to-end pipeline

The energy-aware scheduling pipeline has five stages:

RAPL/NVML readings as OpenTelemetry metrics (or Kepler)
 -> Prometheus (OTLP receiver, or scrape)
 -> Digital twin (NodeTwin.status)
 -> Scheduler extender (filter + score)
 -> Placement decision

Each stage runs independently and communicates through Kubernetes CRDs or Prometheus queries. There is no monolithic scheduling engine; each component does one thing and feeds the next.

Configuration Reference

Mon, 01 Jan 0001 00:00:00 +0000

Complete reference for all Joulie environment variables. These are set via Helm values or directly in the Deployment/DaemonSet manifests.

Defaults listed below are the code defaults. The Helm chart (charts/joulie/values.yaml) overrides some of them: notably, the controller manager NODE_SELECTOR defaults to joulie.io/managed=true in the chart even though the code default is node-role.kubernetes.io/worker.

Booleans. The agent, the controller manager and the simulator read every on/off variable the same way: true, 1, yes or on, and false, 0, no or off, in any case, with surrounding spaces ignored. Unset or empty means the default. Any other value keeps the default and logs one warning naming the variable, so a typo never flips a switch. The exceptions are SIM_OTEL_FILE_METRICS, OTEL_SDK_DISABLED and fakesmi’s HWEMU_UNPRIVILEGED, whose rows say how they are read.

Input Telemetry and Actuation Interfaces

Mon, 01 Jan 0001 00:00:00 +0000

This page describes runtime IO contracts:

how Joulie reads telemetry inputs,
how Joulie sends control intents.

If you want the CRD-level summary first, read CRD and Policy Model. This page is the detailed runtime reference for the telemetry and control contract.

It is not the /metrics exposition contract. For exported metrics, see Metrics Reference.

Why this abstraction exists

Joulie must run in two worlds with the same control logic:

real hardware clusters,
simulator/KWOK clusters.

So agent/controller manager logic depends on provider interfaces, not directly on sysfs or simulator HTTP shape.

Metrics Reference

Mon, 01 Jan 0001 00:00:00 +0000

Joulie exposes Prometheus metrics from multiple components.

This page covers controller manager + agent + scheduler extender metrics. Simulator metrics are documented separately in:

Simulator Metrics

For telemetry/control input interfaces (host/http routing), see:

Input Telemetry and Actuation Interfaces

Endpoints by component

Agent:
- path: /metrics
- default address: :8080
- env override: METRICS_ADDR
Controller manager:
- path: /metrics
- default address: :8081
- env override: METRICS_ADDR
Scheduler extender:
- path: /metrics
- default address: :9877
- env override: METRICS_ADDR

Agent metrics

Backend and selected cap

joulie_backend_mode{node,mode} (gauge)
- mode: none|rapl|dvfs
- active mode is 1, others 0
joulie_policy_cap_watts{node,policy} (gauge)
- current selected policy cap in watts

RAPL power/energy

joulie_rapl_energy_uj{node,zone} (gauge)
- latest raw RAPL energy counter in microjoules
joulie_rapl_estimated_power_watts{node,zone} (gauge)
- per-zone estimated power from energy deltas
joulie_rapl_package_total_power_watts{node} (gauge)
- sum of package-level estimated power

DVFS controller

joulie_dvfs_observed_power_watts{node} (gauge)
- observed package power used by DVFS controller
joulie_dvfs_ema_power_watts{node} (gauge)
- EMA-smoothed power used for decisions
joulie_dvfs_throttle_pct{node} (gauge)
- current throttle percentage
joulie_dvfs_above_trip_count{node} (gauge)
- consecutive above-threshold samples
joulie_dvfs_below_trip_count{node} (gauge)
- consecutive below-threshold samples
joulie_dvfs_actions_total{node,action} (counter)
- action: throttle_up|throttle_down

CPU frequency observability

joulie_dvfs_cpu_cur_freq_khz{node,cpu} (gauge)
- current CPU/policy frequency in kHz
joulie_dvfs_cpu_max_freq_khz{node,cpu} (gauge)
- enforced max frequency cap in kHz

Governors

Written by the agent’s governor control.

Upgrading

Mon, 01 Jan 0001 00:00:00 +0000

What to check when you upgrade a Joulie installation from v0.2 to v0.3. Go through the list before running helm upgrade.

Before the upgrade

Apply the new CRDs. helm upgrade never updates CRDs, and the API server prunes every field the installed CRD does not know, such as the governor fields of NodeTwin and NodeHardware. Apply them first:
```
kubectl apply -f charts/joulie/crds/
```
Without a checkout, take them from the release chart: helm pull oci://registry.cern.ch/joulie/joulie --version <version> --untar --untardir /tmp/joulie-chart, then kubectl apply -f /tmp/joulie-chart/joulie/crds/.

CPU-Only Benchmark

Mon, 01 Jan 0001 00:00:00 +0000

This page reports results from the CPU-only cluster benchmark experiment (KWOK, 40 nodes):

experiments/01-cpu-only-benchmark/

Note (2026-10-09): The results on this page, like the experiment’s REPORT.md and REPORT-standalone.md, were produced with an older simulator model and have not been rerun. The simulator reads the hardware catalog embedded in it, with a SPECpower curve for each CPU that has one, and thermal throttling is off unless configured. Simulator changes lists each change and how far it moves the experiment’s configs/benchmark-debug.yaml.

Governor Exploration

Mon, 01 Jan 0001 00:00:00 +0000

This page reports experiment 05, CPU and GPU governor presets as an energy dimension of their own, next to power caps:

experiments/05-governor-exploration/

Joulie writes a governor as a vendor-neutral preset in the NodeTwin (spec.cpu.governor.preset: performance, balanced, efficiency; spec.gpu.governor.preset: auto, sustained, efficient, low, peak), and the agent maps it to the native words of the node’s CPU frequency driver and GPU family. Governor control is off by default (GOVERNOR_CONTROL, GOVERNOR_WRITES, Configuration reference).

Heterogeneous GPU Cluster Benchmark

Mon, 01 Jan 0001 00:00:00 +0000

This page reports results from the heterogeneous GPU cluster benchmark experiment (KWOK, 41 nodes):

experiments/02-heterogeneous-benchmark/

Note (2026-10-09): The results on this page, like the experiment’s REPORT.md and REPORT-standalone.md, were produced with an older simulator model and have not been rerun. The simulator reads the hardware catalog embedded in it, with a SPECpower curve for each CPU that has one, and thermal throttling is off unless configured. Simulator changes lists each change and how far it moves the experiment’s configs/benchmark-debug.yaml.

Homogeneous H100 NVL Benchmark

Mon, 01 Jan 0001 00:00:00 +0000

This page reports results from the homogeneous H100 NVL cluster benchmark experiment (KWOK, 41 nodes):

experiments/03-homogeneous-h100-benchmark/

Note (2026-10-09): The results on this page, like the experiment’s REPORT.md and REPORT-standalone.md, were produced with an older simulator model and have not been rerun. The simulator reads the hardware catalog embedded in it, with a SPECpower curve for each CPU that has one, and thermal throttling is off unless configured. Simulator changes lists each change and how far it moves the experiment’s configs/benchmark-debug.yaml.

Scoring Formula Validation

Mon, 01 Jan 0001 00:00:00 +0000

This page reports results from the energy-aware scheduling formula validation experiment:

experiments/04-scoring-formula-validation/

Objective

Validate Joulie’s energy-aware scheduling formula by demonstrating that power-aware scheduling improves energy efficiency compared to standard Kubernetes bin-packing (MostAllocated), using a Modelica FMU (DXCooled Airside Economizer) for physically-accurate cooling/PUE computation.

Two scales tested:

Small cluster (28 nodes): formula tuning and component selection
Large cluster (2,500 nodes): production-scale validation with H100 GPUs

The experiment also validated the evolution from a legacy multi-component formula to the current streamlined Joulie scoring formula.