Joulie

Core Concepts

Mon, 01 Jan 0001 00:00:00 +0000

Before installing Joulie, understand the control model.

Problem Joulie addresses

Clusters running AI/scientific workloads need better power control:

reduce energy use and power spikes,
keep workload performance predictable,
provide a path to greener operation (power envelope and carbon-aware strategies).

Joulie is currently a PoC focused on Kubernetes-native control loops and simulation.

Main components

Operator (cmd/operator): cluster-level policy brain
- decides desired node power profile/cap assignments
- writes desired state as NodePowerProfile
Agent (cmd/agent): node-level actuator
- reads desired state and telemetry configuration
- enforces power controls (CPU now, GPU path planned)
- exports metrics/status
Simulator (simulator/): digital-twin execution environment
- keeps scheduling real, simulates telemetry/control behavior
- enables repeatable experiments without requiring real hardware writes

Key CRDs

NodePowerProfile (joulie.io/v1alpha1)
- desired node policy state (performance / eco, optional power cap)
TelemetryProfile (joulie.io/v1alpha1)
- where telemetry/control inputs come from (host, http, …), and how controls are sent

Policy states and intent

Node supply is represented through joulie.io/power-profile:

Quickstart

Mon, 01 Jan 0001 00:00:00 +0000

This page is the fastest path to run Joulie. For conceptual context first, read Core Concepts.

Prerequisites

Kubernetes cluster with worker nodes
Node Feature Discovery (NFD) deployed
Optional for real enforcement: nodes exposing writable power interfaces
- RAPL power limit files, or
- cpufreq sysfs interfaces

Install from release (recommended)

Install directly from OCI chart release:

helm upgrade --install joulie oci://registry.cern.ch/mbunino/joulie/joulie \
 --version <version> \
 -n joulie-system \
 --create-namespace \
 -f values/joulie.yaml

Label nodes managed by the operator

Important: Joulie will only target nodes with a specific label, and ignore all the others. By default, install does not auto-select nodes. Default expected selector value is:

Pod Compatibility for Joulie

Mon, 01 Jan 0001 00:00:00 +0000

Joulie uses Kubernetes scheduling constraints as the single source of truth for workload placement intent.

Power profile supply is exposed on node label:

joulie.io/power-profile=performance
joulie.io/power-profile=draining-performance (temporary transition state)
joulie.io/power-profile=eco

Workload behavior:

performance workload: require joulie.io/power-profile=performance
eco workload: require joulie.io/power-profile=eco
unconstrained workload: no power-profile affinity, can run on either profile

Base Pod Spec

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17


apiVersion: apps/v1
kind: Deployment
metadata:
 name: my-workload
spec:
 replicas: 1
 selector:
 matchLabels:
 app: my-workload
 template:
 metadata:
 labels:
 app: my-workload
 spec:
 containers:
 - name: app
 image: ghcr.io/example/app:latest

Performance-only Pod (lines to add)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25


apiVersion: apps/v1
kind: Deployment
metadata:
 name: my-workload
spec:
 replicas: 1
 selector:
 matchLabels:
 app: my-workload
 template:
 metadata:
 labels:
 app: my-workload
 spec:
 affinity:
 nodeAffinity:
 requiredDuringSchedulingIgnoredDuringExecution:
 nodeSelectorTerms:
 - matchExpressions:
 - key: joulie.io/power-profile
 operator: In
 values: ["performance"]
 containers:
 - name: app
 image: ghcr.io/example/app:latest

Eco-only Pod (lines to add)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25


apiVersion: apps/v1
kind: Deployment
metadata:
 name: my-workload
spec:
 replicas: 1
 selector:
 matchLabels:
 app: my-workload
 template:
 metadata:
 labels:
 app: my-workload
 spec:
 affinity:
 nodeAffinity:
 requiredDuringSchedulingIgnoredDuringExecution:
 nodeSelectorTerms:
 - matchExpressions:
 - key: joulie.io/power-profile
 operator: In
 values: ["eco"]
 containers:
 - name: app
 image: ghcr.io/example/app:latest

Unconstrained Pod

Do not set power-profile affinity. Kubernetes can schedule it on either eco or performance nodes.

Agent Runtime Modes

Mon, 01 Jan 0001 00:00:00 +0000

The agent supports two runtime modes:

daemonset: real-hardware mode, one pod per real node.
pool: simulation mode, one pod hosts many logical per-node controllers.

Chart templates:

charts/joulie/templates/agent-daemonset.yaml
charts/joulie/templates/agent-statefulset.yaml

DaemonSet mode (real hardware)

Required runtime settings

securityContext.privileged: true
Host mount:
- host path /sys -> container path /host-sys
Env:
- NODE_NAME from spec.nodeName
- AGENT_MODE=daemonset (default)
- optional RECONCILE_INTERVAL (default 20s)
- optional SIMULATE_ONLY=true (skip host writes, log requested actions)
- optional METRICS_ADDR (default :8080)

Pool mode (KWOK / simulation)

Pool mode preserves per-node semantics but shards logical node controllers across replicas.

CRD and Policy Model

Mon, 01 Jan 0001 00:00:00 +0000

This page defines Joulie’s core contract:

demand comes from pod scheduling constraints,
supply is exposed by node power-profile labels,
desired state is published through NodePowerProfile.

APIs

Group/version:

joulie.io/v1alpha1

CRDs:

NodePowerProfile (nodepowerprofiles, cluster-scoped)
TelemetryProfile (telemetryprofiles, cluster-scoped)

CRD definitions live in:

config/crd/bases/joulie.io_nodepowerprofiles.yaml
config/crd/bases/joulie.io_telemetryprofiles.yaml

Demand model (workloads)

Workload class is inferred from Kubernetes scheduling constraints on key:

joulie.io/power-profile

Classification:

performance demand:
- pod requires joulie.io/power-profile=performance
eco demand:
- pod requires joulie.io/power-profile=eco
general demand:
- no explicit power-profile requirement (unconstrained)

Classification source is affinity/selector, not a custom intent label.

Workload and Power Simulator

Mon, 01 Jan 0001 00:00:00 +0000

This document defines the Joulie simulator design and how it integrates with Joulie.

Architecture at a glance

The simulator extends the same control path used on real nodes:

Operator writes desired node profile (NodePowerProfile).
Agent reads desired state and sends control intents.
Simulator emulates telemetry/control behavior per node and exposes HTTP endpoints.
Next reconcile loop reacts to updated simulated state.

The diagram shows the end-to-end loop:

Joulie Operator

Mon, 01 Jan 0001 00:00:00 +0000

The operator is Joulie’s cluster-level decision engine.

It does not write host power interfaces directly. Instead, it decides desired node states and publishes them through Kubernetes objects and labels.

Responsibilities

At each reconcile tick, the operator:

selects eligible managed nodes,
classifies workload demand from pod scheduling constraints,
runs a policy algorithm to compute a plan,
applies transition guards for safe downgrades,
writes desired node targets (NodePowerProfile) and node supply labels.

The agent then enforces those targets node-by-node.

Workload Simulator

Mon, 01 Jan 0001 00:00:00 +0000

This page documents the workload-side simulation model.

Scope

The workload simulator handles:

trace/job ingestion,
pod creation and placement via real scheduler,
per-job progress updates,
completion and pod deletion,
class inference from scheduling constraints.

Power/control dynamics are documented separately in:

Power Simulator

Trace-driven workload model

Enable with:

SIM_WORKLOAD_TRACE_PATH=/path/to/trace.jsonl

The simulator loads type=job records and schedules pods over time according to submit offsets.

Helper tools:

simulator/cmd/workloadgen (synthetic traces)
simulator/cmd/traceextract (normalize/extract traces)

Per-job execution model

Each job tracks:

Joulie Agent

Mon, 01 Jan 0001 00:00:00 +0000

The agent is Joulie’s node-side enforcement component.

It consumes desired state and applies node-local controls through configured backends.

Responsibilities

At each reconcile tick, the agent:

identifies its node scope (single node in daemonset mode, sharded set in pool mode),
reads desired target (NodePowerProfile) for each owned node,
reads telemetry/control routing (TelemetryProfile),
applies controls (host or HTTP),
exports metrics and status.

Inputs and outputs

Inputs:

NodePowerProfile for desired profile/cap
TelemetryProfile for source/control backend selection
node capability hints (for example NFD labels)

Outputs:

Power Simulator

Mon, 01 Jan 0001 00:00:00 +0000

This page documents the power/control-side simulation model.

Scope

The power simulator handles:

node power-state variables,
RAPL/DVFS control ingestion,
frequency ramp dynamics,
power computation and capping,
energy integration.

Workload execution model is documented separately in:

Workload Simulator

Per-node state

Each simulated node tracks:

CapWatts
TargetThrottlePct
ThrottlePct
FreqScale in [f_min/f_max, 1]
CPUUtil in [0,1]
CapSaturated

Power model

At update time:

P = P_idle + (P_max - P_idle) * util^alpha * freqScale^beta

Where:

Policy Algorithms

Mon, 01 Jan 0001 00:00:00 +0000

This page documents the controller policy algorithms implemented in cmd/operator/main.go.

Use this page after:

Classification Input

Policy demand classification is derived from pod scheduling constraints on joulie.io/power-profile:

performance-only: pod can run only on performance (and transitional draining-performance).
eco-only: pod can run only on eco.
general (implicit unconstrained): no explicit power-profile constraint, or both profiles allowed.
unknown: unsupported/ambiguous constraint shape.

For safety, unknown is treated as performance-sensitive in downgrade guards.

Simulator Metrics

Mon, 01 Jan 0001 00:00:00 +0000

This page documents Prometheus metrics exposed by the simulator (simulator/cmd/simulator/main.go).

Endpoint:

path: /metrics
address: simulator HTTP listen address (SIM_ADDR, default :18080)

Related debug endpoints (non-Prometheus):

/debug/nodes
/debug/events
/debug/energy

HTTP/request metrics

joulie_sim_requests_total{route,method,status} (counter)
- total HTTP requests by route/method/status
joulie_sim_request_duration_seconds{route,method} (histogram)
- request latency

Control-path metrics

joulie_sim_controls_total{node,action} (counter)
- received control actions by node/action
joulie_sim_control_actions_total{node,action,result} (counter)
- control action outcomes
- result: applied|blocked|error

Per-node simulated state metrics

joulie_sim_node_cap_watts{node} (gauge)
- current simulated effective cap
joulie_sim_node_rapl_cap_watts{node} (gauge)
- simulated RAPL cap value
joulie_sim_node_throttle_pct{node} (gauge)
- simulated DVFS throttle percent
joulie_sim_node_power_watts{node} (gauge)
- simulated node power
joulie_sim_node_cpu_util{node} (gauge)
- simulated CPU utilization
joulie_sim_node_freq_scale{node} (gauge)
- simulated frequency scale
joulie_sim_node_running_pods{node} (gauge)
- running pods observed on the node
joulie_sim_node_class_info{node,class} (gauge)
- class assignment marker (1 on active class)

Workload/job metrics

joulie_sim_job_submitted_total{class} (counter)
- jobs submitted by class
joulie_sim_job_completed_total{class,node} (counter)
- jobs completed by class and node
joulie_sim_job_completion_seconds (histogram)
- job completion latency distribution

Notes

Prometheus metrics capture online simulator state and request/control behavior.
Integrated node/cluster energy totals are exposed through /debug/energy (JSON), not as Prometheus time series in the current implementation.

Input Telemetry and Actuation Interfaces

Mon, 01 Jan 0001 00:00:00 +0000

This page describes runtime IO contracts:

how Joulie reads telemetry inputs,
how Joulie sends control intents.

It is not the /metrics exposition contract. For exported metrics, see Metrics Reference.

Why this abstraction exists

Joulie must run in two worlds with the same control logic:

real hardware clusters,
simulator/KWOK clusters.

So agent/operator logic depends on provider interfaces, not directly on sysfs or simulator HTTP shape.

Telemetry provider model

Telemetry input can come from:

Metrics Reference

Mon, 01 Jan 0001 00:00:00 +0000

Joulie exposes Prometheus metrics from multiple components.

This page covers operator + agent metrics. Simulator metrics are documented separately in:

Simulator Metrics

For telemetry/control input interfaces (host/http routing), see:

Input Telemetry and Actuation Interfaces

Endpoints by component

Agent:
- path: /metrics
- default address: :8080
- env override: METRICS_ADDR
Operator:
- path: /metrics
- default address: :8081
- env override: METRICS_ADDR

Agent metrics

Backend and selected cap

joulie_backend_mode{node,mode} (gauge)
- mode: none|rapl|dvfs
- active mode is 1, others 0
joulie_policy_cap_watts{node,policy} (gauge)
- current selected policy cap in watts

RAPL power/energy

joulie_rapl_energy_uj{node,zone} (gauge)
- latest raw RAPL energy counter in microjoules
joulie_rapl_estimated_power_watts{node,zone} (gauge)
- per-zone estimated power from energy deltas
joulie_rapl_package_total_power_watts{node} (gauge)
- sum of package-level estimated power

DVFS controller

joulie_dvfs_observed_power_watts{node} (gauge)
- observed package power used by DVFS controller
joulie_dvfs_ema_power_watts{node} (gauge)
- EMA-smoothed power used for decisions
joulie_dvfs_throttle_pct{node} (gauge)
- current throttle percentage
joulie_dvfs_above_trip_count{node} (gauge)
- consecutive above-threshold samples
joulie_dvfs_below_trip_count{node} (gauge)
- consecutive below-threshold samples
joulie_dvfs_actions_total{node,action} (counter)
- action: throttle_up|throttle_down

CPU frequency observability

joulie_dvfs_cpu_cur_freq_khz{node,cpu} (gauge)
- current CPU/policy frequency in kHz
joulie_dvfs_cpu_max_freq_khz{node,cpu} (gauge)
- enforced max frequency cap in kHz

Reliability

joulie_reconcile_errors_total{node} (counter)
- reconcile-loop errors

Operator metrics

FSM state and profile label

joulie_operator_node_state{node,state} (gauge)
- state: ActivePerformance|DrainingPerformance|ActiveEco
- active state is 1, others 0
joulie_operator_node_profile_label{node,profile} (gauge)
- operator-applied node label view
- profile: performance|draining-performance|eco
- active profile is 1, others 0

Transition accounting

joulie_operator_state_transitions_total{node,from_state,to_state,result} (counter)
- transition events emitted by operator
- result:
  - applied: transition committed
  - deferred: transition blocked/deferred by safeguards

Notes

Metrics are pull-based; values depend on scrape interval.
Highest cardinality is usually per-CPU frequency series.

KWOK Benchmark Experiment

Mon, 01 Jan 0001 00:00:00 +0000

This page reports the current benchmark results from the first experiment in:

experiments/01-kwok-benchmark/

Scope

The benchmark compares three baselines:

A: simulator only (Joulie-free)
B: Joulie with static partition policy
C: Joulie with queue-aware policy

It evaluates throughput/makespan vs energy under real Kubernetes scheduling with KWOK nodes and simulated power control.

Joulie

Core Concepts

Problem Joulie addresses

Main components

Key CRDs

Policy states and intent

Quickstart

Prerequisites

Install from release (recommended)

Label nodes managed by the operator

Pod Compatibility for Joulie

Base Pod Spec

Performance-only Pod (lines to add)

Eco-only Pod (lines to add)

Unconstrained Pod

Agent Runtime Modes

DaemonSet mode (real hardware)

Required runtime settings

Pool mode (KWOK / simulation)

CRD and Policy Model

APIs

Demand model (workloads)

Workload and Power Simulator

Architecture at a glance

Joulie Operator

Responsibilities

Workload Simulator

Scope

Trace-driven workload model

Per-job execution model

Joulie Agent

Responsibilities

Inputs and outputs

Power Simulator

Scope

Per-node state

Power model

Policy Algorithms

Classification Input

Simulator Metrics

HTTP/request metrics

Control-path metrics

Per-node simulated state metrics

Workload/job metrics

Notes

Input Telemetry and Actuation Interfaces

Why this abstraction exists

Telemetry provider model

Metrics Reference

Endpoints by component

Agent metrics

Backend and selected cap

RAPL power/energy

DVFS controller

CPU frequency observability

Reliability

Operator metrics

FSM state and profile label

Transition accounting

Notes

KWOK Benchmark Experiment

Scope

Experimental setup

Cluster and nodes

Hardware models in simulator