Joulie

Quickstart

Mon, 01 Jan 0001 00:00:00 +0000

Prerequisites

Kubernetes cluster with worker nodes
Node Feature Discovery (NFD) deployed
Access to push images to registry.cern.ch/mbunino/joulie
Optional for real enforcement: nodes exposing writable power interfaces
- RAPL power limit files, or
- cpufreq sysfs interfaces

1. Install from release (recommended)

Install directly from OCI chart release:

helm upgrade --install joulie oci://registry.cern.ch/mbunino/joulie/joulie \
 --version <version> \
 -n joulie-system \
 --create-namespace \
 -f values/joulie.yaml

2. Install from source (when developing)

If you changed source code, from repo root:

Agent Runtime Modes

Mon, 01 Jan 0001 00:00:00 +0000

The agent supports two runtime modes:

daemonset: real-hardware mode, one pod per real node.
pool: simulation mode, one pod hosts many logical per-node controllers.

Chart templates:

charts/joulie/templates/agent-daemonset.yaml
charts/joulie/templates/agent-statefulset.yaml

DaemonSet mode (real hardware)

Required runtime settings

securityContext.privileged: true
Host mount:
- host path /sys -> container path /host-sys
Env:
- NODE_NAME from spec.nodeName
- AGENT_MODE=daemonset (default)
- optional RECONCILE_INTERVAL (default 20s)
- optional SIMULATE_ONLY=true (skip host writes, log requested actions)
- optional METRICS_ADDR (default :8080)

Pool mode (KWOK / simulation)

Pool mode preserves per-node semantics but shards logical node controllers across replicas.

Pod Compatibility for Joulie

Mon, 01 Jan 0001 00:00:00 +0000

Joulie uses Kubernetes scheduling constraints as the single source of truth for workload placement intent.

Power profile supply is exposed on node label:

joulie.io/power-profile=performance
joulie.io/power-profile=eco
joulie.io/power-profile=draining-performance (transition state)

Workload behavior:

performance workload: require joulie.io/power-profile=performance
eco workload: require joulie.io/power-profile=eco
unconstrained workload: no power-profile affinity, can run on either profile

Base Pod Spec

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17


apiVersion: apps/v1
kind: Deployment
metadata:
 name: my-workload
spec:
 replicas: 1
 selector:
 matchLabels:
 app: my-workload
 template:
 metadata:
 labels:
 app: my-workload
 spec:
 containers:
 - name: app
 image: ghcr.io/example/app:latest

Performance-only Pod (lines to add)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25


apiVersion: apps/v1
kind: Deployment
metadata:
 name: my-workload
spec:
 replicas: 1
 selector:
 matchLabels:
 app: my-workload
 template:
 metadata:
 labels:
 app: my-workload
 spec:
 affinity:
 nodeAffinity:
 requiredDuringSchedulingIgnoredDuringExecution:
 nodeSelectorTerms:
 - matchExpressions:
 - key: joulie.io/power-profile
 operator: In
 values: ["performance"]
 containers:
 - name: app
 image: ghcr.io/example/app:latest

Eco-only Pod (lines to add)

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25


apiVersion: apps/v1
kind: Deployment
metadata:
 name: my-workload
spec:
 replicas: 1
 selector:
 matchLabels:
 app: my-workload
 template:
 metadata:
 labels:
 app: my-workload
 spec:
 affinity:
 nodeAffinity:
 requiredDuringSchedulingIgnoredDuringExecution:
 nodeSelectorTerms:
 - matchExpressions:
 - key: joulie.io/power-profile
 operator: In
 values: ["eco"]
 containers:
 - name: app
 image: ghcr.io/example/app:latest

Unconstrained Pod

Do not set power-profile affinity. Kubernetes can schedule it on either eco or performance nodes.

Workload and Power Simulator

Mon, 01 Jan 0001 00:00:00 +0000

This document defines the Joulie simulator design and how it integrates with Joulie.

Goals

Keep Kubernetes scheduling real (real pod placement/lifecycle).
Simulate hardware telemetry and control interfaces (RAPL/DVFS now, GPU later).
Provide reproducible, comparable experiments across Joulie and WAO.

Design choice: hybrid simulation

The simulator is not a fake scheduler.

Pod placement and pod lifetime stay real in Kubernetes.
Simulator reads cluster state (pods/nodes) and derives synthetic hardware state per node.
Joulie reads telemetry and sends control intents through HTTP endpoints.

This gives one source of truth for workload location: Kubernetes API.

Simulator Algorithms

Mon, 01 Jan 0001 00:00:00 +0000

This page documents the core simulator math and control/workload update loop implemented in simulator/cmd/simulator/main.go.

Per-Node State

Each simulated node tracks:

CapWatts
TargetThrottlePct
ThrottlePct
FreqScale in [f_min/f_max, 1]
CPUUtil in [0,1]
CapSaturated flag

Power Model

At telemetry/control update time, simulator computes power:

P = P_idle + (P_max - P_idle) * util^alpha * freqScale^beta

Where:

P_idle = BaseIdleW
P_max = PMaxW
util = clamp(CPUUtil, 0, 1)
alpha = AlphaUtil (defaults to 1 if invalid)
beta = BetaFreq (defaults to 1 if invalid)

Then applies bounds/cap handling:

CRD and Policy Model

Mon, 01 Jan 0001 00:00:00 +0000

CRD

The implemented APIs are:

Group: joulie.io
Version: v1alpha1
NodePowerProfile (nodepowerprofiles, cluster-scoped) for operator-assigned per-node desired state
TelemetryProfile (telemetryprofiles, cluster-scoped) for telemetry source configuration (node + cluster scope)

CRD files:

config/crd/bases/joulie.io_nodepowerprofiles.yaml
config/crd/bases/joulie.io_telemetryprofiles.yaml

Conceptual model (next step)

Policy should be modeled as a cluster-wide mapping:

input: cluster context at time t
output: node -> power state

Minimal states:

ActivePerformance (mapped to profile performance)
ActiveEco (mapped to profile eco)

Initial implementation should remain rule-based and deterministic. Future implementations can be telemetry-driven or model-driven.

Input Telemetry and Actuation Interfaces

Mon, 01 Jan 0001 00:00:00 +0000

This document defines Joulie’s internal input interfaces for telemetry and control.

Important distinction:

this is not about Prometheus metrics exposed by Joulie,
this is about how Joulie components consume input data and apply controls.

Goals

Run against real hardware in bare metal clusters.
Run in virtual/simulated clusters (kind/kwok) with the same control logic.
Keep APIs generic enough for CPU + GPU + future signals.
Avoid policy/API churn when moving from rule-based to data-driven policies.

Core model (simple)

Joulie uses:

KWOK Benchmark Experiment

Mon, 01 Jan 0001 00:00:00 +0000

This document describes the first benchmark harness implementation under:

experiments/01-kwok-benchmark/

Motivation

The benchmark focuses on repeatable comparisons of scheduler+control behavior across baselines:

baseline A: no Joulie control path (simulator only),
baseline B: static partition style,
baseline C: queue-aware style.

The setup keeps real scheduling semantics while using fake KWOK nodes and simulator-driven telemetry/control effects.

Assumptions

Kubernetes scheduler and API are real.
Fake nodes are tainted and selected by workload pods.
Simulator injects and advances batch work from trace input.
Experiment scripts orchestrate install/run/collect/plot.
A/B/C workload fairness is preserved by generating a canonical per-seed trace and deriving baseline A by stripping only power-profile affinity.

What is measured now

Current harness outputs:

Metrics Reference

Mon, 01 Jan 0001 00:00:00 +0000

Joulie agent exposes Prometheus metrics on /metrics (default :8080).

This document is only for exported observability metrics. For input telemetry and control interfaces (real hardware vs simulated HTTP), see:

Input Telemetry and Actuation Interfaces

Endpoint

Path: /metrics
Address: METRICS_ADDR env var (default :8080)

Backend/Policy

joulie_backend_mode{node,mode} (gauge)
- mode values: none, rapl, dvfs
- Active mode has value 1, others 0
joulie_policy_cap_watts{node,policy} (gauge)
- Current selected policy cap in watts

This metric can also be used to derive policy states in Grafana:

Operator Notes

Mon, 01 Jan 0001 00:00:00 +0000

Target concept

Joulie should evolve into a centralized operator that owns the global optimization loop.

At each control step (for example every minute), the operator:

Reads cluster-wide context.
Decides node-to-power-profile assignments.
Writes desired per-node state.
Monitors outcomes and re-plans.

States start simple:

ActivePerformance (mapped to profile performance): unconstrained / HPC-oriented.
ActiveEco (mapped to profile eco): constrained / energy-saving.

Control responsibility boundary

Operator is the control-plane brain. Agent is an actuator/telemetry component.

Policy Algorithms

Mon, 01 Jan 0001 00:00:00 +0000

This page documents the controller policy algorithms implemented in cmd/operator/main.go.

Classification Input

Policy demand classification is derived from pod scheduling constraints on joulie.io/power-profile:

performance-only: pod can run only on performance/draining-performance.
eco-only: pod can run only on eco.
general (implicit unconstrained): no explicit power-profile constraint, or both profiles allowed.
unknown: unsupported/ambiguous constraint shape.

For safety, unknown is treated as performance-sensitive in downgrade guards.

Shared Reconcile Flow

Each reconcile tick:

Select eligible nodes from NODE_SELECTOR, excluding reserved and unschedulable nodes.
Build a desired plan with the selected policy.
Apply downgrade guard (can convert planned eco to draining-performance/performance).
Write NodePowerProfile and update node label joulie.io/power-profile.

`static_partition`

Goal: deterministic fixed HP/LP split.