Timing Models

Every HwModule carries two models. The functional model says what it computes; the timing model says how long that takes — when, in simulated time, the work finishes.

Most of the time you don’t write one. Most Waveflow operations already carry a built-in timing model: when you yield from self.mem_if.read_array(...), the simulation advances the clock by an estimate of how long that transfer takes; a stream get / write does the same. Those are the costs the framework already knows.

What the framework cannot know is how long your compute takes — the body of a custom hook. So in practice a user builds a timing model only for the compute of a custom hook: a small model that expresses the elapsed cycles as a function of the input size — the number of samples, the vector length — the parameter the work scales with.

Declare it, then calibrate it

The section is in two halves, and they are the same model at two stages of its life.

Declaring a model fixes its form — a pipelined loop costs latency + ii·(m − 1) cycles, and that shape is a property of the hardware you wrote. Calibrating it recovers the numbers in that form from measurement, so the fast LT simulation reproduces what the RTL actually did.

A declared-but-uncalibrated model still simulates: it runs on its seed and says UNCALIBRATED, which is the honest state for a design that has not been synthesized yet.

Declaring — the form

  • LT vs CT models — loosely-timed vs cycle-timed simulation, and why Waveflow is LT (the bet that justifies modeling a transaction’s timing rather than every cycle).
  • Adding a timing model to a component — where a timing model plugs in: attach it, and charge the delay it predicts with self.timeout.
  • Timing models for loops — the typical compute model: a pipelined loop costs latency + ii·(m − 1) cycles — linear in two parameters, expressed as a LinCalibModel.
  • Block processing — inserting the model in a block process: the compute runs after the whole block has loaded (the prediction goes after the load, before the store).
  • Streaming processing — inserting the model in a streaming process: the compute overlaps the load and/or store, element by element.

Double-buffering (ping-pong overlap) is no longer a separate timing model: it is built by composing load / compute / store as concurrent sub-components over a stream of blocks, and the compute sub-component is timed exactly like a block process.

Calibrating — the numbers

Two methods, and which one you want depends on how much of the cost the LT sim already charges:

  • Fitting a timing model — the direct method: fit the parameters straight from a sweep of (size, cycles). A loop model’s latency / ii are the two coefficients of a line. Reach for this when the model owns the whole cost.
  • Component residuals — the residual method: when the interfaces already time the transfers, fit only the gap between RTL and pysim — the control overhead pysim misses. StreamTimingModel, fit per (component, platform).
  • The bus-transfer model — BusCalib: how long the interconnect takes to move n words in k bursts is a property of the platform, so it is fit once and reused by every accelerator. Charging it in pysim is what shrinks the component residual to the kernel’s own cost.
  • The mem-stream residual — the reusable MemRStream / MemWStream control residual and the fixture that fits it. Ships calibrated, so a design on a known platform inherits it with no re-calibration.
  • Polling Overhead — the loosely-timed model for MMIFMaster.poll_until: the bandwidth a poll loop steals from the bus it shares, and the discovery latency between the watched event and the next poll. Reached through an interface, but it is a timing model.

The two-level split: bus vs component

For a component that moves data over m_axi, the residual method leans on a split. The run’s cost is:

    RTL cycles  =  bus transfer  +  component control
                    └─ PLATFORM ─┘   └── COMPONENT ──┘

The bus transfer — how long the interconnect takes to move n words in k bursts — is a property of the platform (memory system + AXI adapter), so it is fit once per platform and reused by every accelerator (BusCalib). With that charged in pysim, the component control residual shrinks to the kernel’s own overhead, fit per (component, platform) (StreamTimingModel).

That split is what makes the second level cheap. A new accelerator on a calibrated platform inherits the bus term for nothing and only has to fit what is genuinely its own.

Everything is stored in cycles

Fitted timing numbers are cycles, not seconds, so the artifact is clock-independent: re-deploying at a different simulation frequency needs no refit. The clock that does change them is the synthesis clock, which is why a platform is keyed by part and period.

The machinery underneath both methods — the model base, the corpus format, the confidence levels — is axis-agnostic and lives in Model calibration. Resource models use the same base; only the source of a number differs.

See also

  • Simulation timing model — the Clock, self.timeout, and where transfer vs. compute latency is charged. This section assumes that page.
  • Model calibration — the shared CalibModel, corpus and confidence this section’s calibration half is built on.
  • Timing Analysis Tools — the measurement side: extracting cycle counts and bus spans from a VCD / cosim run, which is where the datapoints come from.
  • Custom hooks — the hand-written compute whose timing you model here.

Table of contents

  • LT vs CT models - Loosely-timed (LT) vs cycle-timed (CT) simulation. Waveflow models timing loosely — one timed event per transaction, not per clock edge — which is what makes a whole-system Python simulation fast, and is the same choice established computer-architecture simulators make.
  • Adding a timing model to a component - How a component sources its compute delay from a timing-model object: attach a model and, in the run loop, charge self.timeout(tm.predict(...) * clk.period). tm.predict returns the cycles the compute will take as a function of the firing size (e.g. n); internally it evaluates a formula in a few parameters (e.g. latency + proc_ii*(ceil(n/U)-1)), which the following pages make concrete and the fitting section recovers by comparison to RTL. The model adds only this component's compute plus unaccounted overhead — read/write times are charged by the transfer yields, and stalls grow the firing on their own.
  • Timing models for loops - The typical custom-hook compute is a loop, and a pipelined loop of m iterations costs latency + ii·(m − 1) cycles — latency is the pipeline depth (the first result), ii the initiation interval (each result after). That is linear in two parameters (latency, ii), so it is a LinCalibModel with a two-term basis map [1, m − 1] and coeff_names ['latency', 'ii'], fit_intercept=False. Unrolling by U reshapes the trip count to m = ceil(n / U). The parameters are set by hand now and fit from RTL later.
  • Block processing - Inserting a timing model in a block process: the compute cannot start until the whole block has loaded, so the predicted compute delay goes AFTER the load and BEFORE the store. Whatever model the custom hook uses (typically the loop model), it is charged there. Shown on a stream-of-blocks Compute sub-component (acquire_read the block, compute, timeout, write). The load and store are already timed by the block/stream yields, so the model adds only the compute, and load/store stalls grow the firing on their own.
  • Streaming processing - The streaming timing model: compute overlaps the load, so the first output appears `latency` cycles after the first input and later outputs are gated by whichever is slower — input arrival or compute rate. The per-element max() collapses to a first/last form; only first and last arrival times are known.
  • Fitting a timing model - The direct method: recover a loop timing model's parameters (latency, ii) from a sweep. Run the kernel at several input sizes, record (n, cycles) into a CalibDataFrame, and fit the LinCalibModel — the coefficients ARE latency and ii. Two points suffice in principle; use more and check R² / held-out error. unroll_factor reshapes the basis (ceil(n/U)), fixed from synthesis or swept. The (n, cycles) ground truth comes from a Vitis HLS cosim sweep.
  • Component residuals - A component's control residual is the delay pysim is MISSING once the bus term is charged: residual = rtl_span - pysim_span + current_dly, fit per (component, platform). TimingModel collects RTL firings (from a trace) and pysim firings (from a run) into independent trees, joins them on the feature point (nwords, num_trans), and fits a CalibModel; predict() returns the delay a FreeRunMod injects via timed_delay. Stored shared (platform_dir -> the committed library) or custom (calib_dir -> a project dir). CollectTimingStep / FitTimingStep automate it.
  • The bus-transfer model - BusCalib fits the platform's m_axi transfer law — span(num_trans, nwords) per direction — and stores it in mm_bus.json beside the platform manifest. The datapoints are measured component-independently off the memory port (measure_bus_span groups beats into transfers by idle gaps); a sweep accumulates a per-run corpus (add_run -> points/) then fits. bus_timing() hands back a configured BusTiming the memory slave charges during pysim, or an unconfigured one (word_bw fallback) on an uncalibrated platform. CalibBusStep automates it as a DAG step.
  • Polling Overhead - The loosely-timed polling-overhead model — MMIFMaster.poll_until with a restricted PollCond, the per-bus bandwidth-steal (ov) derating and the deterministic discovery-latency delay, modeled in O(transactions) rather than by stepping every poll cycle.
  • The mem-stream residual - The reusable m_axi mem-streams (MemRStream / MemWStream) are timed by the bus law plus a per-firing CONTROL residual — a (component, platform) property fit once and reused by every accelerator. The residual is fit by a per-component FIXTURE under waveflow/calib/fixtures/ (mem_w_stream.py, mem_r_stream.py): a ComponentFixture declares the component id, the regression basis, a size sweep, a run_pysim that drives the component STANDALONE (StreamDriver -> component -> sink, bus law on the memory so the residual is control-only), and rtl_firings (measured spans or None). fixture.calibrate(platform) sweeps, collects RTL + pysim, and fits into components//params.json. An accelerator loads it by pointing its mem stages at the platform. memcpy is writer-bound so only the writer fixture existed; the reader-bound interleaver is what forced mem_r_stream.py.