knowledge/hl/backtest-and-data.md
vregistry-c914171 · 34.7 KB
# Hyperliquid — historical data and backtesting: what the API provides and where backtests lie
A reference for HL historical data: what the public API returns (candles, account fills, funding), where its limits are, how to cache data for reproducible runs, and why it is difficult to build an honest backtest on HL.
The short conclusion: at best, an HL backtest is a **relative** comparison of alternatives. Small intervals are retained only for the latest ~5000 bars, the public API provides neither second candles nor historical order books, intrabar price order is unknown, and the order-book queue is invisible. Absolute return figures from a backtest are almost always optimistic.
## TL;DR
1. **`candleSnapshot` returns at most ~5000 candles.** If the window is wider, the NEWEST ones are returned (measurement on 2026-07-30: 5m over 90 days → 5029 candles, roughly the latest 18 days). Depth: 5m ≈ 18 days, 15m ≈ 52 days. 1h over 180 days (4320) and 4h (1080) fit in one response. The minimum interval is `1m`; there are no second candles. Pagination from the old edge with a `break` on the first empty chunk returns **no data**.
2. **`userFillsByTime` returns up to 2000 fills per call**, ordered from `startTime`. Continue pagination with `startTime = newest.time` (without `+ 1`: a page can end in the middle of a millisecond), plus deduplication by `tid` (fixed on 2026-09-23, `fills-and-history.md` §3). One call already returns both perp and HIP-3 fills. A separate call for dex `xyz` produced **duplicates**.
3. **What the API does not provide:** second candles, long small-interval history, historical order books (`l2Book` is only the current snapshot), or the full trade tape for a coin (`recentTrades` is the latest 10). All of this is available only through **your own recording** started in advance (§5, §8).
4. **A cache with `endTime = Date.now()` in the key never hits.** Quantize the window (for example, to an hour) or fix a `period-end`. Historical candles do not change, so a disk cache makes runs repeatable offline and reproducible.
5. **Intrabar price order is unknown.** OHLC rules: if SL and TP occur in the same candle, count the SL. Do not count entry and exit in the same candle. A limit order is filled only if the candle **passes through** its level.
6. **No look-ahead.** A bar is stamped with its OPEN time, while its `c` belongs to `t + interval`. The last bar in a response is not closed. At time `t`, use only events whose time is `≤ t`. Build the equity curve only from realized trades and by EXIT time.
7. **The main sources of optimism:**
- entry at the signal price (zero latency);
- exit exactly at the level, without slippage;
- the order-book queue is ignored;
- funding is ignored (see §4 for how to calculate it);
- every fill is treated as maker, even though some limit orders execute as taker and IoC is always taker. HL base rates are maker 1.5 bps (0.015%), taker 4.5 bps (verified with live `userFees` for a new subaccount on 2026-09-14). Read the actual rates for your account from `userFees`, and actual charges from the fills' `fee` field (details in fees.md).
Compare alternatives **relatively** and disclose sensitivity to fees, latency, slippage, and other uncertain execution assumptions.
8. **Hourly bars are insufficient for stops:** an actual jump past the stop level inside the bar is invisible. This is an argument for minute and second data (recording recipe in §8).
---
## 1. Historical-data sources in the public API
All requests are public `POST https://api.hyperliquid.xyz/info` calls; no keys are required.
| `type` | What it provides | Details |
|---|---|---|
| `candleSnapshot` | A coin's OHLCV series | `req: {coin, interval, startTime, endTime}`; minimum interval `1m`; at most ~5000 candles, with the newest returned for a wide window; candle fields `t` (bar open), `T` (close), and string `o/h/l/c/v`; HIP-3 uses `coin: 'xyz:SP500'` without `dex` (§2) |
| `userFillsByTime` | Account fill history over a period | up to 2000 fills per call, chronologically from `startTime`; parameter `aggregateByTime: false`; one call returns perp + HIP-3 (§3) |
| `userFills` | Latest account fills | ceiling of 2000 fills per request; for an active account, 2000 fills may cover only weeks |
| `historicalOrders` | The latest 2000 account order events (placement, cancellation, execution) | no other parameters; covers from tens of minutes for an active account to hundreds of days for a quiet one (`fills-and-history.md` §10) |
| `userFunding` | Actual account funding payments | required for honest PnL on held positions; shapes and formula in §4 |
| `fundingHistory` | A coin's funding-rate history | public; used to simulate hourly funding (§4) |
| `portfolio` | Account aggregate-equity history | for aggregate reconciliation |
| `recentTrades` | A coin's latest 10 trades | newest first; does not provide tape history (`market-data.md` §10) |
| `l2Book` | Current order-book snapshot | has no time parameter; record book history yourself (§8) |
| `meta` (for each DEX) | `szDecimals`, confirmation that a market exists | required to quantize sizes in the simulator |
Weight: `candleSnapshot` and history exports (`userFills*`, `historicalOrders`, `fundingHistory`, `userFunding`, `recentTrades`) have a base weight of 20 plus, according to the documentation, a response-size surcharge (not measured, `rate-limits.md` §2.1). Budget a backfill of hundreds of pages by the number of returned items, not the number of requests.
---
## 2. Candles: `candleSnapshot`
### 2.1 The 5000 limit and history depth
| Interval | Candles over 180 days | Fits in one response? | Effectively available depth |
|---|---|---|---|
| 1m | 259 200 | no | ≤ 5000 bars per call ≈ 3.5 days; retention depth was not measured (`market-data.md` §8, §16) |
| 5m | 51 840 | no | ≈ 18 days (latest ~5000) |
| 15m | 17 280 | no | ≈ 52 days |
| 1h | 4 320 | yes | the full 180 days |
| 4h | 1 080 | yes | the full 180 days |
A single `[start, end]` request on a small interval returns exactly the recent history that is available. That is what the simulation uses, so it answers the question “what would have happened recently.” The full 180-day window fits in one response only at `1h` and above. Long history at small resolution is available only from your own recording (§8).
Interval durations in ms: `1m=60000`, `5m=300000`, `15m=900000`, `1h=3600000`, `4h=14400000`.
### 2.2 Pagination pitfalls
- **Mistake:** loading forward from the old edge of the window with a `break` on the first empty chunk. The old edge at 5m/15m is empty on HL → the loop stops immediately → **0 candles**, and a 5m/15m backtest loses every entry.
- **Reverse traversal** (newest to oldest) fixes this, but costs 2–3 heavy requests per coin, and a run over many coins saturates the request limit for minutes.
- **Conclusion:** make one request per coin for the entire window. It is correct and three times lighter.
- The alternative “5000-sized chunks from old to new without `break`, swallowing empty chunks” also works (snippet in `market-data.md` §8). But old chunks at small intervals are simply empty, so those are wasted requests.
### 2.3 HIP-3 (dex `xyz` and others)
- HIP-3 candles are available. Confirmed: `candleSnapshot` using the prefixed coin name (`xyz:MU`) returns history. Coverage is not guaranteed: mark an empty response as `no_data`; do not substitute zeroes (`market-data.md` §8).
- A backtest that reads `meta`/markets only from the main DEX silently does nothing for HIP-3. Read meta for every required DEX.
- The unprefixed `coin` + `dex: 'xyz'` variant **does not work**: HL returns HTTP 500 (live check on 2026-07-29). If a loader swallows a chunk error into an empty array, xyz coins silently become NO_DATA. Correct: `coin: 'xyz:SP500'` without the `dex` field.
- If candles are unavailable, the simulator must explicitly fall back to `no_data`, not invent a path. Log the request error instead of silently converting it to an empty array.
- Daily series for xyz equities have weekend gaps, so measure the window by time, not by bar count (`market-data.md` §8).
### 2.4 Series normalization
```ts
type Bar = { t: number; c: number }; // t = bar OPEN time, c = close
function normalize(raw: Array<{ t: number; c: string }>): Bar[] {
const byT = new Map<number, number>();
for (const k of raw) {
const c = Number(k.c);
if (Number.isFinite(k.t) && Number.isFinite(c) && c > 0) byT.set(k.t, c); // deduplicate by t
}
return [...byT.entries()].map(([t, c]) => ({ t, c })).sort((a, b) => a.t - b.t);
}
```
- The last bar in a response is not closed: drop it (`T > now` or `t + intervalMs > now`), otherwise the decision uses a price that did not yet exist at that time (`market-data.md` §8).
### 2.5 Loading: cache → single-flight → network → stale
```ts
const CANDLE_TTL_MS = 3_600_000; // historical candles do not change
const HOUR = 3_600_000;
const cache = new Map<string, { ts: number; bars: Bar[] }>();
const inflight = new Map<string, Promise<Bar[]>>();
const quantize = (ms: number) => Math.floor(ms / HOUR) * HOUR; // otherwise a Date.now() key never hits
async function getBars(coin: string, interval: string, startMs: number, endMs: number): Promise<Bar[]> {
const start = quantize(startMs), end = quantize(endMs);
const key = `${coin}|${interval}|${start}|${end}`;
const hit = cache.get(key);
if (hit && Date.now() - hit.ts < CANDLE_TTL_MS) return hit.bars; // 1) fresh cache
const running = inflight.get(key);
if (running) return running; // 2) parallel runs do not double the weight
const p = (async () => {
try {
const res = await fetch('https://api.hyperliquid.xyz/info', {
method: 'POST',
headers: { 'content-type': 'application/json' },
body: JSON.stringify({ type: 'candleSnapshot', req: { coin, interval, startTime: start, endTime: end } }),
});
const bars = normalize(await res.json()); // 3) ONE request for the entire window
cache.set(key, { ts: Date.now(), bars });
return bars;
} catch {
return hit ? hit.bars : []; // 4) stale or empty → no_data; the run does not crash
} finally {
inflight.delete(key);
}
})();
inflight.set(key, p);
return p;
}
```
It is useful to provide `isCached(coin, interval, start, end)`. It lets you estimate a run's request cost in advance and reject it or put it in a queue.
---
## 3. Account fills: `userFillsByTime`
### 3.1 Forward pagination
```ts
async function fetchFillsByTime(user: string, startTime: number, endTime: number, maxFills = 5000) {
const collected: any[] = [];
const seen = new Set<string>();
let cursor = startTime, pages = 0, truncated = false;
while (cursor <= endTime) {
if (collected.length >= maxFills) { truncated = true; break; } // ceiling: mark the result as incomplete
const chunk: any[] = await post({ type: 'userFillsByTime', user, startTime: cursor, endTime, aggregateByTime: false });
pages++;
if (!Array.isArray(chunk)) throw new Error('userFillsByTime: not an array'); // do not turn an error into []
if (chunk.length === 0) break;
let fresh = 0, oldest = Infinity, newest = -Infinity;
for (const f of chunk) {
if (f.time < oldest) oldest = f.time;
if (f.time > newest) newest = f.time;
const k = f.tid != null ? `tid:${f.tid}` : `${f.hash ?? ''}_${f.time}_${f.coin}_${f.px}_${f.sz}_${f.side}`;
if (seen.has(k)) continue;
seen.add(k); collected.push(f); fresh++;
}
if (chunk.length < 2000) break; // the window is exhausted
if (oldest === newest) cursor = newest + 1; // the whole page is one ms: its tail beyond 2000 is unreachable
else if (fresh === 0) break; // the cursor is not moving
else cursor = newest; // reread the last ms; NOT newest + 1
await new Promise(r => setTimeout(r, 250)); // be gentle with the limit
}
collected.sort((a, b) => a.time - b.time || a.tid - b.tid);
return { fills: collected, truncated, pages };
}
```
The cursor is the page's last millisecond, not `newest + 1`: the page is cut by row count and can end in the middle of a millisecond whose tail would be silently lost by `+ 1`. Before 2026-09-23 this used `cursor = newest + 1`; the reason it was changed and the regression coverage are in `fills-and-history.md` §3.
- **perp + HIP-3 in one call.** A separate call with `dex: 'xyz'` returned the same data, duplicating the fills. Make one call and classify by the coin prefix (`xyz:`).
- **The `maxFills` ceiling.** For a very active account, a window may require hundreds of calls. When the ceiling is exceeded, mark the result as incomplete (`truncated`) instead of presenting it as complete.
- A pause between pages (for example, 250 ms) is gentle with the request limit.
- **Historical depth** (according to the documentation, not verified): only the address's ~10,000 newest fills (`rate-limits.md` §2.1). Pagination will not return history older than that window.
### 3.2 What fills do not contain
- Historical fills contain **no snapshot of balance or leverage** (`fills-and-history.md` TL;DR): account size and leverage at fill time cannot be reconstructed from fills alone; snapshot them yourself.
- Funding is not included in fills (§4). TWAP slices arrive in the separate `userTwapSliceFills` feed (`fills-and-history.md` §9).
---
## 4. Funding: rate history and payments
> Response shapes come from the public HL documentation and were not individually checked against live responses. Verify before parsing.
| Request | Body | Response |
|---|---|---|
| `fundingHistory` | `{type:'fundingHistory', coin, startTime, endTime?}` | `[{ coin, fundingRate, premium, time }]`: a coin's rate history (public, for the market) |
| `userFunding` | `{type:'userFunding', user, startTime, endTime?}` | `[{ time, hash, delta: { type: 'funding', coin, usdc, szi, fundingRate, nSamples? } }]`: actual account payments |
| `metaAndAssetCtxs` | `{type:'metaAndAssetCtxs', dex?}` | `assetCtxs[i].funding` — the current rate, plus `openInterest`, `markPx`, `oraclePx` |
| WS `activeAssetCtx` | `{type:'activeAssetCtx', coin}` | real-time `ctx.funding` |
Mechanics and formula:
- Funding on HL accrues once per hour. At a positive rate, longs pay shorts; at a negative rate, the opposite.
- Hourly payment: `payment = −szi × oraclePx × fundingRate`, where `szi` is signed. A negative result is a debit. For simulation, take the hourly rate from `fundingHistory` and the position at accrual time.
- Simulator reconciliation: the sum of `delta.usdc` from `userFunding` for the period on a real account must match your model.
- Paginate like fills: the response is row-limited, so move forward with `startTime = max(time)` (without `+ 1`) and deduplicate by `(time, coin)`. Payments for all account coins in an hour have the same `time`, and a page cut inside the hour would lose the remaining coins with `+ 1`. Measurement at 2026-09-23 11:45Z: `userFunding` for a child address of the public HLP vault (address from `vaultDetails`) with `startTime` = now − 6 h returned exactly 500 rows in ascending `time`, with only 3 distinct timestamps and 146–177 coins per timestamp—the page ended inside the third hour. 500 appears to be the response ceiling, but this was not separately verified. This is how `hl-kit` paginates (`fetchUserFunding`). Before 2026-09-23 this used `max(time) + 1`.
- Price PnL from fills (`closedPnl`) does not include funding. Total PnL = `Σ closedPnl − Σ fee + Σ funding.usdc`.
- Funding is material when holding a highly leveraged position for a long time, and it works against backtest optimism.
---
## 5. What the public API does not provide
| Needed for a backtest | Available in the API | Conclusion |
|---|---|---|
| Second candles | minimum `candleSnapshot` interval is `1m` | record them yourself from WS `trades` (§8) |
| Long 1m/5m/15m history | only the latest ~5000 bars for an interval (§2.1) | your own recording started in advance; otherwise use 1h or above |
| Order-book history (queue, depth, spread) | `l2Book` is only the current snapshot, with no time parameter | record `l2Book`/`bbo` yourself (§8) |
| Full coin trade tape | `recentTrades` is the latest 10 trades (`market-data.md` §10) | record WS `trades` yourself |
| Mark-price history | this knowledge base does not describe a mark-history request | record `activeAssetCtx.markPx` yourself for simulating native TP/SL (which trigger on mark) (§8) |
| Deep fill history for an active address | according to the documentation, ~10,000 latest fills (not verified, §3.1) | your own recording (WS `userFills`) |
WS does not provide historical replay: anything unavailable from REST must be recorded in advance.
---
## 6. Caching and reproducibility
### 6.1 Disk cache of raw responses (offline backtest)
```
<cache-root>/
fills/ <account_lowercase>_<startTime>_<endTime>.json // fills for your account, used for reconciliation (§7.10), sorted by time
candles/ <safeCoin>_<interval>_<startTime>_<endTime>.json // safeCoin = coin.replace(/[^a-zA-Z0-9:_-]/g, '_')
```
- Unformatted JSON. Candles are deduplicated by `t` and sorted.
- Two useful modes are “ignore cache and fetch again” and “cache only, no API.” In the second mode, a missing candle cache means there are no candles and the market becomes `NO_DATA`.
- **Pitfall:** if `endTime` defaults to `Date.now()`, the filename is different on every run and the cache never hits. Use a fixed end of the period (`--period-end <ms>`) or quantize the window. `periodStart = periodEnd − days·86_400_000`.
### 6.2 In-memory candle cache
| What | TTL | Other behavior | Rationale |
|---|---|---|---|
| Candles | 1 h, with the window quantized to the hour | per-key single-flight, stale on error | historical bars do not change |
### 6.3 Run artifacts
- Write the actual data range (`first.t..last.t`) to the run report, not the requested window: a “90-day” 5m run actually covers ~18 days.
- Include markets and trades rejected for insufficient data in the report with a reason (for example, `NO_DATA`) instead of silently dropping them; otherwise samples from different alternatives are not comparable.
---
## 7. Why an HL backtest is difficult and where it lies
### 7.1 Intrabar price order
A candle provides `o/h/l/c`, but not the order in which price visited `h` and `l`. Any intrabar ordering rule is a guess, so choose it against the strategy:
- SL and TP in the same candle → count the SL (worst case).
- Recording the exit exactly at the level is already optimistic (no slippage).
- The `c.t >= entryTime` condition in the example below skips the bar in which entry occurred: touches during the first minutes after entry are lost (a limitation of the example).
```ts
type Candle = { t: number; T: number; o: string; h: string; l: string; c: string };
function findExit(side: 'LONG' | 'SHORT', entry: number, entryTime: number,
candles: Candle[], slPct: number, tpPct: number) {
const sl = side === 'LONG' ? entry * (1 - slPct / 100) : entry * (1 + slPct / 100);
const tp = side === 'LONG' ? entry * (1 + tpPct / 100) : entry * (1 - tpPct / 100);
for (const c of candles) {
if (c.t < entryTime) continue; // skip the entry bar: touches in the first minutes after entry are lost
const hi = Number(c.h), lo = Number(c.l);
const hitSl = side === 'LONG' ? lo <= sl : hi >= sl;
const hitTp = side === 'LONG' ? hi >= tp : lo <= tp;
if (hitSl) return { reason: 'SL', price: sl, time: c.t }; // both in one candle → SL (worst case)
if (hitTp) return { reason: 'TP', price: tp, time: c.t }; // exit exactly at the level, without slippage
}
return null; // → NO_DATA (no candles) or PERIOD_END (window ended)
}
```
If there are no candles at all, the trade becomes `NO_DATA` (a separate statistics category, not PnL = 0). If the window ends, `exitTime = min(periodEnd, last.T)`.
### 7.2 Limit orders: a touch is not a fill
- A limit buy is filled only if price **passes through** the level (`low` below the price); a sell only if `high` is above the price. Touching the level does not guarantee a fill: there is a queue ahead of the order, and the API does not show it (§5).
- **If entry occurs in a candle, do not count an exit in that same candle.** Intrabar order is unknown; doing otherwise creates a free bounce.
- **Quantization.** Round order prices the same way as the exchange: at most 5 significant figures and at most `6 − szDecimals` decimal places, so the effective tick depends on the price magnitude (`1000.2` at a 0.1 step, `950.02` at a 0.01 step). Quantize size using `szDecimals` from `meta` (`market-data.md` §3).
- **Candle interval.** Rules involving intrabar limit orders cannot be evaluated on 1h bars: the outcome is determined by a guess about order within the hour, usually in the strategy's favor. Evaluate them on 1m; hourly candles are suitable only as a coarse filter (`market-data.md` §8).
### 7.3 Maker or taker, fee tier
- A maker model on both sides (0.015% at the base tier) is optimistic if some limit orders actually execute as taker. A limit order that crosses the book executes as taker (`crossed = true`); IoC is always taker (`fills-and-history.md` §4). Calculate your maker share from your own fills using `crossed === false`.
- Taker model: `fee = notional × takerFeeBps / 10 000 × 2` (entry and exit). Formulas and the maker model are in `fees.md` §6.2.
- The rate is a market parameter, not a global constant: the tier depends on 14-day volume, the rate on a HIP-3 dex may differ from the main dex, and spot has its own tariffs (`fees.md` §1.1). For HIP-3, use actual `fee / (px × sz)` from your own fills.
- Fees are indispensable when comparing alternatives with different trade counts: without fees, the alternative that trades less often looks better than it is.
### 7.4 Funding
Neither candles nor price PnL from fills include funding. It is a separate line item for held positions and often works against the strategy. See §4 for calculation and reconciliation.
### 7.5 Latency and slippage
- Entry at the price at decision time (zero latency) and exit exactly at the level overstate the result. Actual execution is later and worse.
- Calibrate from your own live fills: latency from decision to order (`lagMs`) and the price cost of that latency in bps, `(fillAvgPx − refPx) / refPx × 10 000` with the side's sign, where `refPx` is the price at decision time. Include only filled orders.
- Walk-the-book (how many levels an order consumes) requires an order book, which is unavailable historically (§5); without a recorded book, the model overestimates execution: on a thin market, an order consumes several levels (`market-data.md` §7; for TWAP, see `twap.md`).
### 7.6 Look-ahead
- A bar is marked with its open time `t`, while `c` is the price at `t + interval`. Filtering by open time with `b.t <= closeTime` takes a bar whose `c` is already a price after `closeTime`; the effect is material on 1h/4h. Compare using the bar close time (`T` or `t + interval`).
- The last bar in a response is not closed (§2.4).
- Build the equity curve only from realized trades and sort by **exit** time. Assigning a result to entry time makes the curve show profit before it was earned.
```ts
const closed = results
.filter(r => r.entered && !r.unrealized && r.exitTime != null)
.sort((a, b) => a.exitTime - b.exitTime); // by EXIT time
let cum = 0, peak = 0, maxDd = 0;
for (const c of closed) {
cum += c.pnl;
if (cum > peak) peak = cum;
if (cum - peak < maxDd) maxDd = cum - peak; // $, ≤ 0; initial peak = 0
}
```
### 7.7 Data resolution
- On **hourly** bars, a jump past the stop level inside the bar is unknown. Stops require minute or second data.
- Ambiguity when “SL and TP are in the same candle” decreases with a smaller interval. This is a second argument for small bars and your own recording.
- 1h is unsuitable for intrabar limit-order rules (§7.2). And a single 1m call returns no more than ~3.5 days (≤ 5000 bars); the amount of 1m history retained by HL was not measured (§2.1).
- In addition to the rolling window, run stops and liquidations over a separate extreme window (for example, the October 2025 crash): calm windows do not show gaps and flash crashes.
- A rule that keeps its sign only in-sample but reverses on a held-out window (out-of-sample) is overfit.
### 7.8 The market changes
- **Equities and indices on `xyz`:** outside the US session the oracle freezes while the perp remains tradable, but execution occurs against a thin book at a stale price (`market-data.md` §11). Candles for those hours do not reveal the price at which an order would have filled, so their backtest result is unreliable.
- **The HIP-3 naming format changed:** `meta` with `dex:'xyz'` previously returned bare names and now returns prefixed ones (`market-data.md` TL;DR). Normalize cache keys and series matching idempotently.
### 7.9 Summary of assumptions and cost model
| Assumption | Typical simplification | Direction of error |
|---|---|---|
| Entry price | signal price, zero latency | optimism (actual execution is later and worse) |
| Level touch | based on `high/low` | optimism for limit orders (queue), unknown intrabar order |
| Exit | exactly at the level | optimism (no slippage) |
| Intrabar conflict | SL before TP; no exit in the entry candle | conservative |
| Fee | one rate for every fill | optimism if some fills are actually taker (maker 1.5 bps vs taker 4.5 bps) |
| Funding | ignored | depends on the side; often adverse for held positions |
| Order-book queue | not modeled | optimism |
| Gaps and flash crashes | not modeled | optimism for stops |
**When no order book is available, execution cost is an assumption, not measured exchange behavior.** Make the assumption explicit, vary it in a sensitivity analysis, and report when the comparison of alternatives changes. Do not present a chosen cost as a calibrated rate or the resulting PnL as a prediction.
**Absolute figures** require latency and walk-the-book from a recorded order book, calibrated against live fills (§7.5).
### 7.10 Reconciliation with reality
Reconcile simulated PnL against actual exchange data from your own real trades over the same period: `Σ closedPnl − Σ fee + Σ funding.usdc` (§4). Compare total PnL, trade count, and the sign of PnL for every trade. Until the simulator reproduces real trades, do not trust its conclusions about hypothetical ones.
---
## 8. Recording your own data
Below are the reason to record your own data and a recording recipe. The recipe is not verified.
- **Why.** Hourly bars hide overshoots past stop levels and make intrabar SL/TP order ambiguous. `candleSnapshot` provides no interval smaller than `1m`, small intervals are retained only for the latest ~5000 candles (5m ≈ 18 days), and the API provides no order-book history at all (§5). Long history at small resolution is available only through **your own recording**.
### Recipe for recording second OHLCV (not verified)
```ts
import WebSocket from 'ws';
import { appendFileSync } from 'node:fs';
type Bar = { t: number; o: number; h: number; l: number; c: number; v: number; n: number; buyV: number };
const COINS = ['BTC', 'xyz:SP500'];
const bars = new Map<string, Bar>(); // coin -> current one-second bar
const seen = new Set<number>(); // deduplicate tid (trim above 200k)
function flush(coin: string, b: Bar) {
const day = new Date(b.t).toISOString().slice(0, 10);
appendFileSync(`data/${coin.replace(':', '_')}-1s-${day}.ndjson`, JSON.stringify({ coin, ...b }) + '\n');
}
const ws = new WebSocket('wss://api.hyperliquid.xyz/ws', { perMessageDeflate: false });
ws.on('open', () => COINS.forEach((coin) => ws.send(JSON.stringify({ method: 'subscribe', subscription: { type: 'trades', coin } }))));
ws.on('message', (raw) => {
const msg = JSON.parse(raw.toString());
if (msg.channel !== 'trades') return;
for (const tr of msg.data) {
if (seen.has(tr.tid)) continue; seen.add(tr.tid);
const px = Number(tr.px), sz = Number(tr.sz), sec = Math.floor(tr.time / 1000) * 1000;
let b = bars.get(tr.coin);
if (b && b.t !== sec) { flush(tr.coin, b); b = undefined; }
if (!b) { b = { t: sec, o: px, h: px, l: px, c: px, v: 0, n: 0, buyV: 0 }; bars.set(tr.coin, b); }
b.h = Math.max(b.h, px); b.l = Math.min(b.l, px); b.c = px; b.v += sz; b.n++;
if (tr.side === 'B') b.buyV += sz;
}
});
// + ping every 30 s, reconnect with backoff (see websocket.md), force-flush the bar on a 1–2 s timer
```
Storage and usage rules:
- One append-only file per coin and UTC day. The bar key is `t` (start of the second). When assembling the series: deduplicate by `t` and sort, as in §2.4.
- `trades` frames can repeat after reconnect: deduplication by `tid` is mandatory. `side` is the aggressor side (`websocket.md`).
- Do not invent seconds without trades. Carry forward the last `c` for the price path with `v = 0` and an explicit flag.
- Mark reconnect gaps separately (`gapFrom`/`gapTo`). A backtest over a segment with a gap returns `no_data`, not interpolation.
- Trades are trade prices, not mark. Native HL TP/SL triggers on mark price, so record `activeAssetCtx.markPx` (or mid from `bbo`) in parallel at the same step to simulate it.
- For a limit-order execution model, record `bbo` or `l2Book` every 1–3 s. A price touch does not guarantee a fill: treat a resting bid as filled only on a trade strictly below its price, and a resting ask only on a trade strictly above, not when mid merely touches.
- Resolution: use a second-by-second path for stops, and bars at the interval used by live signal logic.
- WS does not provide historical replay, and REST info does not retain the full coin trade history (`recentTrades` is the latest 10). Start recording in advance.
---
## 9. Pitfalls
| What breaks | Why | Correct approach |
|---|---|---|
| A 5m/15m backtest “loses” every entry | candle pagination starts from the old edge with a `break` on an empty chunk; HL's old edge is empty for small intervals | one `candleSnapshot` for the entire window; remember that only the latest ~5000 candles are available |
| A 5m/90d run saturates the request limit for minutes | reverse pagination = 2–3 heavy requests per coin | one request per coin; single-flight; estimate cost from the cache in advance |
| A “90-day” 5m backtest actually covers ~18 days | the ~5000-candle limit returns the newest candles | calculate the actual `first.t..last.t` range and print it in the report; use 1h or your own recording for long history |
| The disk/in-memory candle cache never hits | `endTime = Date.now()` in the key or filename | quantize the window to the hour or fix `--period-end` |
| Duplicated fills | a separate `userFillsByTime` call with `dex:'xyz'` returns the same records as the main call | one call; classify by the `xyz:` prefix; deduplicate by `tid` |
| Fills are lost at page boundaries | cursor `max(time) + 1`, while the page ended inside a millisecond | cursor `max(time)` + deduplication by `tid` (§3.1) |
| A HIP-3 backtest is silently empty | the loader read markets or meta only from the main DEX | meta for every DEX; explicit “NO CANDLES → fallback” log |
| xyz coins silently become NO_DATA | candles were requested as unprefixed `coin` + `dex: 'xyz'`: HL returns HTTP 500, and the loader swallowed the error into `[]` | `coin: 'xyz:SP500'` without `dex`; log chunk errors |
| PnL for held positions is distorted | funding is omitted | hourly funding from `fundingHistory`, reconciled with `userFunding` (§4) |
| Hundreds of API calls for one very active account | 2000 fills per page | a `maxFills` ceiling and an “incomplete result” marker; disk cache to avoid paginating again |
| Statistics mix in an invented breakeven | a trade without data is valued at 0 | `no_data`, reported separately |
| Look-ahead on the last bar | filtering by the bar's **open** time, while its `c` is the price after that moment | compare by the bar close time (`t + interval` or `T`); the effect is material on 1h/4h |
| A decision uses an unclosed bar | the response's last bar is still forming | discard a bar with `T > now` |
| The equity curve “runs ahead” of the money | PnL is assigned to entry time and unrealized trades are included | realized trades only, sorted by `exitTime` |
| A free bounce in the simulator | entry and exit were counted in the same candle | do not count an exit in the entry candle |
| Limit orders “fill” too often | touching the level is counted as a fill; the queue is invisible | require price to pass beyond the level; on a recorded tape, require a trade strictly beyond the price |
| Absolute backtest ROI is far above reality | entry at signal price (zero latency), exit at the level without slippage, every fill is maker | draw only relative conclusions; calibrate latency and cost from live fills |
| Account size or leverage at fill time is unknown | historical fills contain no balance or leverage snapshot | snapshot equity yourself; otherwise mark these metrics as approximate |
---
## 10. Open questions / not verified
- **Historical depth of `userFillsByTime`/`userFills`.** The per-request ceiling is known to be 2000; for an active account, the latest 2000 fills may cover only weeks. According to the documentation, only the address's latest ~10,000 fills are available—not verified (`rate-limits.md` §2.1).
- **The `userFunding` response ceiling.** A measurement on 2026-09-23 returned exactly 500 rows; it was not separately verified that this is the ceiling (§4). The `fundingHistory`/`userFunding` shapes come from the documentation, and the formula was not reconciled against live `userFunding` using this model.
- **Depth of `1m` history** was not measured; only the ceiling of ≤ 5000 bars per call is known (`market-data.md` §16).
- **Response-size weight surcharge** for `candleSnapshot` and history exports comes from the documentation and was not measured (`rate-limits.md` §2.1).
- **Second candles.** The WS `trades` recording recipe (§8) is not verified. The `trades` frame shape was not checked against a live socket; whether a snapshot of recent trades arrives after reconnect is not verified (`websocket.md`). The effect of second resolution on stop accuracy was not measured.
- **Execution costs:** candles alone do not provide actual latency, slippage, or queue position; these require separate observations, and any substitute is a model assumption.
- **The order-book queue** is not modeled. There is no estimate of the share of level touches without a fill.
- **Look-ahead on the last bar:** the effect on aggregate statistics was not measured.
---
Check dates are given in the text. The HL API changes—recheck limits and response shapes.
---
<!-- license-footer -->
_© markpaper authors. Licensed under [CC BY 4.0](LICENSE.md): when publishing or adapting this material, credit “markpaper — Hyperliquid knowledge base” and link to the original and the license._