knowledge/hl/fills-and-history.md
vregistry-c914171 · 42.2 KB
# Hyperliquid — Fill, Order, and Ledger History
Reference for reading account history on HL: `userFills` / `userFillsByTime`, fill fields, partial fills grouping, position reconstruction and PnL, TWAP slices, `historicalOrders`, `openOrders` / `frontendOpenOrders`, ledger, public dump builder fills, leaderboard and its blind spots.
## TL;DR
1. **`userFills` returns no more than the last 2000 fills**, weight 20 (heavy). For deeper history, use `userFillsByTime`. It also returns a maximum of 2000 per call, with the oldest from `startTime` (inclusive), so you need to scroll forward: **next `startTime` = `max(time)` of the page, without `+ 1`, and dedup by `tid`**; end when the page is shorter than 2000. The cursor `max(time) + 1` silently drops the tail millisecond where the page ended, while fills split milliseconds constantly (§3, corrected and verified on 2026-09-23; previous entry was `max(time) + 1`).
2. **`userFills` ignores the `dex` parameter** (verified on 2026-06-11): the response without `dex` already contains HIP-3 fills (`xyz:*`). A second request with `dex:'xyz'` doubles the weight, and when combined with the first one, each fill is counted twice. **Dedup by `tid` is mandatory** for any source combination.
3. **`frontendOpenOrders`, on the contrary, considers `dex`**: orders from xyz only come with `{ user, dex: 'xyz' }` (verified on 2026-07-09). Collect snapshots of open orders across all dex and mark them as complete or incomplete.
4. **One order comes as N partial fills.** Group by `oid`. If `oid` is absent, group by `(coin, time, dir)`. Otherwise, the number of trades swells, and trade statistics break down.
5. **`startPosition`** (position in coin before fill) — the source of truth for position trajectory. Do not reconstruct the position based on the principle of "accumulating openings, closings eat them".
6. **PnL of a closed position = Σ `closedPnl` across all reducing fills in the chain** (partial closes and final), filtered by `time >= openedAt`. One fill does not give PnL for the position.
7. **TWAP slices in `userFills` / `userFillsByTime` do not appear at all**, regardless of `aggregateByTime`. They are stored in a separate feed `userTwapSliceFills`.
8. **Fill does not contain leverage and balance.** Leverage is only available for the open position in `clearinghouseState`. For historical analysis, take equity snapshots and leverage.
9. **`builderFee` in fill — commission for any builder, not just yours.** Filter your fills by your own `oid`.
10. **Leaderboard** (`stats-data.hyperliquid.xyz/Mainnet/leaderboard`) is not a complete list of accounts: it does not include part of any size accounts (observation on 2026-09).
---
## 1. Endpoint Map
All requests, except leaderboard and dump, are sent as `POST https://api.hyperliquid.xyz/info` with JSON body.
| Request | Body | What Returns | Output Limit | Weight |
|---|---|---|---|---|
| `userFills` | `{type:'userFills', user, aggregateByTime?}` | latest fills across all dexes | ≤2000 | 20 |
| `userFillsByTime` | `{type:'userFillsByTime', user, startTime, endTime?, aggregateByTime?}` | fills in the window, chronologically from `startTime` | ≤2000 per call, oldest first if overflow | 20 |
| `userTwapSliceFills` | `{type:'userTwapSliceFills', user}` | TWAP order slices not in `userFills` | Not measured | Not measured |
| `historicalOrders` | `{type:'historicalOrders', user}` | events on orders (placement, cancellation, execution) | ≤2000 latest events | Not measured |
| `openOrders` | `{type:'openOrders', user}` | `coin, oid, side, limitPx, sz` | — | 20 |
| `frontendOpenOrders` | `{type:'frontendOpenOrders', user, dex?}` | same as above, plus `tif`, `reduceOnly`, `origSz`, `orderType`, `isTrigger`, `triggerPx`, `isPositionTpsl` | — | 20 |
| `userNonFundingLedgerUpdates` | `{type:'userNonFundingLedgerUpdates', user, startTime}` | deposits, withdrawals and transfers | Not measured | 20 |
| `clearinghouseState` | `{type:'clearinghouseState', user}` | current positions, `leverage`, equity | — | — |
| `vaultDetails` | `{type:'vaultDetails', vaultAddress}` | `null` if address is not a vault | — | — |
| `recentTrades` | `{type:'recentTrades', coin}` | recent trades feed for the coin | — | — |
| Leaderboard | `GET https://stats-data.hyperliquid.xyz/Mainnet/leaderboard` | `{leaderboardRows:[...]}` | ~39k rows (2026-06), ~44–45k (2026-09) | not verified |
| Dump builder fills | `GET https://stats-data.hyperliquid.xyz/Mainnet/builder_fills/<builder>/<YYYYMMDD>.csv.lz4` | all fills with builder code for UTC day | — | not verified |
The time is everywhere passed in unix-milliseconds. Numbers in responses (`px`, `sz`, `closedPnl`, `fee`, `usdc`...) come as strings, so they need to be parsed through `Number()`.
---
## 2. `userFills` — recent fills
- The request `{ type: 'userFills', user }` returns **no more than the last 2000 fills**. Weight is 20.
- Parameter `dex` is **ignored**. Comparing `{type:'userFills', user}` and `{type:'userFills', user, dex:'xyz'}` gave identical results: 2000 fills each, matching sets of `tid`, and coins `xyz:*` in both (verified on 2026-06-11). A separate "xyz-request" doubles the weight of reading fills, while concatenation sums them.
- The order of the response is not guaranteed, so sort by `time` before analysis.
- `aggregateByTime: true` (`{ type:'userFills', user, aggregateByTime: true }`) merges partial executions of one order within a time slice into one fill. This is convenient for reconstructing round-trips.
- How much time 2000 fills cover depends on the account's activity: high-frequency accounts get minutes, while rarely trading accounts may get months.
- **Suitable for:** recent trades, PnL of just closed positions.
- **Not suitable for:** historical data over a period. For that, use `userFillsByTime`.
---
## 3. `userFillsByTime` — History by Period and Pagination
**Request Body:**
```json
{ "type": "userFillsByTime", "user": "0xYOUR_ADDRESS", "startTime": 1757000000000, "endTime": 1757086399999, "aggregateByTime": false }
```
- `aggregateByTime: false` returns raw partial fills, while `true` merges them by time.
- `endTime` can be omitted: the request `{type, user, startTime}` works.
- **One UTC day:** `start = Date.UTC(y, m, d)`, `end = start + 86_400_000 − 1` (`endTime` inclusive).
- **Maximum of 2000 per call, in chronological order from `startTime`.** When overflow occurs, the oldest fills in the window are returned, not the newest. An active account's weekly window can easily exceed 2000, so pagination should be done forward.
- **Without `dex`, all fills are returned:** perp and xyz. A second call with `dex:'xyz'` returned the same data and created duplicates. Make one request and distinguish xyz by prefix `xyz:` in `coin`.
- If an error on a request with `dex:'xyz'` silently turns into an empty array, while an error on the main request is re-thrown, data can be lost without any message. Do not pass extra `dex`, do not swallow errors.
### Snippet: Time Pagination
```ts
type HlFill = {
coin: string; px: string; sz: string; side: 'B' | 'A'; time: number;
startPosition: string; dir: string; closedPnl: string; hash: string;
oid: number; crossed: boolean; fee: string; feeToken: string; tid: number;
builderFee?: string;
};
async function post<T>(body: unknown): Promise<T> {
const r = await fetch('https://api.hyperliquid.xyz/info', {
method: 'POST', headers: { 'Content-Type': 'application/json' },
body: JSON.stringify(body), signal: AbortSignal.timeout(20_000),
});
if (!r.ok) { const e: any = new Error(`Hyperliquid API error: ${r.status}`); e.status = r.status; throw e; }
return r.json() as Promise<T>;
}
const sleep = (ms: number) => new Promise(r => setTimeout(r, ms));
const fillKey = (f: HlFill) =>
f.tid != null ? `tid:${f.tid}` : `${f.hash ?? ''}_${f.time}_${f.coin}_${f.px}_${f.sz}_${f.side}`;
async function fetchFillsByTime(user: string, startTime: number, endTime: number) {
let cursor = startTime;
const seen = new Set<string>();
const collected: HlFill[] = [];
const denseMillis: number[] = []; // ms, completely filling a page: their fills above 2000 are unreachable
while (cursor <= endTime) {
const chunk = await post<HlFill[]>({
type: 'userFillsByTime', user, startTime: cursor, endTime, aggregateByTime: false,
});
if (!Array.isArray(chunk)) throw new Error('userFillsByTime: not an array'); // error not to be casted into []
if (chunk.length === 0) break;
let fresh = 0, oldest = Infinity, newest = -Infinity;
for (const f of chunk) {
if (f.time < oldest) oldest = f.time;
if (f.time > newest) newest = f.time;
const key = fillKey(f);
if (seen.has(key)) continue;
seen.add(key); collected.push(f); fresh++;
}
if (chunk.length < 2000) break; // window exhausted
if (oldest === newest) { // entire page — one ms: go only ms + 1 further
denseMillis.push(newest);
cursor = newest + 1;
} else {
if (fresh === 0) break; // cursor not moving — no infinite loop
cursor = newest; // re-read the last ms, NOT newest + 1
}
await sleep(250); // do not hammer heavy-endpoint
}
return { fills: collected.sort((a, b) => a.time - b.time || a.tid - b.tid), denseMillis };
}
```
**Cursor — the last millisecond of the page, not `+ 1` (fixed on 2026-09-23).** The server returns 2000 oldest fills from `startTime` inclusive and slices the page by number of rows, not by a millisecond boundary. Fills from one order that passed through several order books split `time`, and this is not rare. Measurement on 2026-09-23 11:43Z: `userFills` for child addresses of the public HLP vault (addresses from `vaultDetails`) in a full response (2000 fills, six addresses) were considered duplicates of `time`. From 58 to 99% of fills divided a millisecond with another fill; maximum in one millisecond — from 12 to 182 fills depending on the address. If the page ends inside such a millisecond, cursor `max(time) + 1` silently loses its tail. To read the last millisecond for free: the request is the same, duplicates are removed by dedup. The exception is a whole page from one millisecond. Then cursor `ms + 1`, and fills of this millisecond beyond 2000 temporal pagination are unreachable. Such milliseconds should be shown, not silently lost (`denseMillis`).
**Dedup key** — `tid`. If `tid` is absent, the composite key `${hash}_${time}_${coin}_${px}_${sz}_${side}` is taken, with missing `hash` being an empty string. Duplicates at the page boundary here are expected: the last millisecond is read twice.
**Contradiction with old record and how resolved.** Until 2026-09-23 this section, TL;DR, `README.md`, `backtest-and-data.md` and skill `hyperliquid` gave `startTime = max(time) + 1`. The weak point was only the millisecond with more than 2000 fills. **Priority to new record**: the tail is lost on any page that ends inside a common millisecond — already at three fills per millisecond on the first boundary. The new record is pinned by tests `packages/hl-kit/src/history/fills.test.ts` ("pages forward past the 2000 cap and keeps fills cut inside one millisecond": 4500 fills, 3 per ms, all 4500 in place; mutation test on 2026-09-23 — with cursor `+ 1` this test gets 4498: one fill is lost at each of the two page boundaries) and `paginate.test.ts` (cursor for second page — last millisecond, not `+ 1`), as well as the above measurement. `hl-kit` (`fetchFillsByTime`) was implemented this way from the first commit.
---
## 4. Fill Fields
The format is the same for `userFills`, `userFillsByTime`, and WS channel `userFills`.
| Field | Type | Meaning |
|---|---|---|
| `coin` | string | Perp: `BTC`. HIP-3: `xyz:TICKER`. Spot: `@<index>` or a pair of the form `PURR/USDC`. |
| `px` | string | Execution price. |
| `sz` | string | Size (signless). |
| `side` | `'B'` \| `'A'` | `B` — buy/bid, `A` — sell/ask. |
| `time` | number | Time in ms. |
| `startPosition` | string | Position by the coin **before** fill, with sign: negative = short, `"0.0"` = opening from a flat. |
| `dir` | string | `Open Long`, `Close Long`, `Open Short`, `Close Short`; liquidations (substring `Liquidat`), `Settlement`, spot conversions. |
| `closedPnl` | string | Realized PnL for this fill. |
| `hash` | string | Transaction hash. |
| `oid` | number | Order ID. All partial fills of one order share the same `oid`. |
| `crossed` | boolean | `true` — taker (crossed the spread), `false` — maker (order stood in the book). |
| `fee` | string | Fee. |
| `feeToken` | string | Fee token, usually `USDC`. |
| `tid` | number | Unique trade id, key for deduping. |
| `twapId` | number \| null | Present on **each** fill, `null` for non-TWAP execution (live `userFills` of address 2026-09-22: 2000 fills, key present in all; in SDK type `UserFill` version 0.33.3 this field is mandatory). Code that distinguishes TWAP slice by the presence of a key will fail: check the value. |
| `builderFee` | string, may be absent | Fee paid to **any** builder (see §11). |
**What fill lacks:**
- **leverage**: it exists only in the `clearinghouseState.assetPositions[].position.leverage` of an open position;
- **balance or equity** at the moment of the trade.
From this:
- until there is no open position for a token, the leverage for that token is unknown, and margin from fill history cannot be calculated;
- for precise historical analysis, periodically snapshot `clearinghouseState` yourself.
### `dir` → sign of position change
| `dir` | Δposition |
|---|---|
| `Open Long`, `Close Short` | `+sz` |
| `Open Short`, `Close Long` | `-sz` |
| `Settlement`, spot conversions, etc. | not a cycle trade, skip |
- **Closing Fill:** `dir` starts with `Close` or contains `Liquidat` (case insensitive). Only such fills should be summed for `closedPnl`.
- **Substring Check:** `dir.toLowerCase().includes('close') && dir.toLowerCase().includes('long')` (or `'short'`), for opening — includes `'open'` plus side.
**Using `side` and `startPosition` instead of `dir`:** position after fill = `startPosition + (side === 'B' ? +sz : -sz)`.
### `crossed`: Maker or Taker
- `crossed === false` — maker fill.
- **Limit Order Crossing the Book:** its fills that cross the book when placed set `crossed = true` (taker).
## 5. Dedup and Grouping of Partial Fills
### Dedup by `tid`
- `tid` is unique for each fill. Deduplication by it is needed in any merge:
- main + "xyz" `userFills`;
- WS-snapshot `userFills` (`isSnapshot`) + stream;
- WS + REST `userFillsByTime`;
- duplicates from WS.
- Merging `[...hlFills, ...xyzFills]` without deduplication doubles `sz`, `closedPnl`, and PnL of closed trades.
- **Do not remove dedup even if HL starts honestly filtering by `dex`:** it will remain correct.
- **After fixing double counting, recalculate saved derivatives by fills:** sums (`sz`, `closedPnl`, volume) are halved, ratios of such sums do not change.
- **Memory for `Set<tid>` in a long-lived process:** trim the oldest records below a threshold.
### Grouping Partials in Order
- HL sends each partial execution as a separate line.
- **Scale:** Fill sizes are usually much larger than orders. One closing can give dozens of partial fills in one millisecond. A large limit order, which is eaten in pieces, turns 2000 fills into just a few orders.
- **Correct Group Key** (fixed on 2026-07-08):
```ts
const groupKey = (f: HlFill) =>
f.oid != null ? `${f.coin}|oid:${f.oid}` : `${f.coin}|${f.time}|${f.dir}`;
```
- **Why `oid`, not `(coin, time, dir)`**:
- A taker order that passes through several levels gives partial fills with the same `time`, and key `(coin, time, dir)` merges them;
- **A large resting-limit order**, which takers eat in pieces at different times, gives partial fills with **different `time`**, but one `oid`. Key `(coin, time, dir)` does not merge them: the number of "orders" is almost equal to the number of fills.
- **Aggregation within a group**:
- `sz` and `closedPnl` sum up;
- `fee` sum up;
- `px` — weighted average by size.
- Example (hypothetical numbers): an order of size 1.0 was executed as 0.6 + 0.4 at one price, with one `oid`.
- **Alternative on the server:** `aggregateByTime: true`. It merges only partial fills in one time slice, it will not merge a cutting limit order.
---
> **Do not overwrite, but sum up.** Deduplication through `Map.set` with the key `coin:timestamp:type` **overwrites** partial fills instead of summing them: only one fill remains from each group, and volumes with PnL are silently lost.
---
## 6. Position Reconstruction and flat→flat Cycles
**The cycle is counted only by `startPosition`:** position 0 → ≠0 → 0.
Do not reconstruct the position using the rule "accumulate openings, closings eat them up." At the window boundary (2000 fills), such reconstruction glues a phantom cycle from the tail of an earlier position and fails to distinguish exiting to zero from unloading a position opened before the start of the window.
**The zero threshold** is relative: `eps = max(sz · 1e-6, 1e-9)`. Coins have different scales, and sizes come as strings with decimals.
```ts
// fillsOfCoin are sorted by time
let open: { openTime: number; pnl: number } | null = null;
const trips: { openTime: number; closeTime: number; pnl: number }[] = [];
for (const f of fillsOfCoin) {
const sz = Math.abs(Number(f.sz ?? 0));
const before = Number(f.startPosition);
const dir = String(f.dir ?? '');
let delta = 0;
if (/^Open Long/.test(dir) || /^Close Short/.test(dir)) delta = sz;
else if (/^Open Short/.test(dir) || /^Close Long/.test(dir)) delta = -sz;
else continue; // Settlement / spot conversions
const after = before + delta;
const eps = Math.max(sz * 1e-6, 1e-9);
const wasFlat = Math.abs(before) <= eps;
const isFlat = Math.abs(after) <= eps;
if (wasFlat && !isFlat) open = { openTime: f.time, pnl: 0 };
if (open && (/^Close/.test(dir) || /Liquidat/i.test(dir))) open.pnl += Number(f.closedPnl ?? 0);
if (open && isFlat) { trips.push({ openTime: open.openTime, closeTime: f.time, pnl: open.pnl }); open = null; }
}
```
- **Variant through `side`:** `after = start + (side === 'B' ? +sz : −sz)`, threshold "flat" 1e-9.
---
## 7. PnL from Fills
### Realized PnL of a Closed Position
- Take `userFills` and filter close-fills for the desired `coin` and side, where **`time >= openedAt` of the current position**. Without this filter, previous trades on the same pair would be included.
- Sort by descending time.
- `exitPrice = Number(px)` of the last close-fill (if > 0).
- `realizedPnl = Σ Number(closedPnl)` for **all** such fills: partial closes and final. If none of the `closedPnl` are defined, `realizedPnl = undefined`.
- **Realized PnL from HL is more authoritative than `unrealizedPnl`** in the last snapshot before closing: they significantly diverge for volatile positions.
### ROI of the Trade: From Peak Margin
Partial closing reduces the size and margin of the position, while `realizedPnl` sums all close-fills. ROI from residual margin explodes. Example (hypothetical numbers): margin 1000, 90% of the position closed, realized +100, remaining margin 100 → "ROI" 100% instead of 10%.
- **Correctly:** store the peak margin of the position throughout its life (`peakMargin`) and calculate `pnlPercent = pnl / peakMargin * 100`.
### PnL from Own Closures Without Extra REST Calls
| Situation | Formula | Note |
|---|---|---|
| My CLOSE filled, have `entryPx` from the snapshot before the order | `closingPnl = (fillAvgPx − entryPx) × fillSize × dir`, `dir = +1` LONG, `−1` SHORT | Gross PnL without taker- and maker-fees. No race condition with fills. |
| `entryPx` not available, have `unrealizedPnl` of the whole position before the order | `closingPnl = wholePnl × Math.min(1, fillSize / requestedSize)` | On a partial fill, you cannot credit the entire uPnl: the remainder, closed later, will account for PnL twice (fixed 2026-07-08). |
| Exchanged position by an exchange TP/SL, not my order | — | No response to my order. Exact `closedPnl` and `fee` are only in heavy `userFills`. |
**Reconstruction of Implemented PnL from Own FILLED-Orders Log:**
- Order: chronological. Position key: `(account, coin, side)`.
- **OPEN or INCREASE:**
- position is empty → `entry = px`;
- otherwise `entry = (size_old·entry + size·px) / (size_old + size)`.
- **CLOSE:** `closedSize = min(size, pos.size)`, `pnl += (fillPx − entry) × closedSize × dir`.
- Remaining balance less than 1e-12 is considered zero. Rows with no execution (`size` or `px` ≤ 0) are skipped.
- Commissions and funding not included: this is a pure price PnL.
- There are no journal entries for exchange-based TP/SL closures: they are taken from `userFills`.
- Keep trade fills in your records at the moment of closure. The window `userFills` (2000) may already not return them later.
---
## 8. Delay in Indexing `userFills`
For very active accounts, the fill appears in `userFills` **a few seconds later** than the position update is sent via WS. The same can be observed between the disappearance of the position from `clearinghouseState` and the appearance of the closing fill.
- **Normal Path, Not an Error:** close-fill is not yet present → PnL temporarily takes from `pnl` of the last position snapshot (or a later request is made).
---
## 9. TWAP: `userTwapSliceFills` — Separate Feed
- Slices of TWAP orders **do not appear** in `userFills` / `userFillsByTime` whether `aggregateByTime: true` or `false` (verified on 2026-09-05). Their source is only `{ "type": "userTwapSliceFills", "user": "0xYOUR_ADDRESS" }`.
- Records contain `dir` (`Close Short`, `Open Long`…), and `closedPnl`. These are used to calculate the realized PnL for TWAP unwinds; a closing trade is identified by `dir.toLowerCase().includes('close')`.
- **Beware.**
- **Scenario:** Position changes, but `userFillsByTime` in the ±2 min window is empty in both `aggregateByTime` modes. One reason could be TWAP: its slices are only found in `userTwapSliceFills` (other reasons — checklist §15). Thus, conclusions about PnL based on a single `userFills` stream are not verified.
- **Rule:** Any assertion about account PnL from fills should be validated using both streams (`userFills`/`userFillsByTime` and `userTwapSliceFills`).
- Logic relying on **position** is unaffected: position accurately reflects TWAP execution. It's the fill-based analysis that suffers.
---
## 10. Orders: `historicalOrders`, `openOrders`, `frontendOpenOrders`
### `historicalOrders`
- **Request:** `{ type: 'historicalOrders', user }`, no other parameters. Returns the **last 2000 events** related to orders: placements, cancellations, executions.
- **Element Form:** `{ order: {...}, status, statusTimestamp }`. In case of a flat form, code takes `h.order || h`.
- `order.tif`: `'Alo'`, `'Gtc'`, `'Ioc'`;
- `order.timestamp`: milliseconds of order placement;
- `statusTimestamp`: milliseconds of last status change;
- `status`: `'open'`, `'filled'`, `'canceled'` (American spelling, lowercase).
- **How long do 2000 events cover:** from tens of minutes for an active account to hundreds of days for a quiet one.
- **The only place where your own cancellations are visible.** If your code does not log your own cancellations, canceled orders will be restored only here (status `canceled`).
### `openOrders` and `frontendOpenOrders`
| | `openOrders` | `frontendOpenOrders` |
|---|---|---|
| Fields | `coin, oid, side, limitPx, sz` | + `tif`, `reduceOnly`, `origSz`, `orderType` (`'Limit'`, `'Stop Market'`, `'Take Profit Market'`), `isTrigger`, `triggerPx` (string), `isPositionTpsl` |
| Weight | 20 | 20 |
| Purpose | remaining orders of your own | full order form (`tif`, reduceOnly), filtering TP/SL and triggers |
- `side`: `'B'` or `'A'`. `limitPx`, `sz`, `triggerPx` — strings, `oid` — number. Returns orders for all coins, filter by `coin` on the client.
- **`tif` may be absent.** Do not silently substitute a default value (e.g., `'Alo''): handle its absence separately.
- **Regular standing limit order (not TP/SL, not trigger, not reduce-only):** `!reduceOnly && !isTrigger && !isPositionTpsl && String(orderType).toLowerCase() === 'limit'`.
- Weight 20 → read rarely. These requests should not be a second-by-second source of truth.
**xyz / HIP-3 requires `dex`** (verified on SP500 2026-07-09). Without `dex`, orders xyz **do not come in**. `coin` in the response already has a prefix (`'xyz:SP500'`).
```ts
import * as hl from '@nktkas/hyperliquid'; // 0.27.x
const info = new hl.InfoClient({ transport: new hl.HttpTransport() });
const user = '0xYOUR_ADDRESS';
async function fetchOpenOrdersAllDex() {
const results = await Promise.allSettled([
info.frontendOpenOrders({ user }), // main
info.frontendOpenOrders({ user, dex: 'xyz' }), // HIP-3 xyz
]);
const orders = results.flatMap(r => (r.status === 'fulfilled' ? r.value : []));
const complete = results.every(r => r.status === 'fulfilled');
return { orders, complete };
}
```
- **Incomplete snapshot (`complete = false`) does not give the right to make a negative output** "no orders": the check actually did not take place. Positive output remains valid.
- If an error turns into `[]` and further becomes "no orders" without a single log, orders may hide behind a periodic HL crash. Log a warn of the type "check DID NOT occur — HL did not return part of open orders (main/xyz)".
### Statuses `orderUpdates` and oid
- **Statuses in WS `orderUpdates`:** `'open'`, `'filled'`, `'canceled'`, `'badAloPxRejected'`. Any other withdrawal status (the exchange withdrew the order, for example due to margin or reduce-only) should be logged separately: "order withdrawn by exchange: `<status>`".
- **oid on 2026-09-14** — 12-digit, ~5.44e11. They fit into JS `number` without loss of precision for now. Compare as `String(oid)`: in HL responses, this is the number.
- **Local accounting of your own open orders by fills:**
- `applyFill`: `sz -= fill.sz`;
- if the remainder ≤ 1e-12, the order is withdrawn and a "headstone" is set to prevent a late update from reviving it;
- a fill for an unknown `oid` (the response to placing it has not arrived yet) is put in pending and applied when the response arrives.
---
## 11. Builder fee: Field `builderFee` and Public Dump
### API Field
- In `userFillsByTime` / `userFills`, a fill may have a `builderFee` (string, may be absent).
- The match with the daily dump (2026-07-09) was exact to the cent on all checked addresses.
- **But `builderFee` is a commission for ANY builder** (clarified 2026-07-18). Filtering `builderFee > 0` inflates your earnings if the user trades through another application with a builder code: fills with `builderFee` in the API will be more than lines in your builder's dump, and the extras are from other builders.
- **Correctly:** filter your own fills by `oid ∈ <your sent orders>`.
- The window for `oid` of day D is `[D − 1 day, D + 2 days)`. An order sent at the end of a UTC day may be filled in the next one, and vice versa.
### Daily Dump builder fills
```
https://stats-data.hyperliquid.xyz/Mainnet/builder_fills/<builder>/<YYYYMMDD>.csv.lz4
```
- **Path:**
- `<builder>` — builder address in lowercase, path is case-sensitive;
- date — UTC day.
- **Format:**
- LZ4 Frame, inside CSV;
- columns: `time`, `user`, `coin`, `side`, `px`, `sz`, `builder_fee`. Resolve by name through `header.indexOf`; mandatory are `time`, `user`, `builder_fee`, otherwise error `unexpected CSV header`;
- `time` — ISO to seconds with `Z`, no milliseconds, e.g. `2026-01-02T03:04:05Z` (20 characters);
- `side` — words: `'Bid'` / `'Ask'` (in API letters `'B'` / `'A'`);
- `builder_fee` — USDC, actually deducted in favor of the builder. The dump accounts for partial fills, HL rounding, and fills that may not have been recorded by your DB, so this is the source of truth for tracking builder income.
- **Publication Lag:** usually 1–2 days, observed up to 3 (the file for 20260710 appeared on 20260713). If there's no file, S3 responds with **HTTP 403 AccessDenied**, sometimes 404. Both codes mean PENDING, not an error. Data is not real-time.
- **Days without builder fills do not exist:** the file is not created, and 403 will always be returned.
- **HL sometimes loses days forever:** two days in 2026-07 still returned 403 after 10 days, although neighboring days on both sides were published. The threshold for "freezing" is 4 days (maximum lag of 3 + 1 day buffer). Separate older days with 403/404 statuses to avoid a gap hiding behind the status "waiting for HL".
```ts
import LZ4 from 'lz4js';
async function fetchBuilderDay(builder: string, fillDate: string /* YYYYMMDD */) {
const url = `https://stats-data.hyperliquid.xyz/Mainnet/builder_fills/${builder.toLowerCase()}/${fillDate}.csv.lz4`;
let res: Response;
try { res = await fetch(url, { signal: AbortSignal.timeout(60_000) }); } // a daily file can be large
catch (err) { return { state: 'error' as const, error: `fetch failed: ${String(err)}` }; }
if (res.status === 403 || res.status === 404) return { state: 'pending' as const };
if (!res.ok) return { state: 'error' as const, error: `HTTP ${res.status}` };
const buf = new Uint8Array(await res.arrayBuffer());
const text = Buffer.from(LZ4.decompress(buf)).toString('utf8');
const lines = text.split('\n').filter(l => l.length > 0);
const header = lines[0].split(',');
const iTime = header.indexOf('time'), iUser = header.indexOf('user'), iFee = header.indexOf('builder_fee');
if (iTime < 0 || iUser < 0 || iFee < 0) return { state: 'error' as const, error: 'unexpected CSV header' };
const rows = [];
for (const l of lines.slice(1)) {
const c = l.split(',');
const fee = Number(c[iFee]); if (!Number.isFinite(fee)) continue;
rows.push({ time: c[iTime], user: c[iUser].toLowerCase(), fee });
}
return { state: rows.length ? 'ok' as const : 'empty' as const, rows };
}
```
### Dump Catching and Restoration from API
- **Idempotence:** "replace all day's lines" + status log `OK` / `EMPTY` (0 lines) / `PENDING` (403/404) / `ERROR`. Catching the same day again is safe.
- **Catching** should not lose days missed during downtime: candidates are all days from the first successful one to yesterday, which still don't have a status of `OK` / `EMPTY`, not only the last few fixed-length days. Today's day should be left untouched: there's no file for it yet.
- **Restoration of a Lost Day from API** (`userFillsByTime` for a day + `builderFee` + filter by own `oid`):
- the set of addresses should cover all who could pay the fee on that day, not only current users; addresses should be in lowercase;
- **all or nothing.** If any address doesn't respond after retries, the day remains `PENDING`. Otherwise, replacing the day would record an incomplete set, and it would look complete;
- if there are no own `oid`s in the window, distinguishing one's fee from others' is impossible, and the day stays `PENDING`;
- in the log, mark the origin: the day was restored from API, not a dump.
## 12. Ledger: `userNonFundingLedgerUpdates`
**Request:** `{ type: 'userNonFundingLedgerUpdates', user, startTime }`, weight 20. Response: `[{ time, hash, delta: { type, ... } }]`. All amounts — **strings**.
| `delta.type` | Fields | Perp Leg Stream |
|---|---|---|
| `deposit` | `usdc` | `+usdc` |
| `withdraw` | `usdc` | `-usdc` |
| `accountClassTransfer` | `usdc`, `toPerp` | `toPerp === true ? +usdc : -usdc`. This is a spot↔perp transfer; skipped on unified account. |
| `internalTransfer`, `subAccountTransfer` | `usdc`, `user`, `destination`, `fee` | `destination === me ? +usdc : (user === me ? -(usdc + fee) : 0)` |
| `send` | — | Transfers, including between dexes. Fields are not parsed (not verified). |
| `vaultDeposit` | `usdc` | `-usdc` |
| `vaultWithdraw` | `netWithdrawnUsd` | `+netWithdrawnUsd` |
| `rewardsClaim` | `amount`, `token` (e.g. `'USDC'`) | **not stream**: trading result |
| `liquidation` | — | **not stream**: trading result |
| `spotTransfer`, `vaultCreate`, `vaultDistribution` | — | not stream (skipped) |
**Examples of Form:**
```json
{ "time": 1757000000000, "delta": { "type": "withdraw", "usdc": "100.0" } }
{ "time": 1757000000000, "delta": { "type": "accountClassTransfer", "usdc": "50.0", "toPerp": true } }
{ "time": 1757000000000, "delta": { "type": "rewardsClaim", "amount": "12.345678", "token": "USDC" } }
```
**Rules:**
- **Withdrawals or transfers are confirmed only by a ledger entry.** An empty ledger for a period means no withdrawals occurred. One cannot claim "withdrawn funds" without a ledger entry. A drop in accountValue on an expired snapshot is not grounds to invent "withdrawal".
- **Drawdown ≠ loss.** An internal `send` can explain a sharp drop in perp accountValue during a period with positive `closedPnl`. When analyzing drawdown, check the ledger, not just accountValue.
- **PnL for the period based on accountValue:** deposits and withdrawals shift the base, not trade results. Internal spot↔perp transfers of deposits or withdrawals are not considered.
- **Persistently store hashes of processed ledger entries.** Otherwise, a restart will cause the same deposit to shift the base again.
### `userFunding`
Response format — as per HL documentation (`backtest-and-data.md` §4), limits not verified (see "Open Questions"). Funding in the price PnL from fills **does not include** funding, which needs to be accounted for separately.
Verified 2026-09-22 (read-only): a request **without `startTime`** returns 200 (for zero address — `[]`), not 422 — unlike `fundingHistory`, where `startTime` is required (its absence yields 422 `Failed to deserialize…`). The documentation's statement that "startTime is required" for `userFunding` did not hold true in practice; whether HL returns the entire history or a window without `startTime` on an account with funding remains not verified.
---
## 13. Leaderboard
- **Request:** `GET https://stats-data.hyperliquid.xyz/Mainnet/leaderboard`. This is a separate static endpoint, not part of the info API. The response is large: cache locally and reuse.
- **Response:** `{ leaderboardRows: [...] }`. Size: ~39k rows (2026-06), ~44–45k (2026-09).
- **Row:**
- `ethAddress`;
- `accountValue` (string);
- `windowPerformances` — array of pairs `[window, { pnl, roi, vlm }]`, windows `'day'`, `'week'`, `'month'`, `'allTime'`.
```ts
const lb = await (await fetch('https://stats-data.hyperliquid.xyz/Mainnet/leaderboard')).json();
const win = (row: any, w: 'day' | 'week' | 'month' | 'allTime') =>
Object.fromEntries(row.windowPerformances)[w] as { pnl: string; roi: string; vlm: string };
const rows = lb.leaderboardRows.map((r: any) => ({
address: r.ethAddress.toLowerCase(),
accountValue: Number(r.accountValue),
allTime: win(r, 'allTime'),
}));
```
### Blind Spots
Not verified
1. **Threshold is necessary but not sufficient.** Observation (2026-09): there are no rows in the list where volume < $10M and `accountValue` < $100k (0 violations across 44 092 rows). The rule "exists if volume ≥ $10M OR equity ≥ $100k" describes who is **in** the list but not who is **not**.
2. **Large accounts are missed** (Observation 2026-09-06), non-vaults (`vaultDetails` = `null`); the reason is unknown. **Conclusion:** leaderboard is not a complete list for any account size.
3. **Addresses outside the leaderboard have no PnL, ROI, or turnover from this source.** This is data absence, not zero.
---
## 14. Load and Cache
- **Fills are append-only**, so a short TTL cache of successful responses safely dampens spikes of identical requests.
- Cache only **successful** responses: after an error, a real retry is needed.
- **In-flight dedup:** parallel identical requests reuse one Promise. Results in one HTTP instead of two heavy requests (40 weight instead of 20).
- Each `userFills` / `userFillsByTime` — heavy (20 weight); an extra request with `dex:'xyz'` doubles this cost.
---
## 15. Diagnostic Checklists
**"Position perpa changes while fillers remain the same".** Verify in order:
1. `userFills` / `userFillsByTime` with `aggregateByTime` true **and** false within a ±2 min window.
2. Is it a vault address (`vaultDetails`).
3. Subaccounts: position might have moved.
4. Ledger updates (transfers).
5. Spot balance of the corresponding coin: no netting against perpa.
6. **`userTwapSliceFills`**: TWAP slices missing from `userFills`.
**«Equity of the account dropped»:** `closedPnl` in `userFillsByTime` + `closedPnl` in `userTwapSliceFills` + ledger + freshness of snapshot. Do not build history based on two snapshots.
---
## 16. Pitfalls
| What Breaks | Why | How to Fix |
|---|---|---|
| PnL and `sz` doubled | Merging `userFills` without `dex` and with `dex:'xyz'`: HL ignores `dex` | One request; deduplicate by `tid` always |
| Double heavy-weight for each account | Separate "xyz-request" of `userFills` / `userFillsByTime` | One request, xyz — by prefix `coin` |
| Orders of xyz "disappeared" | `frontendOpenOrders` without `dex` does not return them | Query per each dex, flag `complete` |
| "No orders" due to HL failure | Error → `[]` → negative verdict | Incomplete snapshot does not grant right for negative output; log |
| Lost part of volume, PnL distorted | `Map.set` overwrites partial fills by key group | Sum `sz` / `closedPnl` / `fee` in group, `px` weighted average |
| Partial fills of one limit order counted by different orders | Partial fills of a canceling limit order have different `time` | Group by `oid`, fallback `(coin, time, dir)` |
| Phantom cycles at window boundary | Reconstructing position from 2000 fills | Use `startPosition`; eps relative |
| PnL of position = PnL of last fill | Partial closes are separate fills | Σ `closedPnl` for all close-fills with `time >= openedAt` |
| Past trades on the same pair "stuck" | No time-opened filter | `time >= openedAt` |
| ROI of trade exaggerated by orders | ROI from residual margin after partial close | ROI from `peakMargin` |
| PnL calculated twice on partial exit | Whole uPnl of position credited | `wholePnl × min(1, fillSize / requestedSize)` |
| Incorrect PnL output based on `userFills` | TWAP-slices not in `userFills` | Check `userTwapSliceFills` as well |
| Closing fill missing immediately after close | Indexing of `userFills` lags behind WS by seconds | Fallback to snapshot and retry, not an error |
| Leverage ratio for historical fills "unknown" | Fill lacks `leverage` | Only from `clearinghouseState` of open position; snapshot manually |
| Excessive builder revenue reported | `builderFee` — commission to any builder | Filter by own `oid` in window `[D−1, D+2)` |
| Data dump hole hidden behind "waiting for HL" | Day's dump lost forever (permanent 403) | Threshold 4 days → restore from API, all-or-nothing |
| Day recorded incomplete | Some addresses did not respond but day was recorded | Do not write partial success |
| PnL base shifted twice | Same deposit processed again after restart | Persist hashes of ledger entries processed |
| "Withdrawed funds" or "drawdown" without reason | Withdrawal based on two snapshots `accountValue` | Ledger + `closedPnl` from both streams + freshness of snapshot |
| Silently empty data for xyz | Request error with `dex` swallowed as `[]` | Do not swallow errors, do not pass extra `dex` |
| Tail of fills dropped during pagination | Cursor `newest + 1`, but the page broke off within milliseconds (a common occurrence for an active account — §3); more than 2000 fills in one millisecond are unachievable with any cursor | Cursor = last millisecond of the page + dedup by `tid`; drop through `ms + 1` and show (`denseMillis`) |
| Leaderboard taken as a full list of accounts | The leaderboard is incomplete for any account size | Do not consider it complete; no data ≠ 0 |
| Incorrect `tif` for order | Missing `tif` replaced with `'Alo'` | Handle missing `tif` separately |
---
## 17. Open Questions / Not Verified
- **`userFunding`:** response format (except documentation), limits, weight and pagination not verified. Funding in the price PnL from fills does not include it. Measurement on 2026-09-23: a 500-line response by increasing `time`, payments of all coins per hour — with one label (`backtest-and-data.md` §4); that 500 is a ceiling, and the weight has not been verified.
- **Field `liquidation` in fill:** structure is not fixed. Only known is that liquidation fills are recognized by the substring `Liquidat` in `dir`, and their `closedPnl` should be included in the closing PnL. There is also a type `delta` for `liquidation` in the ledger.
- **`twapId` in fill:** key exists for each `userFills` entry and equals `null` for normal execution (2026-09-22, §4). Whether it is filled with an id of TWAP by type SDK — live testing not done: TWAP slices do not appear in `userFills`, their stream is `userTwapSliceFills`.
- **Depth of `userFillsByTime`:** no limit after which old fills are unavailable has been measured.
- **Weight of `historicalOrders` and `userTwapSliceFills`**, as well as the dependency of heavy-query weight on response size, not measured.
- **`aggregateByTime: true` and pagination:** how aggregation interacts with the 2000 limit and cursor for the last millisecond of the page (§3) is unverified.
- **Order of `userFills`:** in one observation it was "not guaranteed to be sorted." The sort direction was not established; sort it yourself.
- **Reason for leaderboard blind spot** unknown. Threshold ($10M / equity $100k) — empirical on a single snapshot from 2026-09; the volume window is not explicitly confirmed.
- **`send` in ledger:** fields and flow sign for dex-to-dex transfers not analyzed.
### Conflicts Between Records and How Resolved
- **Key for partial fills grouping.** Early records (2026-05 and 2026-06): `(coin, time, dir)`. Later correction (2026-07-08): `oid`, fallback `(coin, time, dir)`. **Priority to the later**, because `(coin, time, dir)` does not join a slicing limit order with different `time`.
- **`builderFee > 0` = own income.** 2026-07-09: "reproduces dump down to the cent". 2026-07-18: "commission for any builder, overestimates". **Priority to the later**: match is correct only for users without foreign builder codes, filter by own `oid`.
- **Leaderboard size:** ~39k (2026-06) and ~45k (2026-09) — growth over time, not a conflict.
- **Separate requests for main and xyz when reading fills.** Early record made them separately. Later it was found that without `dex` everything comes in one go, and the second call gives duplicates. **Priority to the later:** one request.
---
Dates of checks are mentioned in the text. API HL changes — recheck limits and response forms.
---
<!-- license-footer -->
_© markpaper authors. Licensed under [CC BY 4.0](LICENSE.md): when publishing or adapting, credit “markpaper — Hyperliquid knowledge base” and provide links to the original and the license._