Automating Hand History Ingestion

Building range tools is only useful when the data is trustworthy. Our ingestion service chews through raw hand histories from multiple poker rooms and normalizes them into a shared format that downstream features can rely on.

Normalizing the inputs#

The import queue runs inside our worker cluster. Each worker pulls a batch from S3, validates the schema, and forwards any problems to a dead-letter topic. When everything looks good we translate the hand into our canonical JSON representation and store it in Postgres.

// simplified parser entry point
export async function parseAndStore(hand: RawHand) {
  const parsed = await parseHand(hand)
  await db.transaction(async (tx) => {
    const playerIds = await upsertPlayers(tx, parsed.players)
    await tx.insert(hands).values(toPersistedHand(parsed, playerIds))
  })
}

Watching for regressions#

Every build triggers contract tests that replay tricky hands we have seen in production. If a parser update changes the output we get alerted before the change ships. The same suite powers our documentation snippets, so the examples never drift from reality.

What is next#

The next milestone is surfacing ingestion metrics directly in the app. Once that lands you will be able to see how many hands were parsed, rejected, or retried in the last 24 hours without leaving Pokerscope.