Where does one state machine stop and the next begin? What goes in, what stays out, how do machines cooperate, and how do their diagrams stay honest?

Machines model behaviours, not apps

The thing being modelled is not “the application”; it’s not even “the component”; it’s a behaviour. One behaviour with a single lifecycle. A story with a beginning, middle, and a discrete set of possible moves: the draft save flow, the checkout wizard, the connection to the sync server.

Your state machine is not trying to model the whole app; that was never the goal. An application is a collection of running machines.

Imagine explaining the feature to product at a whiteboard. What you would draw is the experience a person goes through when they use a feature (the possible states, how they connect, the “only if” conditions); that is the machine. You would never draw a button’s hover colour, the exact...

There are 5 concepts you need to understand to start using Effect.

1. Effect.tryPromise

Think of Effect like a souped-up Promise that’s honest about errors.

// The Promise Version of getUser
try {
  const user = await getUser("123")
  console.log(user.name)
} catch (error) {
  // Is this a network error? 404? 500? Who knows!
  console.error("Something went wrong:", error)
}

Versus…

// The Effect Version

import { Effect } from "effect"

Effect.tryPromise({
    try: () => getUser(id),
    catch: () => `DatabaseError` as const,
  });

Since we saved DbError as a TypeScript literal, hovering over this Effect will show us the error at the type level <User, DatabaseError, never>.

The first argument is the success value (the user we wanted), the second argument covers any known errors associated with this “promise” (in this case, a DatabaseError), and the third argument is the...

The problem with using LLM Judges is that it’s hard to trust them. If an LLM judge rates your output as “clear”, how do you know what it means by clear? How clear is clear for an LLM? What kinds of things does it let slide? or how reliable is it over time?

In this post, I’m going to show you how to align your LLM Judges so that you trust them to some measurable degree of confidence. I’m going to do this with as little setup and tooling as possible, and I’m writing it in Typescript, because there aren’t enough posts about this for non-Python developers.

Step 0 — Setting up your project

Let’s create a simple command-line customer support bot. You ask it a question, and it uses some context to respond with a helpful reply.

mkdir SupportBot
cd SupportBot
pnpm init

Install the necessary dependencies (we’re going to the ai-sdk and evalite for testing).

pnpm add ai @ai-sdk/openai dotenv

One big barrier to testing prompts systematically is that writing evaluations usually requires a ton of setup and maintenance. Also, as a TypeScript engineer, there aren’t that many practical guides on the topic, as most of the literature out there is for Python developers.

I want to show you how write your first AI evaluation framework with as little setup as possible.

What you will need

• LLM API key with a little credit on it (I use Gemini for this walkthrough).

Step 0 — Set up your project

Let’s start with the most basic AI feature. A simple text completion feature that runs on the command line.

mkdir Summarizer
cd Summarizer
pnpm init

Install the AI SDK package, ai, along with other necessary dependencies.

pnpm i ai dotenv @types/node tsx typescript

Once you have the API key, create a .env file and save your API key:

GOOGLE_GENERATIVE_AI_API_KEY=your_api_key

Create an...

Getting back into development after years, I started writing a little Express server for a new project. I realised I don’t have an implicit checklist of best practices in my head anymore.

I know I need to handle errors on my endpoints and functions, especially the async ones. I’ve forgotten what errors I need to defend against. The specifics are all fuzzy. It feels vague and overwhelming.

What I need is an explicit checklist. Like a list of 16 things I must check before publishing a commit.

Maybe it’s not 16 things; it could be 36. the point is once I have a checklist, it will be easier to add, adjust, or change things as needed. Now I’m just guessing and I can already see the mess I’m going to get myself into.

Variables

let apples = 5;
let speed: string = 'fast';
let hasName: boolean = true;
let nothingMuch: null = null;
let nothing: undefined = undefined;

Built in objects

let now: Date = new Date();

Arrays

let colors: string[] = ['red', 'green', 'blue'];
let myNumbers: number[] = [1, 2, 3];
let truths: boolean[] = [true, true, false];

Classes

class Car {}
let car: Car = new Car();

Object literals

let point: { x: number; y: number } = {
  x: 10,
  y: 20,
};

Functions

const logNumber: (i: number) => void = (i: number) => {
  console.log(i);
};

//or

const logNumber =  (i: number): void  => {
  console.log(i);
};

I learned how to test code using Jest. Today I’m working on a codebase that doesn’t have Jest and I have to use Jasmine and Sinon instead.

The most confusing bit is going through the documentation and trying to understand what fakes, spies, stubs, and mocks are. In Jest, I just dealt with mocks. I thought everything was a mock.

I’ve learned that all of these things fall under the umbrella of test doubles. I think I’ve figured out what the difference between each type of test double is now…

Spies - Listen

So a spy is when you just listen to the implementation of a dependency. You run the original function, but you just want to test whether it was run, how many times it was run, and what parameters it was run with.

Mocks - Bypass

A mock is when you bypass a function altogether. If you have a dependency that’s coupled to your code that does important stuff but is unrelated to what...

You can use the test scheduler to test sequences in epics but integration testing breakdown if you don’t include redux observable in your test redux wrapper.

Here is how I successfully managed to integrate redux observable with react testing library by mocking out the store. This wrapper replaces the default react-testing-library render method with a render method that gives components access to redux, react-router and redux observable in a test environment.

import React from 'react';
import { render as rtlRender } from '@testing-library/react';
import { Router } from 'react-router-dom';
import { createMemoryHistory } from 'history';
import { createStore, applyMiddleware } from 'redux';
import { Provider } from 'react-redux';
import { createEpicMiddleware } from 'redux-observable';
import { rootReducer, rootEpic, dependencies } from './store';

function configureStore(initialState) {

You can make redux complicated but at the most fundamental level, you have a view layer that lets you fire actions that can update the state. There is middleware and actions hit reducers which update the store, but let’s consider those implementation details. Practically speaking, it’s a tiny action loop where View > Action > State.

Redux observable introduces a separate action loop that runs alongside the tiny loop. The view layer lets you fire actions that trigger epics, which can fire off more actions, that can update the state. So it’s View > Action > Epic > Action > State.

This diagram helped me put all this all together.

A few important details:

1. All actions will run through the tiny loop before they run through the epic loop.
2. All Epics must return an action or it’s doom.
3. If your epic returns the same action it received, you will create an infinite loop of doom.
4. If your...