Building reliable AI agents with Effect (AI Engineer World's Fair 2025)

This week, our co-founder and CTO Michael Fester gave a talk at the AI Engineer World's Fair (June 3–5), a conference bringing together leading engineers and builders working on agentic systems, developer platforms, and AI infrastructure. Michael’s session focused on how we use the Effect TypeScript library to power production-grade AI-native customer support systems at 14.ai.

Watch the talk

If you are building with LLMs or interested in agentic system design, Michael’s talk offers a practical look into the decisions and tradeoffs that come with building these systems in production.

Why Effect?

TypeScript is a strong foundation, but it begins to fall short when dealing with long-running workflows, uncertain outputs from LLMs, and tightly coupled service interactions. Effect extends TypeScript with tools for handling complexity in a structured way. This includes stronger concepts of typing via schemas, built-in concurrency, structured error handling, dependency injection, and observability.

Effect integrates naturally into an existing TypeScript codebase. It allows teams to adopt it incrementally and provides guardrails that help new engineers stay productive while avoiding common pitfalls.

• Strong type guarantees across the stack
• Built-in concurrency, streaming, interruption and retry mechanisms
• Clean dependency injection system
• Powerful composition primitives
• Structured error modeling
• Easy observability (OpenTelemetry)

Modeling agents and workflows

Agents in 14.ai act as planners. They evaluate input, generate a plan, select the appropriate action or workflow, and repeat until the task is complete. Actions are modular units of execution. Workflows are deterministic processes like canceling a subscription or retrieving logs. Sub-agents group related workflows and actions into domain-specific modules.

To model these workflows, we created a domain-specific language that uses Effect’s functional, pipe-based design. It allows us to handle branching, retries, state transitions, and other advanced logic in a clear and composable way.

const cancelSubscriptionFlow = pipe(
  Flow.first(askUserForCancellationReason),
  Flow.andThen(getSubscriptionDetails),
  Flow.doIf("The user has only been subscribed for one month", {
    onTrue: (flow) =>
      flow.pipe(
        Flow.andThen(offerOneMonthFree),
        Flow.doIf("The user says yes to a free month", {
          onTrue: Flow.andThen(addOneMonthFree),
          onFalse: Flow.noOp,
        }),
      ),
    onFalse: Flow.noOp,
  }),
  Flow.doIf("The user still wants to cancel", {
    onTrue: (flow) =>
      flow.pipe(
        Flow.andThen(cancelSubscription),
        Flow.andThen(emailUserAboutCancellation),
      ),
    onFalse: Flow.noOp,
  }),
);

Reliability and testing

One of the most critical challenges in deploying LLMs is ensuring graceful degradation when systems fail. For example, if one model provider goes down, we fall back to an alternative with similar capabilities. Effect enables retry policies that can track failed attempts and avoid repeating them. Token streams from LLMs are duplicated for analytics and storage with minimal overhead, all handled through Effect’s streaming features.

For testing, we rely heavily on dependency injection to simulate failure modes and mock external services. This makes it easy to test edge cases and ensure consistent behavior under real-world conditions.

• Fallbacks — If one LLM provider fails, we fall back to another with similar performance characteristics. State tracking prevents retrying failed providers.

• Streams — We duplicate LLM token streams—one to the user, one for storage and analytics. Effect allows us to do this easily.

• Testing — Dependency injection lets us simulate LLM failures in tests. Each service has mock versions we can swap in.

Lessons learned

Effect brings rigor and structure, but it also requires discipline. It is easy to make assumptions based on the "happy path", only to discover errors being silently swallowed in production. Managing dependencies across large systems can become difficult without clear conventions. But while the learning curve is steep, the long-term payoff in terms of developer velocity and system resilience is significant.

• Happy paths are easy — You can get a false sense of safety (e.g. catchAll).
• Dependency injection can get messy — Service providers are sometimes hard to trace.
• Learning curve is real — But it "clicks" and pays off eventually.

Effect lets us write predictable, durable systems. But it's not magic: you still have to think!

Key takeaways

Effect is particularly well suited to teams building LLM/agentic applications. Its features for controlling non-deterministic or concurrent situations, managing service dependencies, and enforcing strict type safety make it easier to create systems that behave predictably at scale.

You do not need to be an expert in functional programming to benefit from Effect. Start small, apply it to one service or component, and let the benefits grow over time.