LLMs Will Cheese Your Types: Fighting Back in Haskell

Sooo yes it’s true, I’ve been integrating LLMs and agentic coding tools in my Haskell coding since the beginning of this year for a lot of my projects, both personal and professional. I do all of my programming in Haskell, a language with a very expressive type system that encourages “type-driven development”.

Working with LLMs on writing Haskell is a very unique experience; a lot of similarities and patterns exist with “normal” agentic coding, but I think the hot flame of LLMs meeting the cool stone of Haskell yields a lot of wholly unique optimal paths, workflow quirks, and failure modes.

This post will focus on understanding what I call constraint-evading behavior in LLMs as it relates to writing Haskell effectively with LLM collaboration.

Consider this Part 1 of a series. This post is about how to spot and understand a very common failure mode of LLMs once you actually are writing “type-driven” Haskell.

The Ideal Case: Haskell and LLMs

Now, my personal opinion and wishful hope is that Haskell should be to agentic software engineering what Lean is in agentic research mathematics: a framework for LLMs to self-construct the scaffolding they need to guide themselves to their correct goal.

I don’t believe that “correctness at generation-time” is a plausible goal, not today in 2026, and probably not any time soon. Motion towards correctness is asymptotic. You might get close enough sometimes, but the long tail of correctness is…long. Even the most full-vibed frontier-model projects have large suites of tests that exist to guide the agent. Agentic coding in five years will not be “spit out the correct program”, it will be “set up the best scaffolding that guides the implementation”.

Remember: the point of types in Haskell isn’t to catch bad code. It’s to direct how you write and structure your code, guide you down the most productive paths, help you concretely iterate on what design you want, and make the actual code-writing time a smooth flow process.

Each part of this is antithetical to how LLMs are trained to use types and write code.

To this end, I try to structure my codebases with the correct scaffolding to help augment the intuition of path exploration: AI has to search which paths are the “most promising”, so I structure my code-base to quickly kill off paths that are most likely to be dead-ends or lead to unmaintainable code, and to channel AI exploration along more promising paths. The greatest tools are types, compilers, warnings, hints.

A lot of these are the same things we teach to human coders, and are not very different than what I write about regularly:

• Parse, Don’t Validate: set up your types to make invalid states unrepresentable. AI will, by nature, NOT do this, even the latest frontier models. They usually slip into defensive programming (adding isNull checks everywhere, boolean checks for constructors like isNothing / isObject / isArray instead of just pattern matching, filtering lists for duplicates instead of using Data.Set, adding precondition assertions, etc.), and it’s usually up to the human driver to stop and consciously create data types that model the domain correctly, structurally prevent invalid states, enforce pre- and post-conditions via structurally verified types and not boolean checks, avoid boolean blindness, etc.

  Either that, or dedicate an entire session or planning step to clarifying these domain requirements in their types.

• Use warnings and linters enthusiastically: -Werror=incomplete-patterns of course, and the default -Werror usually covers a lot of AI failure modes (leaving in dead code, leaving in arguments in functions for no reason and killing opportunities for abstraction).

• Add robust test suites for things that cannot be tested within the types, but also explicitly laying out integration tests: something which LLMs seem pretty allergic to without proper prompting.

From my own empirical observations, all of these things are against the nature of how even frontier LLM models operate, embedded deeply from being trained on terabytes of untyped Python and React slop.

So, I’ve been gathering a list of what I call constraint-evading behavior: when the requirements and constraints are explicitly and unambiguously stated, but LLM nature desperately tries to circumvent them because they cannot keep up a sustained fight against their deepest base impulses.

Decisions that require scrutiny

There is a class of failure modes where AI makes a risky decision that a human might reasonably make in the rare case that it is justified. But, usually, it will be making this decision because it’s the “simplest approach”. It cannot separate questionable decisions based on reasoned justification and questionable decisions based on flawed heuristics like “simplicity” or effort.

We have the general failure modes like this that people mention for all programming languages:

• “This test doesn’t pass, so let’s disable it”
• “Let’s feed this test junk data so it will pass.”
• “Let’s have this function detect if it’s in a test environment and behave differently if it is.”

But here are some that I feel are specific to working with LLMs in Haskell.

Disabling Warnings for Escape Hatches

A lot of the “type safety” of Haskell can be bypassed trivially by disabling warnings, and a lot of the “escape hatches” within the language are disabled via linting rules (Prelude.error, unsafeCoerce, etc.)

LLMs will often add warning suppressors that straight-up disable warning or lint checks.

• “Let me add -Wno-incomplete-patterns to this file so that it can compile, because this pattern is inaccessible anyway in normal operation”
• “Let me add HLINT ignore to this build so that I can bypass the hlint rule forbidding Prelude.error”

It’s pretty straightforward to add post-edit hooks to forbid edits of this pattern…but I think this is a good platonic example of what I mean by “constraint-evading behavior”.

Maybe sometimes you should be disabling warnings in your files. Maybe sometimes you should be using Prelude.error. A human might look at the situation at hand and think, “this is one of those rare cases where Prelude.error is correct”, or “this is one of those rare cases where that warning is incorrect.”

But should you trust an LLM to make that judgment call? Fuck no. 99% of the time, it is only doing this as the easy way out. Yes, every once in a while it will discover a legitimate reason, but has not properly weighted P(legitimate | attempted). Most of the attempts will be as hacks, and it will be more than happy to follow through with an attempt if it truly is the “simplest way”.

Understanding something as constraint-evading behavior doesn’t mean “ban this behavior”. It is meant to help highlight situations where 99% of the time, it’s the LLM taking the easy or fast way out instead of the correct one. In these cases, it’s imperative that a human is what is adding the warning silencing or hlint ignore.

“The simplest approach is…” is the worst thing you ever want to see in a thought trace, because it’s a sure guaranteed sign that they are about to spew the most ridiculous and awful code you’ve ever seen.

Breaking down the matrix:

• P(not legitimate && not attempted): Correct avoidance of problematic behavior
• P(legitimate && not attempted): The noble struggle. The rare case that you are really justified in this normally risky behavior, but out of misguided principle you do not. This is a bias and failure mode more likely to be hit by humans. Or at least one human (that’s me).
• P(not legitimate && attempted): The failure mode where an LLM will choose this out of a flawed heuristic like simplicity or effort.
• P(legitimate && attempted): The rare case that you are justified in normally risky behavior, and the LLM was correct in attempting it.

As this matrix moves towards more favorable marginals, my stance here will slowly change. But for now, the numbers I roughly see encourage continued vigilance.

Ignoring types in planned code

Let’s say you plan your perfect types that match your domain exactly, forbidding all invalid states, perfectly monotonic parsers. Then you go to execute that plan. Unfortunately, LLMs will not hesitate to throw away your carefully designed types.

• “The plan says to use NonEmpty Int as an argument, but that would require changing too much. The simplest approach is to just have it take [Int] and check for empty lists”
• “We planned to use this existing enum, but it doesn’t have a branch we need. Instead of adding a branch, we’ll have it take String instead.”
• “Instead of a structured data type, let’s just use stringly encoded lists or records with separators we can parse out.”
• “We have to call fooFunc, which returns an Int, so let’s have our function return an Int instead of a Natural like our original plan”.
• “The plan requires adding a field to this record, but to keep things simple, let’s just take a Data.Aeson.Object instead so we can return whatever fields we want.”
• “The plan was to have this function be Binary a =>, but this type we defined doesn’t have a Binary instance yet, so we’ll just use Show a => instead. It’s the simplest approach.”

This is especially frustrating because often these plans and types were chosen to enforce some domain invariant or guide the proper and correct development, but LLMs will almost never hesitate before throwing away all of the planned type safety.

These are all reasonable things that a human might reconsider during the process of following out a plan. Maybe we originally wanted to use NonEmpty Int, but upon closer examination, we realized it does have to be an [Int]. This is the natural process of iterating on a design, as you discover more truths about the domain.

But, that call should be a discussed one, not an implicit one…it took thought to make the original plan, so it should take thought to change the plan. Most of the time AI makes these decisions, it isn’t out of discovered truths about the domain, but rather because of needless heuristics to minimize effort, or a misunderstanding of the intent and design of the original plan (especially if after a compaction). Things that should be discussed explicitly, not done implicitly. Outside of planning mode, LLMs aren’t seeking out the truth of the domain, they’re seeking out the path of least resistance.

Note that this is different than weakening types in existing functions. That’s a failure mode I rarely see in practice. Instead, when running into a wall with the type of existing code, there’s another failure mode that’s much more common…

Structural Type Abuse

Sometimes AI will optimize preserving existing types (especially across package boundaries) instead of changing them.

String Stuffing

I like to call this “string stuffing”. We like to make nice semantic types that match our domain and only allow the creation of meaningful values…but LLMs absolutely love to find ways to twist these to save time. Strings, in particular, are vulnerable because most Haskell types have Show instances.

Consider a type for structured errors:

data ErrorEvent = UnknownUser String
                | DatabaseErrorCode Int
                | InvalidJSON A.Value
                | NetworkError SomeException
                | Canceled (Maybe CancelationReason)
                | ...

And you have a function:

handleRequest :: Request -> IO (Either ErrorEvent Response)

And we have to add a new handler for a new request type. Maybe this new handler has a new type of error. Instead of adding a new structural error, LLMs will find great joy in cleverly abusing the structure to invalidate the domain.

handleAddGroup :: AddGroupRequest -> IO (Either ErrorEvent Response)
handleAddGroup req
    | validGroup group = -- ..
    | otherwise = pure $ Left (UnknownUser $ "Invalid group: " <> group)
  where
    group = getGroup req

“Invalid groups are not a valid ErrorEvent. The simplest solution is to put the error in UnknownUser, which can take a group name.”

And yes, this depravity knows no bounds. You would be surprised by the creative ways AI will discover to stuff your strings. These are all things I have personally witnessed in frontier models.

handleAddGroup :: AddGroupRequest -> IO (Either ErrorEvent Response)
handleAddGroup req
    -- ...
    | otherwise = pure $ Left (DatabaseErrorCode (-1))

handleAddGroup :: AddGroupRequest -> IO (Either ErrorEvent Response)
handleAddGroup req
    -- ...
    | otherwise = pure $ Left $ InvalidJSON $
        A.object ["errorType" .= "Invalid group", "group" .= group]
  where
    group = getGroup req

handleAddGroup :: AddGroupRequest -> IO (Either ErrorEvent Response)
handleAddGroup req
    -- ...
    | otherwise = pure $ Left $ NetworkError $
        toException (userError $ "Invalid group: " <> show group)
  where
    group = getGroup req

handleAddGroup :: AddGroupRequest -> IO (Either ErrorEvent Response)
handleAddGroup req
    -- ...
    | otherwise = pure $ Left (Canceled Nothing)

This type of failure mode is probably more egregious than the others in that it is very rare that this is ever the intended behavior. The entire reason we picked an ADT to describe our type is so that we can structurally match on them later, treat them semantically, etc., and string stuffing to abuse our structure has pretty much zero legitimate use-cases other than quickly hacking a printf debug session. However, it truly is often “the simplest solution”.

The main way I deal with this is to be very very careful of putting abusable fields like String, A.Value, Int, SomeException in my data types…just a single field or branch that has an abusable field, AI will find it, and you will feel very stupid for missing it. But hey, the whole point of using properly structured values was to avoid stuffing things into String too, right? The fix for this is a fix that helps human coders, too.

data ErrorEvent = UnknownUser UserName
                | DatabaseErrorCode ErrorCode
                | InvalidJSON ParseError
                | NetworkError NetworkException
                | Canceled CancelationReason
                | ...

Field Abuse

This isn’t just limited to sum types. AI will often stuff data into record value fields that are strings or lists.

data Report = Report
    { reportName :: String
    , reportAuthors :: [String]
    , reportDate :: Day
    , ...
    }

“I need to specify the affiliations of the authors in this report. The simplest solution is to add this to the list of reportAuthors after the authors.”

AI will also stuff sentinel values everywhere: instead of changing the type to take Maybe Day, it might add ModifiedJulianDay 0 for missing days.

There’s also the dual, where the AI will be happy to use existing record fields in overloaded ways instead of adding a new field.

data Targets = Targets
    { fooTarget :: String
    , barTarget :: String
    , -- ...
    }

Let’s say you need to add a new feature or code path that requires a new target for a baz service.

“I need to get a new target…instead of adding a new field to Targets, let’s re-use barTarget. That’s the cleanest approach.”

It will optimize keeping existing types instead of extending them to match your domain as your domain expands, especially if those existing types cross a library boundary.

I believe there are three heuristics at play that drive this behavior.

1. The heuristic to avoid extra risk in modifying upstream types across library boundaries with heavy dependencies. This might be very risky behavior in an untyped language like Python, where each type change might introduce new regressions that are not immediately obvious. So, avoiding upstream type changes avoids potential regression.

2. The heuristic to avoid extra work in modifying upstream types across library boundaries and compilation units. In Haskell, however, upstream type changes force you to address each possible regression point downstream. So changing an upstream type will require you to address every place it is used, which can be time-consuming and avoided by LLMs, especially if compilation is expensive, or multiple new typeclass instances might need to be added.

   I have literally seen LLMs say “Adding this field would require adding typeclass instances on several other types, which would be a huge change. The simplest approach would be…”

   However, updates require work by design: API changes should require lots of thought, and the compiler enforcing that is the whole point.

   The big irony here is that these updates and changes are largely mechanical in nature, and are exactly the boilerplatey task that LLMs are optimally good for. These are the reasonable one-shots. So it’s kind of funny when you let an agentic coder take on a “self-directed” mode, it refuses to use “itself” in the way that a real human would for these smaller tasks.

3. The heuristic to avoid extra risk in touching data types that might already be used in prod code or databases. Config files that might have to be updated to new schemas, inter-op with existing services that might not be easily deployed in sync, working with data at rest…all of these are real risks you have to manage when changing data types. In practice, a lot of our type changes will not be relevant to any of these concerns, but it is understandable that the LLM would develop an instinct to blanket-avoid them.

These are also the types of failure modes that are most difficult to catch during code review. Diff views will analyze that code has changed, so code that didn’t change is especially difficult for human monkey brains to spot, with no green or red bright highlighting. You must be especially vigilant to catch code that did not change but should have.

Resisting New Types

Sometimes AI does modify existing sum types, but doesn’t quite adapt existing code correctly.

For example, if your domain has a specific meaningful universe:

data State = Alaska | Arkansas | Arizona | ...

processState :: State -> IO ()

We might want to start supporting countries alongside US states. An LLM might recognize that the domain needs to be expanded, but it might expand it flatly:

data Region = Canada | Mexico | Alaska | Arkansas | Arizona

processRegion :: Region -> IO ()
processRegion = \case
    Canada -> ...
    Mexico -> ...
    st -> processState st

processState :: Region -> IO ()
processState = \case
    Canada -> pure ()
    Mexico -> pure ()
    Alaska -> ... -- actual logic

The real solution would be to have all your pattern matches strictly reduce the space of what they cover (and be monotonically decreasing), and to be suspicious of “ignored case matches” that have dummy values like pure ():

data Region = Canada | Mexico | USState State
data State = Alaska | Arkansas | Arizona | ...

processRegion :: Region -> IO ()
processRegion = \case
    Canada -> ...
    Mexico -> ...
    USState st -> processState st

processState :: State -> IO ()
processState = \case
    Alaska -> ... -- actual logic

All things that are code smells in normal human code (not necessarily wrong, but invite further scrutiny), but are maybe amplified in the age of LLMs because of a mis-tuned heuristic on not defining new types and instead trying to re-use or abuse existing types.

So, if there is some pressure against modifying types, there might be an even greater pressure against adding types.

Meditations

None of these behaviors are blanket-wrong, but they usually signal that the LLM is under stress or duress and attempting to find ways to take the easy or “low-effort” path over the correct one. All of them are worth human intervention and guidance as soon as possible, at least until the day where P(legitimate | attempted) approaches 1.

Will there be a day when LLMs can generate the correct types to match the domain, and resist their tendency to “defensive-program” their way into correctness? Maybe. But I have rarely ever had Opus 4.8 crank out a sufficient domain model for any non-trivial product. And, when I do reach a plan I find sufficient, a few compactions later and all of the original motivations seem to get washed out.

There might be a way to uber-prompt all of these issues away, but I feel that effectively using LLMs isn’t necessarily something you can address from the prompt level: it’s something that demands constant vigilance and care. I’ve found automated hooks (like forbidding warning-disabling, detecting hlint bypassing…ask claude for help writing these lol) also help me flag areas that need attention immediately.

Who knows, maybe all of these things will be solved within a year. But I still think of software development as something that’s worth scrutinizing for anything of importance. As failure modes like these become less common…the long tail of correctness, I predict, will remain long.

Anyway, that’s it for this topic, but if I find the time I’ll continue on with some other topics I’ve been thinking about during my Haskell and LLM adventures:

1. Effective ways to plan out Haskell code and approaches, ways to encourage the best possible types
2. Structuring your libraries mechanically for the best build-and-test rapid development cycle
3. Starting and maintaining full “vibe-coded” Haskell projects (when correctness is not critical) and the advantages over untyped vibes.

Let me know if there are any you’d like to see first, or if there are other aspects of Haskell LLM usage you might like me to address!

And hey, since we’re here, why not train your agentic friend to take these ideas to heart?

<https://blog.jle.im/entry/llms-and-haskell-1-constraint-evading-behavior.html>

Read this post and make me a Claude Code skill that reviews a Haskell diff for
the constraint-evading compromises it describes: suppressed warnings, string
and field stuffing, and weakened types that differ from any recorded plans.
Some of these hide in code that did not change but should have, so the skill
should start from the functions that changed and evaluate how they use or abuse
the types involved, but also spot type changes that look suspicious.