Untangling the Hairball: What to Do When You Can't Feed the LLM Your Code

What can you do if you're not allowed to use LLMs at work — or you can use them, but you can't feed them CUI (Controlled Unclassified Information), which includes your code?

You can build better harnesses.

In essence, questions and metaphors can be generic even when the codebase isn't. "How do I boot a Spring context inside JShell?" "How do I render the SQL a Hibernate Criteria would run, without executing it?" "How do I structure a test that proves two implementations of the same function return identical results?" There is no CUI in any of those questions. They're the kinds of generic questions we used to ask on StackOverflow. You can ask for a generic framework, then plug in your contextual specifics.

For example —

One of my clients has a Java hairball: a two-decade-old application whose behavior is unspecified because it is the accumulation of features and bugfixes applied one on top of another. Fix one issue and another appears somewhere else. It had no tests, because it was written in the pre-testing era, and every change carried the low-grade dread of not knowing what you'd broken.

Two practices changed that, and an LLM helped build both, without ever providing it with the client's code.

First, the standing rule: I never ship new code without tests. My client's application used to have none. It now has a suite wired into CI that fails if behavior drifts. The valuable pattern here is the parity harness. When you refactor something critical, you don't just test the new code: you run old and new side by side and assert the results are deep-equal. That harness design is a generic problem, and generic problems are exactly what you can take to an LLM.

Second, cycle time. Coming from Rails, I missed the console. Type an expression, see the answer. Java's traditional loop is write, compile, deploy to Tomcat, then poke an HTTP endpoint and hope the logs tell you something. So we built one: JShell, plus a startup script that boots the application context and loads every service. Now I run .\bin\console.bat and I'm inside the live application. Want to know what query the search engine will actually generate? Call the criteria builder and see the rendered SQL instantly, with the parameter bindings, no debugger required. Every piece of that console — the context bootstrapping, the transaction wrapper, the SQL-rendering trick that reaches into Hibernate's internals — came from generic questions with generic answers.

The console is the lab where tests are born. I can experiment interactively, against our real system, until I understand its behavior, compare old to new behavior, to give me confidence I haven't broken anything, then harden the experiment into a JUnit test. Confidence is the name of the game, and we have it now.

So, in conclusion: even if you can't share the hairball, you can generate better tools to give you the confidence to untangle it.

I advise engineering teams on AI systems architecture, security boundaries, and high-reliability modernization. See my current advisory availability and engagement models.