We often end up devoting too much time perfecting LLM prompts or over-relying on LLMs to autonomously run the workflow for us end-to-end. In this article, we propose a different, more effective approach for solving problems by engaging LLMs to build deterministic logic, which can then be used in a repeatable manner, reliably.
We’ve had access to powerful AI agents for over a year now, and you’ll notice a recurrent pattern: for every complex problem, the chatbot will generate code to solve it. However, the challenge here is that each time you run a prompt, the chatbot regenerates the code, which compounds errors rather than compounding success. Over time, the noise exceeds the signal, and you end up with a result that might look good on the surface but fails miserably.
Our key learning from our experiments, as you’ll see in this article, is to take a pause with our LLM conversations and use them to turn the code into a tool. This way, you learn about the problem through the LLM and end up capturing what you learn into deterministic code rather than using the non-deterministic LLM processes to tackle the problem.
Our experiments failed because we combined exploration and execution
We wanted to use Claude to build a sales Business Development Report (BDR) that could track how reps move prospects across the sales funnel, extracting data from HubSpot (meetings, contacts, lead statuses, etc.). When we prompted Claude, it did generate a report artefact.
However, we noticed data misinterpretations as some HubSpot fields did not match how the business defined those metrics, some activity types had not been accurately captured, and while phone calls were visible in the BDR, email and LinkedIn records were untrustworthy.
Now you can’t exactly classify these as hallucinations because AI is using data correctly, but it’s the available data that is not adequate to answer some questions in the original request.
A similar problem surfaced when another team used an AI agent to manage their HubSpot CMS to draft copy, generate pages, and publish updates through the connector. The agent wouldn’t publish some pages, and some changes would appear in the Design Manager but fail to show up on the live site. All this while the API call succeeded, so the agent had no clue any of this was happening.
Notice the parallel in these two examples, where the agent was being asked to handle complex tasks and do too many jobs at once, from understanding the underlying structure, formulating logic, calling the right APIs and in the right sequence, and verifying the final result for correctness. In both cases, there was no separation between exploration and execution, no validation step, and no stable ground truth to check against.
How we used Claude to build a tool with deterministic logic
We fixed both the BDR report and CMS issues by not handing out everything to the LLM and letting it run wild. Instead, we asked it first to build a proper, repeatable, deterministic tool, and then instructed it to use it to solve the problem.
For the CMS problem, we built a dedicated sync tool to extract the CMS workflow, which could pull content and themes out of HubSpot, let us work on them locally, validate them, and finally push them back in a controlled way.
Each command did just one thing and reported clearly what happened. So, we effectively shifted the LLM’s role. Instead of operating the CMS autonomously through a connector, it was reading the existing content corpus, proposing changes, drafting copy, and reasoning about structure. Our tool handled the parts that needed to be deterministic: pulling, validating, diffing, and pushing.
Earlier, the agent was guessing its way through HubSpot’s quirks on every run. Now, it had a defined path to follow, with checkpoints at each stage.
We followed this same approach for our BDR Claude-HubSpot integration project:
Step-1: We wrote the report spec in English: what sales leadership needed to know, which funnel stages mattered, and which decisions the report was supposed to support.
Step-2: We used AI as an API exploration partner: search the HubSpot objects, inspect example responses, test likely fields, and record which sources supported each metric. A lot of asking, checking, reading API responses, and adjusting vis-à-vis a single query against a reporting table.
Step-3: We made every useful discovery a part of the working documentation. We noted why certain fields were useful and which ones were only rough stand-ins for what we wanted.
Step-4: We used the LLM to build a small data-gathering tool that pulled the same HubSpot sources, applied the same rules, and returned the same report inputs each run.
How to build a tool using LLMs to solve the problem at hand
The first question to ask yourself is which parts of the problem should actually become a tool. We’ve found that a useful way to identify them is to look for tasks that produce the same kind of output repeatedly, but require considerable exploration before you can determine the correct process.
Working with APIs is a good example, because APIs appear much simpler from the outside than they are in practice. An LLM can help you read the documentation, inspect responses, test endpoints, understand relationships between objects, and write the first code version. However, you do not want the model interpreting these afresh every time the workflow runs.
In our HubSpot CMS example, a website page cannot be interpreted as simply a block of text with a URL attached to it. An agent editing a page needs to understand which template the page uses, which areas of that template are editable, how a content block is embedded, whether a module is referenced using a standard module tag or a drag-and-drop tag, and what other page data must be preserved when making the change.
Forms create another layer of relationships. A form might be inserted through HubSpot’s default form module, exposed as a configurable field within a custom module, or added using an embed script. Each method behaves differently and gives content editors different levels of control. The selected form is also identified through values such as the portal ID, form ID, and account region, meaning that generating something which merely resembles valid embed code is not sufficient.
Pagination is a less visible example of the same problem: an API request may return a valid list of pages or records that contain only the first set of results. HubSpot’s page APIs support paginated responses, so a workflow that does not consistently follow the paging information can complete successfully while quietly ignoring a substantial portion of the site.
The main challenge is that an LLM operating directly through an API has to remember all of them, apply them in the correct sequence, and notice when one of its assumptions is wrong. A successful API response does not confirm that the right template was used, that all the pages were retrieved, that the form points to the correct account, or that an update preserved every nested content block.
This is precisely why this kind of knowledge is best built inside a dedicated tool.
Holding LLMs accountable
LLMs can help discover and implement rules you’ve set, but the finished tool should enforce them without requiring the model to reason through them again during every run. This also gives you a way to hold the LLM accountable.
Instead of asking Claude to “update the website” and then trusting that it has interpreted the request correctly, you give it a small set of commands with explicit behaviour. It can pull the current site, edit the content locally, run validation, review the diff, and request a publish operation. If the validation fails, the tool stops; if a template reference is missing, the tool reports it; if the API returns only part of the expected dataset, the tool knows to retrieve the next page.
So the agent is still useful, but it has to play by the rules established by the system.
It is equally important that building a tool should complement instead of removing the people who currently work inside the CMS. HubSpot itself recommends using its content editor for modifying page content rather than relying exclusively on the page APIs. Content teams can continue using the interface they understand, while the tool can handle the mechanical work around synchronisation, validation, comparison, and deployment. Before making a change, it pulls the latest state from HubSpot. Before publishing, it checks that the local changes do not overwrite something edited by another person. After publishing, it verifies the resulting state.
We recommend that the tool be designed around the actual workflow rather than only around the API, as only you know how your team wants writers, designers, developers, and LLMs to work together.
Deterministic logic still cannot capture taste or intent
There is, however, a boundary to what should be turned into deterministic code. Tools are useful for capturing rules that should behave consistently but are less suited to questions involving taste, intent, judgment, or context.
You can build a tool to verify that a page contains a title, but it cannot decide whether that title makes the right promise to the reader. Similarly, it can confirm that a content block is present, but it cannot determine whether the page is saying the right thing. You can again use an LLM for this purpose.
The team’s intent needs to be captured in documents: what the product is meant to do, which audience it serves, which principles should guide decisions, which constraints must be respected, and how the team will judge whether a result is correct. Without this in place, the model may follow the technical rules perfectly and still produce something that has drifted away from what people actually wanted.
HELIX provides one possible framework for doing this, as it’s a document discipline for teams building software with agents, where project intent, requirements, constraints, decisions, architecture, test plans, and implementation plans are maintained as a connected set of artefacts. The idea is to give the agent a form of shared project memory that can be checked and updated as the work changes.
So you basically engineer three separate layers working together:
- The documents capture intent and judgment.
- The LLM helps explore the problem, interpret those documents, and build or modify the solution.
- The tool captures the repeatable execution path and enforces the rules every time it runs.
That is a more useful way to think about building with LLMs. As you can see, there’s no need to choose between writing everything manually and allowing an autonomous agent to operate without limits. You can use the model to investigate an unfamiliar system, work through ambiguities, generate and critique code, and help you identify the correct process. You then preserve what you have learned in documentation and deterministic tools.
The conversation is where you work out the problem. The tool houses the solution. That’s how you separate exploration and execution.
Deterministic logic, context, and AI reliability
It became clear to us that access to APIs and connectors alone cannot help an LLM interpret underlying data correctly. No AI model has a built-in understanding of metric ownership, source quality, cache behavior, or publication states. These are all things that have to be given to it explicitly. When you think about it, it’s more than context, which is just the operating knowledge that makes the output safe to use in production.
The official documentation helped us understand HubSpot’s API, endpoints, objects, fields, limits, and examples, but it did not provide the architecture of the local usage layer: the decisions the business made, the fields our team actually trusts, the sequences that actually work in our specific portal. That’s why we feel documentation is really a starting point rather than the final word. The system also needs an explicit structure: definitions, source mappings, caveats, validation checks, and examples of correct output.
It’s like the modern version of GIGO in the AI era. If the AI sees only part of the system, it will build from that partial view, and the gaps will be invisible until they matter. As engineers, we need to ask LLMs to build repeatable and inspectable logic that can classify, acquire, and interpret data, check it, validate the results, and generate the right outputs.
Innsigle seal: human-authored by Entropy Exchange
- Composition
- human-authored
- Issuer
- Entropy Exchange
entropy-exchange - Signing key
ed25519:1557…d034d742e92a- Signed
- 2026-08-25T14:05:48Z
- Content digest
sha256:4db12357…f53917d5
Raw attestation JSON
{
"payload": {
"innsigle": "1",
"type": "https://innsigle.dev/claim/colophon/v1",
"issued_at": "2026-08-25T14:05:43Z",
"issuer": {
"id": "entropy-exchange",
"name": "Entropy Exchange",
"key_id": "ed25519:1557684e5374d7409073d034d742e92a",
"key_url": "https://entropy-exchange.synaptiq.ai/.well-known/innsigle/keys.json"
},
"subjects": [
{
"uri": "https://entropy-exchange.synaptiq.ai/posts/conversations-as-code/index.qmd",
"digest": {
"alg": "sha256",
"value": "4db12357b0e4207035ae04d3df8d3aa5309009eb93d21ae913f334fef53917d5"
}
}
],
"colophon": {
"schema_version": "1",
"composition": "human-authored",
"ingredients": [
{
"kind": "human",
"name": "operator",
"role": "author"
}
],
"notes": null
}
},
"payload_encoding": "json",
"signatures": [
{
"key_id": "ed25519:1557684e5374d7409073d034d742e92a",
"alg": "ed25519",
"sig": "VCtvU0xNqInCuDYkdd9DBDxFdHhpZQ0nu1IK_PYSoihqK5BlqLVznet2dbxlnCrdS8nWN95vj6JBlSlE4opLCw",
"signed_at": "2026-08-25T14:05:48Z"
}
]
}