Improving a platform that already works
As agentic AI transforms user expectations and how people interact with information, more of our clients are turning to us to help them understand modern user needs and deliver modern digital experiences. The Irish Tax Institute (ITI) was no different. The scope for this project came out of strategic stakeholder workshops, customer research, concept ideation and prioritisation, covering improvements to search, visual design, usability and a new AI search capability.
That raised an obvious question: how do you approach improving a ten-year-old piece of software that basically works but needs enhancing?
Our approach came down to three things. First, keep the good stuff. The document ingestion process works well. MarkLogic search works well too, though we needed some new UI to allow more complex searches. We chose not to change the visual display of legislation at all, since we couldn't risk accidentally misrepresenting its meaning or interpretation. A Scala-Play webserver producing HTML, with some jQuery, is still the right tool for the job. And the CI/CD pipeline worked fine. It's a little slow, but overhauling it to speed up deployment would have taken more time than it saved over the life of the project.
Second, remove anything that's genuinely obsolete, in the spirit of Chesterton's fence: don't take something down until you understand why it's there. Once we'd done that thinking, we found several features and implementations that were no longer needed, and retired them.
Third, upgrade the libraries. jQuery stayed, because it's still doing its job, but we moved to a newer version.
Alongside that, we carried out user research and used it to shape the visual design, taking the best of what we learned about how people actually use the site. We also reviewed the code and architecture for security issues and bugs, fixed the small number we found, and added post-deployment TestCafé tests so we'd know immediately if a release had broken something.
Because we wanted to develop and test the new UI without disturbing the live site, and then release everything at a single point in time, we built it behind feature flags. That let us keep shipping the latest code to Test and Live throughout development, with the new features and new UI hidden until we were ready to switch them on.
Early on, we picked eighteen representative sample documents to test against, two from each of nine categories. If the updated site handled those well, we could be confident it would handle everything else.
What matters when you build an AI search
The single most important requirement for this project was trustworthy output. The AI mode needed to interpret ordinary English queries, find the right documents in our database, summarise them, and refer back to the documents behind that summary. We were helping people find documents that already exist in our database, not anything else, so hallucinated references to documents that don't exist were never an option.
Your first design will be wrong. It took us three, maybe four, iterations before we were all happy with it. So we built something quickly, put it in front of people, took the feedback, and went again.
We also had to plan for cost with genuinely limited visibility. Estimating how much usage the new AI mode would get was next to impossible, which meant estimating token usage and cost was next to impossible too. So we built in careful, configurable limits, both to guard against abuse of the feature and to let us monitor usage and get the most benefit we could from what we spent on tokens.
Where it landed
On the day we deployed, customers liked the new system, and we moved quickly to fix the handful of small problems people found. We're planning some extra reporting on AI usage next, and otherwise the site can tick over happily for a while. I enjoyed working with the team at ITI, and I'm glad we've been able to build these improvements on the solid foundations 67 Bricks laid ten years ago.