If you’re writing a Rails app and you’re anything like us at Cleo, your top-level directory structure probably looks something like this:

├──app/

│  ├──controllers

│  ├──models

│  ├──serializers

│  ├──services

│  ├──views

│  ├──workers

├──test/

   ├──etc.

You’ve got your models for loading objects from the database, controllers to serve endpoints, views and serializers to render data, workers to execute async logic, and then a directory called services that is full of...stuff. Stuff you want to do. Stuff like GivePlusToCleoEmployee or Gpt::GenerateCompletionResponse (both real examples from the current Cleo Rails app). What do they do? Well, obviously, they give Cleo Plus to a company employee, or generate a completion response. 

What’s wrong with that? Sometimes nothing. But sometimes the impulse to reach for a service is your codebase telling you something else.

Hidden abstractions

I was inspired to write this post by my colleague Gavin, who likes to talk about hidden abstractions. A hidden abstraction is a concept that exists in your domain but hasn’t been named or modeled in your code yet, leaving the code an imperfect representation of reality. Abstraction is a tool virtually all professional software engineers are aware of, and part of our craft is knowing when to reach for it.

Let me ground this in a real example from Cleo’s chat system.

Before the hidden abstraction

In an old design of the Cleo app, conversation history looked like a single long stream of messages, similar to a conversation with a friend on WhatsApp, rather than a series of discrete conversations, like those you’d have with ChatGPT. 

To render these conversations, we had three tables:

• bot_requests: a message from a user to Cleo

• bot_responses: a message from Cleo to a user

• chat_messages: a join table between the two, with a polymorphic reference to each, allowing us to load conversations in reverse chronological order and paginate appropriately

In the pre-LLM world, we had little need to understand the context of a given conversation. But after ChatGPT launched and we integrated LLMs into our system, this changed. Conversation history became a vital part of the request payload to an LLM in order to generate a coherent response. 

So what did we do? We reached for a service, of course. 

We wrote Gpt::FetchBaseConversationHistory to handle this logic. It depended on some config, but generally, we’d look at the chat_messages table, step backward through messages, check if each one was within a certain time frame, repeat the process if it was, and break out of the loop if it wasn’t.

This service was slow, inefficient, and full of sneaky edge cases that were difficult to handle. It required inelegant hacks to avoid hitting the database in a loop (like preloading all messages in the last 24 hours) and brittle code that could easily introduce N+1s.

And I undersold the problem when I said we introduced one service for this; we actually introduced four. Alongside Gpt::FetchBaseConversationHistory, we also had:

• Gpt::FetchDynamicConversationHistory

• Gpt::ExtractConversationChatHistory

• Gpt::GenerateChatCompletionMessages

Finding the hidden abstraction

Through working with these services, something became clear: We were missing an internal concept of what constituted a conversation, largely because we’d never needed it before with our infinitely scrolling chat history. Enter ConversationSession.

ConversationSession became a new database model that acted as a parent collection of chat_messages records, written to the appropriate session at runtime. Instead of stumbling through several services to load the appropriate conversation history into memory for each request, working through brittle hacks to find the right message, we now had an actual concept of a conversation. 

Four different services instead became:

conversation_history = conversation_session

  .chat_messages

  .includes(:message)

  .map(&:as_chat_completion_message)

This allowed us to lean on ActiveRecord for the heavy lifting: eager-loading associations and scoping automatically to the right subset of chat_messages records. Moving the constructor method onto the models themselves centralized how we built these objects into the shape an LLM expects. By introducing one abstraction, we were able to kill all four services, because our domain model now better reflected reality.

Finding the right balance

Services can be useful instruments. They express what your software wants to do and help you translate business requirements into application logic. But they’re also blunt instruments. They paper over cracks in your domain modeling, hiding the signals you need to build software that reflects the reality of your system.

Abstractions are hiding everywhere, and part of our craft as engineers is knowing when to abstract software and when not to. It keeps us delivering value to our users, rather than writing elegant code for its own sake. And in growing and evolving that craft, services can be a very useful check, prompting us to ask ourselves, “Is my domain model missing something here?” It might save you another three services down the line.