From @Transactional to AI Orchestration: The Pattern You Already Know

Spring AI Orchestration introduces a declarative model for working with large-language models — you define what you want, and the framework handles how it happens.

Just as @Transactional simplified database transactions, Spring AI Orchestration abstracts the complexity of prompt handling, model coordination, and response flow behind a familiar Spring-style programming model.

Spring’s @Transactional annotation (Spring Framework 1.2) established a pattern that changed how we handle infrastructure concerns: declare intent; let the framework handle the mechanics. Before that, transaction management meant explicit PlatformTransactionManager calls, manual commit/rollback, and connection lifecycle juggling.

Spring AI applies the same idea to large‑language‑model (LLM) integration. The mental model you already use—declarative programming, infrastructure abstraction, and composable operations—maps cleanly from database transactions to AI orchestration.

Baseline config (OpenAI)

spring:
  ai:
    openai:
      api-key: ${OPENAI_API_KEY}
      chat:
        options:
          model: gpt-4o
          temperature: 0.7

This is a solid production starting point. Defaults live in configuration; business code stays clean.

The Declarative Pattern: Separating Intent from Implementation

Manual transaction handling (pre‑annotation) mixed infrastructure with domain logic:

public void transferFunds(Long fromId, Long toId, BigDecimal amount) {
    DefaultTransactionDefinition def = new DefaultTransactionDefinition();
    TransactionStatus status = transactionManager.getTransaction(def);
    try {
        Account from = accountRepository.findById(fromId).orElseThrow();
        Account to = accountRepository.findById(toId).orElseThrow();

        from.debit(amount);
        to.credit(amount);

        accountRepository.save(from);
        accountRepository.save(to);

        transactionManager.commit(status);
    } catch (Exception e) {
        transactionManager.rollback(status);
        throw new FundTransferException("Transfer failed", e);
    }
}

With Spring, you keep business logic in the method body and infrastructure in metadata:

@Transactional
public void transferFunds(Long fromId, Long toId, BigDecimal amount) {
    Account from = accountRepository.findById(fromId)
        .orElseThrow(() -> new EntityNotFoundException("From account not found"));
    Account to = accountRepository.findById(toId)
        .orElseThrow(() -> new EntityNotFoundException("To account not found"));

    if (from.getBalance().compareTo(amount) < 0) {
        throw new InsufficientBalanceException();
    }

    from.debit(amount);
    to.credit(amount);
}

And when contention matters, you push concurrency semantics to the repository:

public interface AccountRepository extends JpaRepository<Account, Long> {

    @Lock(LockModeType.PESSIMISTIC_WRITE)
    @Query("select a from Account a where a.id = :id")
    Optional<Account> findByIdForUpdate(@Param("id") Long id);
}

Keep @Transactional simple, let the DB default isolation do its job, and use entity‑level locking only where the domain requires it.

The Same Pattern in AI Integration

A raw, hand‑rolled OpenAI call couples HTTP details, JSON shapes, and error handling to your feature:

@Service
public class ContentAnalyzerRaw {
    private final RestClient restClient;
    private final ObjectMapper objectMapper;
    private final String apiKey;

    public ContentAnalyzerRaw(
            @Value("${openai.api.key}") String apiKey,
            RestClient.Builder restClientBuilder,
            ObjectMapper objectMapper) {
        this.apiKey = apiKey;
        this.restClient = restClientBuilder
                .baseUrl("https://api.openai.com/v1")
                .build();
        this.objectMapper = objectMapper;
    }

    public String analyzeContent(String content) {
        Map<String, Object> body = Map.of(
            "model", "gpt-4o",
            "messages", List.of(Map.of("role", "user", "content", content)),
            "temperature", 0.7,
            "max_tokens", 500
        );

        String response = restClient.post()
            .uri("/chat/completions")
            .header("Authorization", "Bearer " + apiKey)
            .contentType(MediaType.APPLICATION_JSON)
            .body(body)
            .retrieve()
            .onStatus(s -> s.value() == 429, (req, res) -> {
                throw new IllegalStateException("Rate limited");
            })
            .body(String.class);

        try {
            JsonNode root = objectMapper.readTree(response);
            return root.path("choices").get(0).path("message").path("content").asText();
        } catch (IOException e) {
            throw new IllegalStateException("Failed to parse response", e);
        }
    }
}

With Spring AI, the intent/implementation boundary is clear. Your code depends on a portable ChatClient:

@Service
public class ContentAnalyzer {
    private final ChatClient chat;

    public ContentAnalyzer(ChatClient chat) {
        this.chat = chat; // injected portable client
    }

    public String analyzeContent(String content) {
        return chat.prompt()
                   .user(content)
                   .call()
                   .content();
    }
}

The provider, model, and options are configured once (YAML above). Business logic remains unchanged.

Pattern Mapping: Transactions and AI Operations

Default Configuration

Transactions: sensible defaults (@Transactional without tuning)

AI: sensible defaults (model, temperature in YAML)

public String chat(String message) {
    return chat.prompt()
              .user(message)
              .call()
              .content();
}

Explicit Configuration

Transactions: override isolation, timeout, readOnly only when needed

AI: tune per call via ChatOptions

public String generateReport(String data) {
    return chat.prompt()
              .user(data)
              .options(ChatOptions.builder()
                      .model("gpt-4o")
                      .temperature(0.2)
                      .maxTokens(1000)
                      .build())
              .call()
              .content();
}

Composition

Transactions: a single unit of work spans multiple repos

AI: RAG = vector search + prompt construction + model call

@Service
public class KnowledgeService {
    private final ChatClient chat;
    private final VectorStore vectorStore;

    public KnowledgeService(ChatClient chat, VectorStore vectorStore) {
        this.chat = chat;
        this.vectorStore = vectorStore;
    }

    public String answer(String question) {
        List<Document> docs = vectorStore.similaritySearch(
            SearchRequest.query(question).withTopK(3)
        );

        String context = docs.stream()
                             .map(Document::getContent)
                             .collect(Collectors.joining("\n\n"));

        return chat.prompt()
                   .user(u -> u.text("""
                       Use only the following context to answer the question.
                       If the answer isn't in the context, say so.

                       Context:
                       {context}

                       Question: {question}
                       """).param("context", context)
                         .param("question", question))
                   .call()
                   .content();
    }
}

Error Handling

In practice, teams handle rate limits/timeouts with Resilience4j and keep a fallback path:

@Service
public class ContentService {
    private final ChatClient primary;
    private final ChatClient fallback;

    public ContentService(ChatClient primary, ChatClient fallback) {
        this.primary = primary;
        this.fallback = fallback;
    }

    public String generate(String prompt) {
        try {
            return primary.prompt().user(prompt).call().content();
        } catch (RuntimeException ex) {
            // e.g., rate limit/timeouts—retry policy or fallback model
            return fallback.prompt().user(prompt).call().content();
        }
    }
}

Production Patterns: Resilience and Observability

Resilience4j + Micrometer is the de‑facto pairing in Spring Boot 3.x apps.

Gradle (BOM + starters)

dependencies {
    implementation platform("org.springframework.ai:spring-ai-bom:${SPRING_AI_VERSION}") // e.g., 1.0.x GA
    implementation "org.springframework.ai:spring-ai-starter-model-openai"
    implementation "org.springframework.boot:spring-boot-starter-actuator"
    implementation "io.github.resilience4j:resilience4j-spring-boot3"
    implementation "io.micrometer:micrometer-registry-prometheus"
}

Service with breaker + metrics

@Service
public class ResilientContentService {
    private final ChatClient chat;
    private final CircuitBreaker breaker;
    private final MeterRegistry metrics;

    public ResilientContentService(ChatClient chat,
                                   CircuitBreakerRegistry registry,
                                   MeterRegistry metrics) {
        this.chat = chat;
        this.breaker = registry.circuitBreaker("aiProvider");
        this.metrics = metrics;
    }

    public String analyze(String content) {
        Timer.Sample sample = Timer.start(metrics);
        try {
            String out = breaker.executeSupplier(() ->
                chat.prompt().user(content).call().content()
            );
            sample.stop(Timer.builder("ai.analysis.duration")
                             .tag("outcome", "success")
                             .register(metrics));
            return out;
        } catch (CallNotPermittedException e) {
            sample.stop(Timer.builder("ai.analysis.duration")
                             .tag("outcome", "circuit_open")
                             .register(metrics));
            return "Temporarily unavailable. Please retry.";
        } catch (RuntimeException e) {
            sample.stop(Timer.builder("ai.analysis.duration")
                             .tag("outcome", "error")
                             .register(metrics));
            throw e;
        }
    }
}

Multi‑Step AI Workflows

As with transactional workflows, you compose steps without orchestration glue:

@Service
public class ResearchService {
    private final ChatClient chat;
    private final VectorStore vectorStore;

    public ResearchService(ChatClient chat, VectorStore vectorStore) {
        this.chat = chat;
        this.vectorStore = vectorStore;
    }

    public ResearchReport generateReport(String topic) {
        List<String> questions = generateQuestions(topic);

        Map<String, List<Document>> research = questions.stream()
            .collect(Collectors.toMap(
                q -> q,
                q -> vectorStore.similaritySearch(
                        SearchRequest.query(q).withTopK(3))
            ));

        String synthesis = synthesizeFindings(research);
        String summary = generateSummary(synthesis);

        return new ResearchReport(topic, questions, synthesis, summary);
    }

    private List<String> generateQuestions(String topic) {
        String response = chat.prompt()
            .user("""
                Generate 5 specific, answerable research questions about: {topic}
                One question per line, no numbering.
                """)
            .user(u -> u.param("topic", topic))
            .call()
            .content();

        return Arrays.stream(response.split("\n"))
                     .map(String::trim)
                     .filter(s -> !s.isEmpty())
                     .collect(Collectors.toList());
    }

    private String synthesizeFindings(Map<String, List<Document>> research) {
        String context = research.entrySet().stream()
            .map(e -> "Question: " + e.getKey() + "\n\nSources:\n" +
                      e.getValue().stream()
                               .map(Document::getContent)
                               .collect(Collectors.joining("\n\n")))
            .collect(Collectors.joining("\n\n---\n\n"));

        return chat.prompt()
            .user("""
                Synthesize the following research into a concise analysis.
                Cite specific sources where relevant.

                {context}
                """)
            .user(u -> u.param("context", context))
            .options(ChatOptions.builder()
                    .temperature(0.3)
                    .maxTokens(1500)
                    .build())
            .call()
            .content();
    }

    private String generateSummary(String synthesis) {
        return chat.prompt()
            .user("""
                Create a 3–4 sentence executive summary of:

                {synthesis}
                """)
            .user(u -> u.param("synthesis", synthesis))
            .options(ChatOptions.builder()
                    .temperature(0.2)
                    .maxTokens(200)
                    .build())
            .call()
            .content();
    }
}

Function Calling: LLM‑Orchestrated Operations

Function calling inverts control: the LLM invokes your code. Spring AI makes this safe and declarative via @Tool.

@Component
public class OrderTools {

    @Tool(description = "Check if a product is in stock")
    public boolean checkInventory(@ToolParam(description = "Product SKU") String sku) {
        // domain/service call
        return true;
    }

    @Tool(description = "Get the current price of a product")
    public BigDecimal getPrice(@ToolParam(description = "Product SKU") String sku) {
        return new BigDecimal("19.99");
    }

    @Tool(description = "Place an order for a product")
    public String placeOrder(@ToolParam(description = "Product SKU") String sku,
                             @ToolParam(description = "Quantity") int quantity) {
        return "Order placed for " + quantity + " of " + sku;
    }
}

@Configuration
public class AiConfig {
    @Bean
    public ChatClient chatClient(ChatModel model, OrderTools tools) {
        return ChatClient.builder(model)
                         .defaultTools(tools)
                         .build();
    }
}

@Service
public class OrderAssistant {
    private final ChatClient chat;

    public OrderAssistant(ChatClient chat) {
        this.chat = chat;
    }

    public String handleRequest(String customerRequest) {
        return chat.prompt()
                   .system("You are a helpful order assistant.")
                   .user(customerRequest)
                   .call()
                   .content();
    }
}

Testing AI‑Integrated Applications

Unit tests: mock ChatClient and assert domain behavior—fast and reliable.

@ExtendWith(MockitoExtension.class)
class OrderAssistantTest {

    @Mock
    private ChatClient chat;

    @InjectMocks
    private OrderAssistant assistant;

    @Test
    void shouldRespond() {
        // Wrap ChatClient behind a small port or mock the fluent chain as needed
        // Assertions focus on your domain logic, not provider calls.
    }
}

Integration tests: use Testcontainers and WireMock to stub provider behavior while exercising your Spring AI wiring end‑to‑end.

@SpringBootTest
@Testcontainers
class ContentAnalyzerRawIT {

    @Container
    static WireMockContainer wiremock =
        new WireMockContainer("wiremock/wiremock:3.3.1")
            .withMappingFromResource("openai-mappings.json");

    @DynamicPropertySource
    static void props(DynamicPropertyRegistry r) {
        r.add("openai.api.key", () -> "test-key");
        // If the raw client uses a baseUrl property, point it to wiremock.getBaseUrl().
    }

    @Autowired
    private ContentAnalyzerRaw service;

    @Test
    void shouldAnalyzeContent() {
        String result = service.analyzeContent("Test content");
        assertThat(result).contains("test analysis response");
    }
}

Conclusion: Pattern Recognition Over New Paradigms

Spring AI doesn’t ask you to learn a new paradigm. It applies the @Transactional pattern you already know to AI orchestration:

• Declarative configuration over imperative plumbing
• Infrastructure abstraction to avoid vendor lock‑in
• Composable operations from simple calls to multi‑step workflows
• Production‑ready defaults with clear override points

If you’ve built production Spring Boot apps with transactional databases, you already have the mental model to build production AI‑integrated apps with Spring AI. Configure the model once, keep business code clean, and let the framework handle the rest.