What It Actually Takes to Become an AI Product Architect — Part 1: Discovery to Specification

How experienced engineers can lead AI initiatives end-to-end

Many companies don’t have a job title for the person who owns AI features end-to-end. They have ML engineers, data scientists, product managers. But the role that bridges all of them? That’s often unnamed.

AI Product Architect is a term I’m using to describe this role. It’s not a standard industry title. It’s how I make sense of a set of responsibilities that don’t fit neatly into existing job categories.

A software architect designs systems. An AI Product Architect, as I define it, does the same, but for AI features inside products. Same technical leadership. Different technical domain.

If you’re a senior engineer or software architect with years of experience shipping software, you don’t start over to move into this role. You add AI to what you already know: systems thinking, product delivery, technical leadership. The core skills transfer.

In my role as AI Architect at IBM Db2, I technically lead the development of AI features for the product. Db2 has architects for different areas such as compiler, security, and data movement. My responsibility is planning, designing, and implementing AI features in Db2 products.

That’s what an AI Product Architect does, at least in my experience: technically owns AI features end-to-end within a larger product.

What the Role Looks Like in Practice

In 2025, I was technically responsible for three major AI initiatives in Db2. I owned and architected the end-to-end implementation of each, from concept to release: planning, designing, implementing, testing, releasing, promoting, and adopting.

The three projects:

1. Native vector data type: introducing vector storage and similarity search functions inside Db2

2. Tooling integration: extending existing Db2 utilities to recognize and work with the new vector type

3. LLM API integration: connecting external language model APIs to Db2, making them callable via SQL functions

These weren’t research experiments. Each one shipped in a product release to enterprise customers running real workloads.

Each project followed the same pattern. Let me walk you through what that looks like.

Phase 1: Proposal and Validation

Every initiative starts with identifying a market opportunity. A problem worth solving. That problem may translate into multiple functions, new capabilities, new use cases for customers. But it starts with one question: what’s worth building?

Market Research

Analyst reports give you the landscape: Gartner Magic Quadrant, Forrester Wave, industry blogs from major conferences. Competitor announcements show you where the market is moving.

But reports tell you what’s popular, not what’s valuable. That requires talking to customers.

When we proposed vector support for Db2, I studied how AI practitioners were using vector types and similarity search: what problems these capabilities solved, what use cases they enabled.

Customer Validation

I spoke with customer representatives, internal data scientists, and machine learning practitioners. Many of them work directly with real customers. I also reached out to actual customers already building these kinds of use cases.

The goal isn’t to confirm your idea is good. It’s to find out where it’s wrong. What are customers already doing? What’s broken? Would they pay for this?

This back-and-forth sharpens the idea. You validate assumptions, crystallize the concept into specific problem categories, and identify where real demand exists.

Quick Prototyping

Before presenting to executives, when possible, I build low-fidelity prototypes using existing features. Not performant. Not production-ready. But tangible.

For the vector proposal, I implemented vector storage and similarity search using existing Db2 data types and user-defined functions. It was slow, but it demonstrated the experience and capability we were proposing.

A working prototype changes the conversation. Stakeholders stop debating whether it’s possible and start debating whether it’s worth it.

Building the Proposal

The initial proposal is still high level: a suggested set of new features, the problems they solve, evidence of customer interest, and the industry outlook.

I never go straight to executives. First, I validate with peer architects, including some more senior than me, and make them co-authors on the proposal. We go back and forth, refine the thinking, pressure-test the idea. Then we present to executives together.

Phase 2: Solution Externals

Once the proposal has stakeholder support, the next step is writing solution externals.

In software engineering terms, this is a software requirements specification. But it reads like a user manual, written before any code exists. You define how real users will interact with the proposed features: the interface, the syntax, the experience.

The goal is precision without code. By the time engineering starts, everyone (architects, developers, stakeholders) should have the same understanding of what the feature will do.

For each feature, I document:

• What it does: a clear description of the function or capability

• How it looks: syntax, parameters, inputs, outputs

• Processing: what the function does with those inputs to produce the output

• Restrictions: what this feature will not cover in this phase

• User scenarios: sample use cases with realistic values

For example, when I wrote the solution external for the vector functions in Db2, I documented each function. For vector distance calculations, that meant defining parameters, expected inputs, outputs, processing logic, and restrictions. Everything was documented textually and visually, without writing a single line of implementation code. The visual above shows what this looks like in practice.

This solution external becomes the foundation for the actual user documentation later.

This document saves months of rework. Ambiguity in requirements becomes bugs in production. The solution external eliminates ambiguity before a single line of code is written.

At this point, the feature exists on paper. Everyone agrees on what we’re building.

Now comes the hard part: making it real.

Next post: Design and Implementation.