After 10 years as a DevOps engineer, I kept running into the same question from the inside: what does an AI engineer actually do, day to day? Not the conference-talk version, the actual work. I read explanations, searched around with a few LLMs, watched videos, and Alexey Grigorev's writing was the first thing that made the role click and showed me where infrastructure experience actually transfers.
Two posts from AI Shipping Labs made the decision concrete. Their breakdown of what an AI engineer actually does pulled from 4,894 real job descriptions and found that production and reliability work — deployment (78.3%), infrastructure and platforms (69.2%), evaluation and quality (68.5%) — rivals feature-building as a core responsibility, not an afterthought. It also laid out a complexity ladder: a simple prompt call gets roughly 5x harder once you add retrieval, and 10x harder once you add multi-tool agents. That's a different shape of complexity than I expected, and a lot closer to the operational discipline I already know. Their analysis of the job market, built from nearly 7,000 postings, backed it up: CI/CD shows up in 39.4% of listings, Docker in 26.9%, Kubernetes in 25.3% — infrastructure skills sitting right next to RAG (43.2%) and agentic workflows (42.0%) as the in-demand stack. That overlap is what convinced me this wasn't starting over from zero.
Reading about it only goes so far, so I enrolled in AI Shipping Labs' AI Engineering Buildcamp — a 9-week, hands-on program built around real projects instead of "Hello World" demos. This post is from week 3, after the first two weeks: building a retrieval-augmented generation (RAG) pipeline from scratch.
What actually went into it
A RAG pipeline sounds abstract until you build one. In practice, it broke down into pieces I could reason about individually:
- Calling an LLM through the OpenAI API — the easy part, and the part everyone assumes is "the AI work."
- Pulling real documentation data from GitHub rather than a toy dataset, which immediately surfaces the mess real content has: inconsistent formatting, varying lengths, dead links.
- Indexing the documents for search with
minsearch, a lightweight keyword-search library, before reaching for anything heavier. - Splitting documents into smaller chunks so retrieval returns something focused instead of an entire page.
- Wiring the retrieved chunks into the LLM prompt so the model answers from the actual source material, not just its training data.
- Assembling all of that into one working, end-to-end document assistant.
- Exploring structured output — forcing the model to return a defined schema instead of free-form text, which matters the moment anything downstream needs to parse the response reliably.
Where the DevOps instincts transferred, and where they didn't
The part that felt familiar was the discipline question: how do I know this is actually retrieving the right thing? That's the same question as "how do I know this service is actually healthy," just pointed at a different kind of system. Years of defining SLIs and SLOs made it natural to ask for evidence instead of trusting that a pipeline which runs without errors is a pipeline that's correct.
What didn't transfer as cleanly: a RAG pipeline can run perfectly — no exceptions, no timeouts, every step green — and still retrieve the wrong chunk and hand the model irrelevant context. There's no equivalent to a 5xx error for "technically works, but the answer is wrong." That's a different failure mode than almost anything in traditional infrastructure, and it's the part I'm still building real intuition for.
More of this as the buildcamp continues — next up is evaluation, which is where I expect the DevOps habits to matter most.