AI Engineering was recently named the #1 fastest growing job by LinkedIn, with data science cited as one of the top roles transitioned from. Meanwhile, every data scientist is in the middle of at best an identity crisis and at worst an existential crisis.
I recently switched from data science to AI engineering and have some bad news to report: it is a trap. But it need not be, and my goal in writing this candid article is to show the industry that there is an alternative.
My Path from Data Science to AI Engineering
I’ve been a practicing data scientist for almost 10 years. A year and a half ago, my work as a data scientist hit an inflection point where almost every project my team was working on involved AI agents instead of machine learning. We no longer needed to collect training data to train a custom model for each task. LLMs and agents could solve the tasks we used to solve (like document classification, named entity recognition, etc.) with just a prompt and some tools. We began looking beyond these simple tasks towards more complex ones that only agents can solve, like automating entire workflows or answering arbitrary questions.
I predicted in that moment that within a year the whole team would become AI engineers, whether de facto or formally by title. I completed an AI Engineering bootcamp, then joined a new company as an AI Engineering manager. One of my DS team mates joined me, and my prediction appeared to be true. So far so good.
But my new team was composed of 3 AI engineers who were all former data scientists. The company had no research department and we were asked to operate as engineers, following scrum by the book and shipping features on rigid timelines. Things fell apart quickly before we corrected course. To help you and your team avoid the same fate, below are the frameworks and mental models that I believe need to emerge to successfully organize work in our new AI world.
Composition of ML Teams
Let’s turn back the clock a bit to set some context on the current moment in AI.
This is not my first rodeo at a company that lacked a research function. When I joined Toast in 2017 they had exactly 0 data scientists and approximately 100 software engineers. I had 7 managers over 2.5 years as the org learned how to organize work around machine learning. I adapted and influenced where I could along the way. When I left we had a well-designed team of 6 that was researching and shipping advanced ML models.
That team was split between data scientists (research) and ML engineers (implementation). They worked together to own the full ML lifecycle from lab to production. This has emerged as an industry best practice for ML teams.
The next two companies I worked for had large research teams of 10+ data scientists. But both lacked a concept of ML engineering, which forced the data scientists to either figure out their own deployments or leave good research on the shelf. Because of that gap, as a data scientist I acquired far more engineering experience than most.
But for a team, this is a very inefficient design. The truth is that no one is good at both of these tasks. Scientists optimize for learning what works and what doesn’t via experimentation and prototyping, operating in uncertainty. Engineers optimize for shipping what is already proven to work and focus on building robust, scalable, and secure systems.
I should note that Toast is a wildly successful company, while the other 2 companies I joined sort of…floundered. I won’t go so far as to attribute those company-level successes and failures to these team designs, but the correlation is noteworthy.
On a final note, the only other company I’ve worked for in a data science capacity (before Toast) ran their data science team as a scrum team. We had a PM tell us what experiments to run and had to report results every 2 weeks, ready or not. So we ran the wrong experiments and reported unvalidated results in a public setting. In my opinion, scrum and research are simply incompatible.
Composition of AI Teams
To apply these learnings to the new AI world, let’s first be clear about what’s different today compared to before the advent of LLMs.
- ML is only relevant for a shrinking subset of AI problems, but a modern AI team still needs ML muscle.
- Agent design is the core new skill set that all AI team members must master.
- AI coding agents make prototyping easier than ever.
But something important isn’t different: both AI and ML are fundamentally non-deterministic, and that fact necessitates a scientific process.
So while my prediction about my team of data scientists becoming AI engineers turned out true, I was wrong to want this. The truth is that the distinction between scientist and engineer has nothing to do with AI, and that science is more important than ever for agents.
So how do we fix it? Just like the optimal ML team design is composed of data scientists and ML engineers, the optimal AI team design must be composed of both AI Scientists and AI Engineers.
Introducing the AI Scientist
Let’s define what an AI Scientist is.
AI Scientist: possesses all the skills of a data scientist and adds proficiency in agent design and full-stack app prototyping
And let’s compare that to the definition of an AI Engineer.
AI Engineer: possesses all the skills of a software engineer and adds proficiency in agent design.
Note that agent design is the core new skill that both of these new roles add on top of traditional roles. (Also note that, in my opinion, an AI Engineer is not an evolution of an ML engineer but rather of a software engineer. An ML engineer could become an AI Engineer by acquiring skills in agent design, but an AI Engineer need not possess skills in ML infrastructure and deployment.)
Comparing Data Science to Software Engineering
From the definitions above for AI Scientist and AI Engineer it’s clear that what is shared is agent design and that what is different is which skill set is the foundation: data science and software engineering, respectively. So to truly understand the difference between an AI Scientist and an AI Engineer, we must understand the difference between a data scientist and a software engineer.
A data scientist operates in a research setting. Their goal is to explore, experiment, and prototype. Their department is an innovation center that operates multiple quarters ahead of the product roadmap. They are an input to the product roadmap, not an output. They operate on quarterly goal cycles with kanban boards to track work. They get more value from weekly hour-long problem solving sessions than daily 15 minute standups. They explore the solution space of open-ended problems, and would never think about shipping their work before evaluating it.
A software engineer operates in an implementation setting. Their goal is to ship features defined by the product team. They execute the roadmap and are measured by velocity and quality. They operate on 2-week sprints and observe scrum ceremonies. They solve well-defined problems that are easy to plan and trivial to validate (and reject or reframe problems that aren’t).
To build a successful AI team, AI scientists must operate as data scientists always have, and AI engineers must operate as software engineers always have.
Risks of AI without Science
Nearly every company has an AI strategy in 2026, including many who did not previously have a research function. To those companies, I want to issue a warning: shipping AI-powered features without a research team will be over-budget and under-loved.
AI is fundamentally non-deterministic and cannot be designed and deployed with the same processes and techniques that are used for traditional, deterministic software. Specifically, AI needs evals. AI will be wrong or unacceptable X% of the time, and you better know what X is before you put it in front of customers so you can minimize it and set realistic expectations. If you wait to learn what X is until it’s in customer hands, you’ve probably wasted a large amount of expensive engineering resources, because the team will likely realize they need to rework some or all of the solution.
A research function is needed to fail early, long before expensive engineering resources are spent and long before you’ve lost customers’ trust.
Conclusions
- The industry needs a new role titled AI Scientist that is an evolution of data scientist, adding agent design as a core skill. Data scientists should feed this role, and the role should operate within a research team as data scientists always have.
- Data scientists should be cautious about AI engineering roles, as they may be a trap that expects them to operate as an engineer not a scientist.
- Companies should not attempt to execute on their AI strategy by staffing AI engineers without also staffing AI scientists. To do so is to risk wasting resources and losing customer trust.
