Getty Images
MLSecOps: Bridging security and AI development processes
MLSecOps ensures AI security by automating controls, fostering collaboration and addressing vulnerabilities early in the development lifecycle.
As security practitioners, we know that securing an organization isn't necessarily a monolithic exercise: We don't -- literally can't -- always focus equally on every part of the business.
This is normal and natural, for many reasons. Sometimes, we have more familiarity in one area versus others -- for example, an operational technology environment, such as industrial control systems, clinical healthcare devices or IP-connected lab equipment -- might be less directly visible. Other times, focus might be purposeful -- for example, when one area has unmitigated risks requiring immediate attention.
Shifts in attention like this aren't necessarily a problem. Instead, the problem arises later, when -- for whatever reason -- portions of the environment don't ever get the attention and focus they need. Unfortunately, this is increasingly common on the engineering side of AI system development.
Specifically, more and more organizations are either training machine learning (ML) models, fine-tuning large language models (LLMs) or integrating AI-enabled agents into workflows. Don't believe me? As many as 75% of organizations expect to adapt, fine-tune or customize their LLMs, according to a study conducted by AI developer Snorkel.
We in security are well behind this curve. Most security teams are well out of the loop with AI model development and ML. As a discipline, we need to pivot. If the data is right and we're heading into a world where a significant majority of organizations might be training or fine-tuning their own models, we need to be prepared to participate and secure those models.
That's where MLSecOps comes in. In a nutshell, MLSecOps attempts to project security onto MLOps the same way that DevSecOps projects security onto DevOps.
Security participation is key, as we see an ever-increasing number of AI-specific attacks and vulnerabilities. To fully prevent them, we need to get up to speed quickly and engage. Just as we had to learn to become full partners in software and application security, we also need to include AI engineering in our programs. While techniques for this are still evolving, emerging work can help us get started.
Examining the role of MLSecOps
MLOps is an emerging framework for the development of ML and AI models. It consists of three iterative and interlocking loops: a design phase, which is the designing the ML-powered application; a model development phase, which includes ML experimentation and development; and an operations phase -- ML operations. Each of these loops includes the ML-specific tasks involved in model creation, such as the following:
- Design. Defining requirements and prioritizing use case.
- Development. Data engineering and model training.
- Operations. Model deployment, feedback and validation.
Two things to note about this. First, not every organization out there is using MLOps. For the purposes of MLSecOps, that's OK. Instead, MLOps just provides a useful, abstract way to look at model development generally. This gives security practitioners inroads for how and where to integrate security controls into abstract ML -- and thereby LLM -- development and support pipelines.
Second -- and again much like DevSecOps -- organizations that embrace MLOps aren't necessarily using it the same way. Security pros have to devise their own ways to integrate security controls and representation into their process. The good news though, is that practitioners who have already extended their security approach into DevOps/DevSecOps already have a roadmap they can follow to implement MLSecOps.
Keep in mind that MLSecOps -- just like DevSecOps -- is about automating and extending security controls into release pipelines and breaking down silos. In other words, making sure security has a role to play in AI and ML engineering. That sounds like a lot -- and can represent significant work and effort -- but essentially comes down to the following three things.
Step 1: Remove silos and build relationships
Establish relationships and lines of communication with the many teams of specialists involved in model development. These include the data scientists, model engineers, product managers, operations specialists and testers, to name just a few, involved in the final outcome. Just like security engineers in a DevSecOps shop work closely with development and operations teams, so too does the security team need to build relationships with the specialists in the AI development pipeline. In most organizations, it means not only discovering who and where this activity is occurring -- not always obvious -- but it also requires educating these folks about why they need security's input at all. It's an outreach and credibility-building effort.
Step 2. Integrate and automate security controls
Work within the existing development process to establish the security measures that help ensure secure delivery. For those of us with experience in DevSecOps, we're accustomed to automating security controls into the release chain by working with build and support teams to decide upon, plan, implement and monitor the appropriate controls. The same is true here. Just like we might implement code scanning in a software context, we can implement model scanning to find malicious serialization or tampering in foundation or open source LLM models slated for fine-tuning. Just like we perform provenance validation on underlying software libraries, we might validate the common open source fine-tuning tools and libraries, such as Unsloth, or common open source software integration tools, such as LangChain.
3. Design measurement and feedback loops
Work with the new partners you've engaged in Step 1 to decide upon -- and establish mechanisms to track -- the key performance metrics germane to security. At a minimum, this involves using data from the tooling established during Step 2. Remember that the point is to inject maturity into the security surrounding the engineering. What that looks like varies significantly from firm to firm. Work with partners to establish the most critical metrics for your organization and its security program.
Making MLSecOps a reality
As you can see, implementing MLSecOps is less a hard-and-fast set of rules than it is a philosophical approach. The MLSecOps and MLOps community pages are fantastic starting points, but ultimately what's important is that we security practitioners begin examining the flow of how, where and who is involved in AI development -- and that we work collaboratively to apply appropriate security controls and emerging AI security techniques to those areas.
Decades ago, software development pioneer Barry Boehm articulated his famous maxim -- often called Boehm's Law -- that vulnerabilities cost exponentially more to fix the later in the lifecycle they're found. This principle applies equally -- if not more -- to AI. Getting security involved as early as possible pays dividends.
Ed Moyle is a technical writer with more than 25 years of experience in information security. He is a partner at SecurityCurve, a consulting, research and education company.
