metamorworks -

Logistics firm taps Netflix Conductor service to manage AI

SPI Logistics looks to a microservices orchestration service provider founded by the creators of Netflix Conductor as it prepares to weave AI into developer workflows.

Like many companies amid the generative AI boom, SPI Logistics has been eager to add AI-driven automation to its business workflows. It first needed an IT automation platform that would be easy for both developers and a small IT ops team to use.

The logistics management and freight transportation services company in Surrey, British Columbia, began to modernize its architecture from monoliths into microservices more than two years ago. Microservices, a set of loosely coupled components that operate independently, have become popular over the last five years to manage complex cloud-native distributed applications, where each DevOps team can take responsibility for a single logical portion of the overall microservices workflow.

Still, organizations adopting microservices must also adjust their workflows to link business processes to the underlying distributed app framework. Initially, SPI's four-person ops team used a combination of the AWS Step Functions serverless event-driven application framework and open source Apache Airflow for workflow management.

But as SPI's microservices multiplied in 2021 and the company began to contemplate adding AI apps into the mix, its IT ops staff knew it needed a more formally packaged alternative.

"We're not a software company, even though software is an integral part [of the business]," said Ezequiel Peralta, vice president of technology at SPI Logistics. "So we needed to make decisions about how to speed up development without compromising security and focus on building products."

SPI weighs microservices orchestration vs. choreography

In 2022, Peralta's team evaluated alternatives to its internally built microservices workflow approach and came across microservices orchestration tools -- one of two major categories for microservices management, the other being microservices choreography.

Ezequiel Peralta, vice president of technology, SPI LogisticsEzequiel Peralta

"We had some sequences of data synchronization services that were very compatible with orchestration, and we were doing data orchestration with Airflow, which was cool. But that's more for loading our data warehouse, which we still do," Peralta said. "But we were starting to get into something Airflow wasn't designed for: more transactional operations. And we decided to look into something that's more compatible with that."

Microservices orchestration tools use a centralized control plane to manage sequentially ordered workflow steps that are distributed among compute nodes called workers. Microservices choreography distributes workflows using a more loosely coupled event-driven system. Microservices orchestration allows more control over workflows, but microservices orchestrators can represent a single point of failure if they're not managed well.

AWS Step Functions can be used in either type of workflow. Microservices choreography can also be managed with event-driven architecture frameworks, such as Apache Kafka and Amazon EventBridge. Microservices orchestration tools include Temporal; Camunda; and open source Conductor, created at Netflix and released to open source in 2016.

SPI considered commercial microservices orchestration tools from Camunda, implementing microservices orchestration on its own using AWS Step Functions and tools other than Airflow for data management, and self-managing Netflix Conductor.

The AWS Step Functions pricing model charges per state transition or each time a step of a workflow runs and would quickly become expensive under SPI's transactional workflows, according to Peralta. In the meantime, Conductor stood out because of its approach to scheduling compute nodes or workers.

"Conductor is a queue system. We're setting up the task queue, and workers are calling it for work," he said. "It might seem like a small difference, but then you can come up with very generic workers, and the orchestrator handles rate limiting. … You're scaling your system without touching the orchestrator. I think that makes it more reliable."

In Peralta's view, supplying security credentials only for workers that need it is also more secure than using an orchestration system that pushes workloads to a waiting set of credentialed workers via an internet connection.

Conductor runs on a Kubernetes cluster, which can automatically restart containerized workloads within the orchestrator if necessary, to shore up the resilience of that potential single point of failure. But given the inherent complexity of managing Kubernetes on top of managing microservices, SPI would need help from a vendor to keep it running.

Eventually, Peralta encountered Orkes, a cloud managed service provider founded by the creators of Conductor to run the microservices orchestration framework for enterprises. A year ago, SPI put Orkes Cloud into production.

"Conductor has a lot of configuration of moving parts to optimize it for the reduced latency and output we need to have. [And] you need to have multiple databases running," Peralta said. "The value [of Orkes] for us is huge. We get a lot of the features that go into the open source version earlier because of the support we have."

A comparison of microservices orchestration to microservices choreography.
Microservices orchestration tools utilize a centralized control plane to manage workflows in an orderly manner. Microservices choreography has no centralized point for management and uses an event-driven system.

At SPI, a newly efficient platform awaits AI apps

Orkes added a set of prepackaged workflows for AI microservices orchestration this month. Peralta said he's eager to try them as the company looks to add generative AI and machine learning based apps to replace rules-based automation in the company's fraud detection and freight spot market bidding processes.

Orkes' AI Orchestration feature, which supports large language models, including Microsoft's Copilot, OpenAI's GPT-4, Google's Vertex AI, and vector databases from Pinecone and Weaviate, also allow for human-driven tasks to be incorporated into these workflows.

Having microservices orchestration in place as the company adopts generative AI to analyze images and documents for classification or summarize available contracts in the bidding process will help SPI work more efficiently, Peralta said.

"[Orkes Cloud] allows us to parallelize our work more because you can have, for example, business specialists designing the workflows while the backend developers are working on all the deployment pieces and operations handles API access and database access," he said. "You can actually have the entire workflow designed with placeholder tasks [as it's implemented]."

We needed to make decisions about how to speed up development without compromising security and focus on building products.
Ezequiel PeraltaVice president of technology, SPI Logistics

Enterprises face dizzying array of AI workflow choices

As the initial generative AI hype dies down and enterprises search for ways to use it that align with existing workflows and governance practices, vendors from virtually every corner of the IT market are vying for their attention. These include microservices choreography and orchestration vendors as well as Kubernetes-based developer platforms, such as Red Hat OpenShift Data Science, and emerging MLSecOps pipelines from JFrog and others. Business process management vendors and integration-platform-as-a-service providers comprise an entirely separate landscape of tools for enterprises to consider.

"The lines are blurring among all of these, because modern apps are inherently distributed," said Matt Brasier, an analyst at Gartner. "People are also realizing that one great big [AI] model is not the best way to solve problems. Often, business problems are a mix of classification and content generation, so you want to chain some stuff together."

Deciding which IT automation tools to use will come down to the nature of the distributed applications businesses need. This and the skills available among IT teams to manage various automation tools are factors to consider when selecting the choreography or orchestration approach, Brasier said.

"I normally tell clients to begin with the skills of the person doing the orchestration, who is going to be creating that and managing that workflow," he said. "These things, like any integration, are going to break; a new version will come out that breaks all of your orchestration. So the skills of the person that is building and managing that orchestration will be key."

Beth Pariseau, senior news writer at TechTarget, is an award-winning veteran of IT journalism. She can be reached at [email protected] or on Twitter @PariseauTT.

Dig Deeper on Systems automation and orchestration

Software Quality
App Architecture
Cloud Computing
Data Center