Challenges of implementing SaaS successfully

1. Data and integration requirements

Data and integrations often cause issues -- especially when they are complex or have been customized to work with specific legacy applications. An organization should understand and document all data types, complexity, data state transitions and the integrated applications through which it flows. This includes all extract, transformation and load (ETL) processes throughout the workflow. 

As SaaS packages are considered, it is important to determine the complexity of the data conversion, possibly using a small proof of concept. For example, one organization's data was very complex because of the locations being plotted. Unfortunately, the implications weren't anticipated, resulting in a significant schedule slip. This could have been mitigated by documenting the risk on a risk register and allowing additional time in the schedule. 

2. Configurations vs. customizations

SaaS products are designed to provide the workflows that meet the usual requirements for that type of application. For example, an HR application will include workflows for onboarding and offboarding employees and performance management. Although they will achieve the same outcome, the workflows might not work in the same way as the legacy system. Customizations and configurations can fully meet the client's requirements.

When an organization is considering SaaS products, it will usually find that most required workflows are available out of the box. Most of these products enable configurations that client organizations can use to align their workflows as closely as possible with current workflows. Customizations can also align with current workflows and ensure that all of the client's requirements are met. However, unlike configurations, these are actual code changes made exclusively for the individual client. The client will be responsible for testing these for each new software release.

Often, client organizations request customizations to minimize the impact of change on users. Sometimes, vendors and integrators quickly offer customizations to facilitate the sales process. In one case, an organization implementing an ERP became mired down when the business requested multiple customizations to keep their workflows unchanged. Customizations were made; however, the schedule slipped, and this affected the ability to accept new releases. This happened because all the customizations needed to be retested for release, whereas the vendor was responsible for testing the configurations.

Organizations should use configurations whenever possible and customize only when necessary. Having documented the workflows upfront, a gap analysis against the SaaS product will identify the configurations and customizations required, enabling development and testing to be scheduled appropriately. Additionally, add change management risks to the risk register, be transparent with the users and begin empathetic change management as early as possible in the implementation.

3. Vendor selection

Vendor selection is probably the most crucial decision that an organization will make after deciding to implement the SaaS model. Once a SaaS product is implemented, the organization is committed to that vendor -- known as vendor lock-in. Once data has been converted -- and upstream and/or downstream applications have been implemented -- switching to a different vendor's product is quite complicated and costly. This makes vendor selection a high-risk decision.

The best way to mitigate risk in vendor selection is to thoroughly analyze the organization's requirements and to conduct an extensive review or vetting of available vendors. It is critical to thoroughly assess the vendor's product and reputation, which includes interviewing current clients. A proof-of-concept to flush out any major issues is also worthwhile. 

Organizations might fail to consider their broader business strategy and goals, especially if there are any significant changes in direction, mergers or acquisitions on the horizon. Since SaaS implementation is often led by IT, higher-level strategic directions might not be entirely on decision-makers' radars. They might be aware of strategic initiatives, but the impact might not be fully considered. 

For example, in one organization, a SaaS implementation of an ERP system was in process at the same time as a merger. The acquiring organization was already using the ERP from a different vendor. After a project pause and evaluation, it made more sense to switch to the vendor that the acquiring organization used. However, it was decided that too much money had been invested, so the project was continued. The result was that the merged organization could not integrate that operation and had to continue using both SaaS systems independently.

4. Client/vendor relationship management

A strong, mutually beneficial client/vendor relationship is critical to a successful SaaS implementation. When an organization moves critical business systems to the cloud, there is an inherent loss of control and a requirement for trust in the cloud vendor. Comprehensive vetting during the selection process is very important; however, the ongoing relationship must be managed. 

Ensure that the contract includes flexibility -- including an exit and migration strategy if possible. The statement of work (SOW) and the service level agreement (SLA) are the two most important documents on which the relationship is based. The SOW is the "go-to" document for the implementation, whereas the SLA is the working agreement for the ongoing operations.

In addition to scope, schedule, roles and responsibilities, and deliverables, the SOW should clearly define acceptance criteria. Issues can arise during acceptance testing, so it is important that the client has control over the testing and defect assessment processes. This includes enabling the client to create their own test cases and to have a documented path of escalation for defect remediation. For example, some SOWs require the use of vendor test plans and lack mutually agreed-upon definitions of defect priority and severity. These do not provide sufficient client control and may result in incomplete acceptance testing or open defects that cause issues in production.

5. Change management

Change management is one of the most critical challenges in a SaaS implementation; without it, an implementation can easily get derailed or result in low adoption of the new SaaS product. Effective change management begins long before the implementation. All stakeholders, especially the users-to-be, should be included in requirements gathering and kept informed throughout the selection process. Virtually all SaaS implementations require change management. 

For example, in one implementation of a new email system, although it was observed that the current application stored large volumes of emails in individual accounts, it was assumed that users would not need their old data converted. The decision to convert only recent data did not work well with the users and negatively affected their compliance with using the new application. This could have been avoided with upfront transparency and an understanding of users' approach to email management. 

Issues stemming from ineffective change management can be mitigated by including all change management tasks in the implementation schedule. These tasks include involving the users upfront, understanding their requirements, ensuring they understand how the new package will help them and training. Risks involved in change management, such as user buy-in and adoption, should be included in the risk register. The appointment of a SaaS application champion can also mitigate change management issues.

Finally, it is important to note that most of the critical challenges in SaaS implementations can be mitigated using project management techniques and tools. The two most important project management tools are the risk register and the schedule. 

The risk register -- sometimes called the RAID -- should include Risks, Assumptions, Issues and Dependencies. The risk register then informs us of the project schedule, especially regarding tasks and durations. All tasks, their dependencies, and the expected and actual start and completion dates should be included. Both the risk register and the schedule are dynamic documents and should be reviewed and updated continually throughout the implementation project.

Gerie Owen is a QA engineering manager at Roobrik. She is a conference presenter and author on technology and testing topics, and a certified Scrum master.