How businesses should deal with enterprise search issues

The biggest enterprise search issues

If a business wants to improve enterprise search, it should consider addressing the following issues:

• Search system support. Enterprise search requires daily support by a team dedicated to governance, including maintaining metadata standards, keeping crawls on schedule, staying on top of security policy and troubleshooting issues.
• User expectations. Enterprise search often fails its users because they do not understand how it works. Google gives them exactly what they're looking for, so they may wonder why enterprise search can't do the same. The answer is that enterprise search engines work differently than the major commercial search engines. Commercial engines machine-learn usage patterns across hundreds of millions of users and adjust their search albums accordingly, while enterprise search engines require meticulous metadata tuning to achieve comparable results.
• Crawl errors. Businesses must crawl their content in order to properly index it. A crawl means that the search engine must access the content to extract its embedded links. Sometimes, the crawl of the company's content, which should be done often, errors out on some pages. When this happens and users don't realize it, content is excluded from the crawl, goes unindexed and is consequently unfindable.
• Multiple versions. Differing versions of the same documents are inevitable in a business, and they can badly clutter a search result.
• Incorrect/incomplete metadata. If the metadata is wrong, the content won't surface. This means that a user might enter search terms that should be correct but aren't associated with the desired content.
• Unstructured data. There is more unstructured data than ever in company content management systems (CMSes), owing largely to social media. This data is hard to find because it doesn't fit within any particular data model. This can include a variety of data, including stored emails and text messages, presentation files and audio content.
• Security. Many businesses pay too little attention to the essential balance between access and privacy. Some content should be available to all, while some should be carefully restricted. Many organizations don't apply the necessary effort to define permissions and appropriate controls.

These issues can create problems on their own or in combination with one another. Even when the system itself is well maintained, users can botch it all by themselves.

User frustration is not the only cost of search failure. The M-Files 2019 Intelligent Information Management Benchmark Report stated that more than 80% of workers recreated a document that already existed because they could not locate it.

Vendor fails

Enterprise search issues are not entirely the fault of the user, system administrator or poor methodology. There's a tension between applied enterprise search and the vendors who create it, resulting in products that are overhyped and too complex to easily maintain.

Enterprise search issues are not entirely the fault of the user, system administrator or poor methodology.

Microsoft SharePoint, despite spreading content management to the masses over the past two decades, is the pacesetter in vendor fails. It can break down from innumerable causes, including incomplete indexing -- businesses should be careful about what they change during a crawl -- and web config issues on the search server to missing ASPX (Active Server Page Extended) pages. Microsoft promises improved search functionality with each new release of SharePoint and always comes up short. Most recently, Microsoft rolled out the SharePoint associated hubs feature at the Ignite 2019 conference, which improved content discovery between hubs but isn't enterprise-wide.

But SharePoint isn't alone. Many enterprise search vendors fail to provide adequate visibility into search fails, crawl fails and indexing issues by offering only logs and rudimentary dashboards, with little drill-down capability. And few offer administrative documentation or troubleshooting resources that accommodate the full range of enterprise search file types and source incompatibilities.

Getting it right

Fortunately, a business has many options when it comes to solving the enterprise search issues listed above:

• Employee training in enterprise search should be standard. If businesses provide employees with an overview of the enterprise system -- with attention to how it works, how to query properly and the guidelines for effective metadata tagging -- employee experience would improve along with the quality of the system.
• Metadata as business intelligence. Businesses should treat the creation and use of metadata as a digital asset, the product of strategic and well-governed policy. This not only improves search functionality, but also increases the value of the metadata. When the integrity of search is measurable, so is the usage of key documents, opening the door to the discovery of trends and the investigation of reduced interest in content.
• The rule of least privilege. The database administrator's standard for balancing access and privacy is an ideal approach to securing enterprise content in search: Grant users the minimum permissions they require to do their work. Apply the same principle to enterprise search security in the CMS.
• Unstructured data and metadata. Finally, there's unstructured data -- the data that streams in from social media, email and other sources and gets dumped into the CMS almost randomly. If a business tags content accurately and conforms to enterprise-wide metadata standards, it doesn't matter that data doesn't conform to a model. When a business conforms to this strategic tagging approach, users can easily surface all data -- including metadata.