Copyright TechTarget - All rights reserved https://cyber.law.harvard.edu/rss/rss.html Techtarget Feed Generator en Tue, 14 Apr 2026 22:34:54 GMT https://www.techtarget.com/searchenterpriseai editor@techtarget.com <p>Enterprise AI systems are increasingly capable of <a href="https://www.techtarget.com/searchenterpriseai/definition/anomaly-detection">detecting anomalies</a>, but most can't explain them. When automated decisions behave unexpectedly, organizations often struggle to determine whether the cause is model drift, data corruption, environmental change or rational adaptation. This diagnostic gap is emerging as one of the most significant operational risks in enterprise AI.</p> <p>As organizations move from experimentation to large-scale automation, AI increasingly drives mission-critical workflows. From fraud detection to logistics planning, <a href="https://www.techtarget.com/searchitoperations/tip/Beyond-automation-Using-GenAI-to-modernize-IT-operations">automated systems influence operational decisions</a> every minute. But when outputs conflict with operational reality -- approving the wrong transaction, misclassifying a customer or triggering false alerts -- teams often struggle to determine why.</p> <p>Most systems can detect that something unusual occurred. Few can explain it. That gap is quickly becoming a governance challenge.</p> <section class="section main-article-chapter" data-menu-title="Detection isn't diagnosis"> <h2 class="section-title"><i class="icon" data-icon="1"></i>Detection isn't diagnosis</h2> <p>Consider a real-time logistics system in which estimated arrival times suddenly spike in variance. Monitoring tools flag the anomaly immediately. But what caused it?</p> <p>Was it caused by degraded GPS signals? A change in driver behavior? A modification in the road network? Or simply the model responding rationally to new traffic conditions?</p> <blockquote class="main-article-pullquote"> <div class="main-article-pullquote-inner"> <figure> Without structured diagnostic intelligence, organizations risk misattributing failure. </figure> <i class="icon" data-icon="z"></i> </div> </blockquote> <p>Without diagnostic tools, engineering <a href="https://www.techtarget.com/searchenterpriseai/tip/AI-model-optimization-How-to-do-it-and-why-it-matters">teams often default to retraining the model</a>. Days later, they might discover that the true cause was an upstream data encoding change introduced by a third-party mapping provider. Engineering time is lost, and operator confidence declines.</p> <p>This pattern appears across industries. Detection answers one question: Did something unexpected happen?</p> <p>Diagnosis answers the more important one: Why did it happen?</p> <p>Without structured diagnostic intelligence, organizations risk misattributing failures. A frontline operator is blamed for noncompliance when the root cause lies in outdated model assumptions. A data science team retrains a model unnecessarily when the underlying issue is ambiguous data encoding. Executives lose confidence in automation simply because the system can't explain its own behavior in operational terms.</p> <p>In complex production environments, unexpected outcomes rarely have a single cause. They emerge from interactions among model behavior, data pipelines, environmental shifts and human adaptation. Treating every anomaly as model failure oversimplifies reality.</p> </section> <section class="section main-article-chapter" data-menu-title="The trust problem"> <h2 class="section-title"><i class="icon" data-icon="1"></i>The trust problem</h2> <p>Successful enterprise AI depends not only on model performance but also on <a href="https://www.techtarget.com/searchenterpriseai/feature/How-to-ensure-AI-transparency-explainability-and-trust">institutional trust</a>.</p> <p>When operators repeatedly encounter AI recommendations that appear inconsistent or unexplained, override behavior increases. Manual reviews return. Governance committees grow cautious. Automation initiatives slow down.</p> <p>The technical issue becomes an organizational one.</p> <p>Without diagnostic clarity, accountability becomes diffuse. Audit trails might show what happened, but they rarely explain why. As AI systems gain autonomy in operational workflows, this gap widens, increasing risk rather than reducing it.</p> </section> <section class="section main-article-chapter" data-menu-title="The diagnostic maturity check"> <h2 class="section-title"><i class="icon" data-icon="1"></i>The diagnostic maturity check</h2> <p>Before <a href="https://www.techtarget.com/searchenterpriseai/tip/Best-practices-for-building-scalable-AI-infrastructure">scaling automation</a> further, technology leaders should ask the following four questions:</p> <ol class="default-list"> <li><b>Attribution. </b>If a model produces highly unusual outputs, can the organization distinguish between a data pipeline failure and a genuine change in external conditions?</li> <li><b>Override analysis. </b>Are teams analyzing why human operators override AI recommendations, or are they simply recording that overrides occurred?</li> <li><b>Drift versus context. </b>Do monitoring tools treat deviations as binary errors, or can they determine whether the model is adapting rationally to new environmental constraints?</li> <li><b>Audit speed. </b>If an auditor or regulator asks for the reasoning behind an automated decision, can the organization produce a clear explanation within minutes, or does it require a week of data analysis time?</li> </ol> <p>The answers to these questions often reveal whether an organization truly understands how its AI systems behave in production.</p> </section> <section class="section main-article-chapter" data-menu-title="What diagnostic intelligence looks like"> <h2 class="section-title"><i class="icon" data-icon="1"></i>What diagnostic intelligence looks like</h2> <p>Diagnostic intelligence means embedding structured reasoning into AI operations. It requires systems that can investigate their own behavior.</p> <p>In practice, four capabilities define this approach.</p> <h3>Behavioral stability monitoring</h3> <p>Traditional monitoring focuses on accuracy metrics or threshold alerts. Diagnostic systems track how model behavior evolves over time, <a href="https://www.techtarget.com/searchenterpriseai/tip/How-to-identify-and-manage-AI-model-drift">identifying patterns that signal drift</a>, instability or environmental shifts.</p> <h3>Data integrity validation</h3> <p>Many AI failures originate upstream in data pipelines. Diagnostic systems verify that input representations align with business intent and that encoding changes or schema mismatches are detected before they propagate downstream.</p> <h3>Contextual reality assessment</h3> <p>Not all deviations indicate failure. Sometimes the environment changes while the model behaves rationally. Diagnostic frameworks incorporate external context, such as operational disruptions, regulatory changes or supply chain events, to evaluate model behavior relative to current conditions rather than historical baselines.</p> <h3>Structured evidence aggregation</h3> <p>True diagnosis requires combining signals from multiple subsystems. Diagnostic frameworks synthesize evidence from models, data pipelines and operational signals to produce traceable explanations.</p> <p>Instead of a generic alert like <i>anomaly detected,</i> a diagnostic system might report the following:</p> <p><i>Variance increase attributed to upstream data encoding changes affecting 14% of records between 9:00 and 11:30. Model behavior consistent with historical performance on clean data. Recommended action: data pipeline remediation rather than model retraining.</i></p> <p>This added level of specificity transforms alerts into actionable investigations.</p> <blockquote class="main-article-pullquote"> <div class="main-article-pullquote-inner"> <figure> Organizations that adopt diagnostic intelligence shift their response to AI deviations. Instead of asking who made the mistake, leaders ask more productive questions. </figure> <i class="icon" data-icon="z"></i> </div> </blockquote> </section> <section class="section main-article-chapter" data-menu-title="From blame to continuous improvement"> <h2 class="section-title"><i class="icon" data-icon="1"></i>From blame to continuous improvement</h2> <p>Organizations that adopt diagnostic intelligence shift their response to AI deviations. Instead of asking who made the mistake, leaders ask more productive questions, such as the following:</p> <ul class="default-list"> <li>Did model assumptions drift?</li> <li>Did the operating environment change?</li> <li>Did data representations miscommunicate intent?</li> <li>Is the system behaving rationally under new constraints?</li> </ul> <p>This shift reframes AI governance from reactive troubleshooting to continuous improvement.</p> <p>Diagnostic layers reduce unnecessary retraining cycles. They preserve operator trust, strengthen auditability and, most importantly, they align automation with accountability.</p> </section> <section class="section main-article-chapter" data-menu-title="What CIOs should do next"> <h2 class="section-title"><i class="icon" data-icon="1"></i>What CIOs should do next</h2> <p>Enterprise leaders can begin building diagnostic maturity by focusing on several practical steps:</p> <ul class="default-list"> <li>Treat anomaly detection as the starting point of AI governance, not the final safeguard.</li> <li>Establish diagnostic workflows that investigate system behavior before retraining models.</li> <li>Track operator overrides as signals of system misalignment.</li> <li>Build audit trails that explain why decisions occurred, not just what decisions were made.</li> </ul> <p>This approach transforms AI monitoring from simple alerting into <a href="https://www.techtarget.com/searchbusinessanalytics/definition/operational-business-intelligence">operational intelligence</a>.</p> </section> <section class="section main-article-chapter" data-menu-title="What's next for enterprise AI"> <h2 class="section-title"><i class="icon" data-icon="1"></i>What's next for enterprise AI</h2> <p>The first wave of enterprise AI focused on prediction. The second emphasized automation. The next phase must focus on diagnosis.</p> <p>As AI systems increasingly influence financial approvals, operational workflows and safety-critical decisions, the cost of misattributed failures grows. Detection alone is no longer sufficient. Organizations need AI systems capable of explaining and correcting their own behavior.</p> <p>For CIOs and technology leaders, investing in diagnostic intelligence isn't merely a technical enhancement. It's a governance imperative, ensuring that reliability, transparency and trust scale as automation scales.</p> <p>The organizations that succeed in the next decade won't simply deploy AI. They will deploy AI systems that can explain themselves before trust, compliance or safety is compromised.</p> <p><i>Rashmi Choudhary is a data scientist specializing in large-scale AI systems for routing, navigation and operational intelligence. An IEEE senior member and inventor on multiple patents in transportation AI, she focuses on building reliable and accountable AI systems for safety-critical environments. She writes about AI governance, infrastructure intelligence and production system reliability.</i></p> </section> Enterprise AI detects anomalies but can't always explain why they happen. Next up for these systems is a focus on diagnosis to enhance trust, compliance and safety. https://cdn.ttgtmedia.com/rms/onlineimages/machine%20leaning_g1150854211.jpg https://www.techtarget.com/searchenterpriseai/post/Why-enterprise-AI-needs-diagnostic-intelligence Wed, 25 Mar 2026 16:45:00 GMT <p>Amazon Bedrock -- also known as AWS Bedrock -- is a machine learning platform used to build <a href="https://www.techtarget.com/searchenterpriseai/definition/generative-AI">generative artificial intelligence</a> (AI) applications on the Amazon Web Services cloud computing platform. Bedrock uses <a href="https://www.techtarget.com/whatis/feature/Foundation-models-explained-Everything-you-need-to-know">foundation models</a> to simplify the creation of these apps and make the process more efficient.</p> <p>Foundation models are adaptable AI models trained on <a href="https://www.techtarget.com/searchdatamanagement/definition/big-data">large data sets</a> to perform many kinds of tasks. They're versatile, reusable and don't require retraining for each new task. Bedrock replaces the physical infrastructure typically used to <a href="https://www.techtarget.com/searchenterpriseai/tip/Assessing-different-types-of-generative-AI-applications">build generative AI apps</a> with foundation models, simplifying the app building process.</p> <p>Bedrock is a competitor to OpenAI <a href="https://www.techtarget.com/whatis/definition/ChatGPT">ChatGPT</a> and <a href="https://www.techtarget.com/searchenterpriseai/definition/Dall-E">Dall-E</a> 2. It is also compared to <a href="https://www.techtarget.com/searchaws/definition/Amazon-SageMaker">Amazon SageMaker</a>, which is used to build and train complex machine learning models; Bedrock is more focused on building generative AI apps.</p> <figure class="main-article-image full-col" data-img-fullsize="https://www.techtarget.com/rms/onlineimages/amazon_quicksight_generative_ai_screenshot-f.jpg"> <img data-src="https://www.techtarget.com/rms/onlineimages/amazon_quicksight_generative_ai_screenshot-f_mobile.jpg" class="lazy" data-srcset="https://www.techtarget.com/rms/onlineimages/amazon_quicksight_generative_ai_screenshot-f_mobile.jpg 960w,https://www.techtarget.com/rms/onlineimages/amazon_quicksight_generative_ai_screenshot-f.jpg 1280w" alt="Screenshot of Amazon QuickSight Q generative business intelligence (BI) dashboard built using Amazon Bedrock" data-credit="AWS" height="334" width="560"> <figcaption> <i class="icon pictures" data-icon="z"></i>Amazon QuickSight Q is a generative business intelligence platform built using Amazon Bedrock, which provides access to foundation models. </figcaption> <div class="main-article-image-enlarge"> <i class="icon" data-icon="w"></i> </div> </figure> <section class="section main-article-chapter" data-menu-title="How Amazon Bedrock works"> <h2 class="section-title"><i class="icon" data-icon="1"></i>How Amazon Bedrock works</h2> <p>Amazon Bedrock gives software developers access to a wide range of foundation models from AI startups, such as AI21 Labs, Anthropic, Cohere and Stability AI, through a serverless application programming interface (<a href="https://www.techtarget.com/searchapparchitecture/definition/application-program-interface-API">API</a>). For example, <a href="https://www.techtarget.com/whatis/definition/large-language-model-LLM">large language models</a> such as Anthropic's Claude 2 and open source, text-to-image models such as Stability AI's Stable Diffusion XL can be used with Bedrock to simplify the delivery of generative AI apps.</p> <p>On their own, foundation models are adept at comprehending <a href="https://www.techtarget.com/searchenterpriseai/definition/natural-language-processing-NLP">natural language</a> inputs and processing them to produce text or images as responses or outputs. However, they can't perform complex tasks or actions without direction.</p> <p>AWS <a target="_blank" href="https://aws.amazon.com/bedrock/agents/" rel="noopener">released</a> Agents for Amazon Bedrock to designate and automate complex tasks for a model without requiring a developer to manually write the code needed to do so. Specifically, developers can use agents to connect foundation models to their <a href="https://www.techtarget.com/searchdatamanagement/tip/Open-source-vs-proprietary-database-management">proprietary data sources</a> so the apps they build will produce up-to-date answers based on their own data. When a user employs a generative AI app built with Bedrock, an agent makes API calls that retrieve the data needed from proprietary sources to answer the user's requests or queries.</p> <p>In addition to third-party foundation models, Amazon allows <a target="_blank" href="https://aws.amazon.com/bedrock/amazon-models/titan/" rel="noopener">access to its own</a> Titan foundation models, which include Titan Text to generate text and Titan Embeddings to translate textual inputs into numerical representations.</p> </section> <section class="section main-article-chapter" data-menu-title="Applications developers can make with Amazon Bedrock"> <h2 class="section-title"><i class="icon" data-icon="1"></i>Applications developers can make with Amazon Bedrock</h2> <p>Amazon Bedrock can be used to build the following apps that are useful when applied to real-world use cases and workloads:</p> <ul class="default-list"> <li><b>Text generation.</b> Apps built with Amazon Bedrock can generate original written text in various forms, such as short stories, blog posts, news articles and social media posts.</li> <li><b>Conversational AI.</b> Customized chatbots or virtual assistants built using Bedrock are based on foundation models having access to proprietary data owned by a developer or software vendor. This provides good <a href="https://www.techtarget.com/searchenterpriseai/definition/conversational-AI">conversational AI</a> responses to users' queries.</li> <li><b>Text summarization.</b> A simple Bedrock app offers the capability to summarize text without forcing a user to pore over lengthy documents or materials.</li> <li><b>Image generation.</b> Bedrock's API can interface with various types of foundation models, including those with <a href="https://www.datasciencecentral.com/a-guide-to-how-text-to-speech-works/" target="_blank" rel="noopener">text-to-speech</a> AI capabilities. Given that, an app built on Bedrock can take a request or prompt for a specific image from a user and generate that image.</li> </ul> <div class="extra-info"> <div class="extra-info-inner"> <h3>Learn more about generative AI and how it might change enterprise processes.</h3> <p><a href="https://www.techtarget.com/searchenterpriseai/tip/Generative-AI-landscape-Potential-future-trends">Generative AI landscape: Potential future trends</a></p> <p><a href="https://www.techtarget.com/searchenterpriseai/feature/Generative-AI-in-business-Fast-uptake-earmarked-funding">Generative AI in business: Fast uptake, earmarked funding</a></p> <p><a href="https://www.techtarget.com/searchenterpriseai/feature/New-skills-in-demand-as-generative-AI-reshapes-tech-roles">New skills in demand as generative AI reshapes tech roles</a></p> <p><a href="https://www.techtarget.com/searchcustomerexperience/tip/Generative-AI-tools-to-consider-for-marketing-and-sales">Generative AI tools to consider for marketing and sales</a></p> <p><a href="https://www.techtarget.com/whatis/feature/Will-AI-replace-jobs-9-job-types-that-might-be-affected">Will AI replace jobs? Job types that might be affected</a></p> </div> </div> </section> <section class="section main-article-chapter" data-menu-title="Is Amazon Bedrock generally available?"> <h2 class="section-title"><i class="icon" data-icon="1"></i>Is Amazon Bedrock generally available?</h2> <p>Amazon Bedrock became generally available in September 2023. Two <a target="_blank" href="https://aws.amazon.com/bedrock/pricing/" rel="noopener">pricing plans</a> are available. One plan lets users access foundation models on a pay-as-you-go basis. The other lets users provision throughput to meet their performance requirements, but it requires a time-based term commitment.</p> <p><i>As businesses delve into generative AI and the myriad ways to customize generative AI applications, it's worth </i><a href="https://www.techtarget.com/whatis/feature/Pros-and-cons-of-AI-generated-content"><i>understanding the pros and cons of AI-generated content</i></a><i> to be prepared for the future.</i></p> <div class="youtube-iframe-container"> <iframe id="ytplayer-0" src="https://www.youtube.com/embed/1U83yhGY_pI?si=efl2tRiWTrs2QWa7?autoplay=0&amp;modestbranding=1&amp;rel=0&amp;widget_referrer=null&amp;enablejsapi=1&amp;origin=https://www.techtarget.com" type="text/html" height="360" width="640" frameborder="0"></iframe> </div> <p></p> </section> Amazon Bedrock -- also known as AWS Bedrock -- is a machine learning platform used to build generative artificial intelligence (AI) applications on the Amazon Web Services cloud computing platform. https://cdn.ttgtmedia.com/visuals/digdeeper/2.jpg https://www.techtarget.com/searchenterpriseai/definition/Amazon-Bedrock-AWS-Bedrock Tue, 17 Dec 2024 00:00:00 GMT <p>One year after investing in Microsoft 365 backup-as-a-service provider Alcion, Veeam Software acquired the startup with roots in AI and security.</p> <p>Analysts and Veeam executives said the acquisition will strengthen Veeam's budding as-a-service offerings. Earlier this year, the vendor launched Veeam Data Cloud backup as a service to protect Microsoft 365 and Azure workloads.</p> <p>"Veeam, after resisting it for many years, finally went into the as-a-service business," said Christophe Bertrand, an analyst at TheCube Research.</p> <p>The Veeam acquisition, which closed in mid-September, is the data protection vendor's second purchase of a company founded by Niraj Tolia and Vaibhav Kamra. In 2020, Veeam acquired their Kubernetes backup provider Kasten. In September 2023, <a href="https://www.techtarget.com/searchdatabackup/news/366552363/Veeam-leads-funding-round-for-SaaS-backup-provider-Alcion">Veeam led a $21 million funding round</a> for recently out-of-stealth Alcion.</p> <p>The data protection market has seen a surge in acquisition activity in the last year. For example, <a href="https://www.techtarget.com/searchdatabackup/news/366611857/Commvault-acquisition-of-Clumio-for-S3-speaks-volumes">Commvault acquired Clumio</a> this week, <a href="https://www.techtarget.com/searchdatabackup/news/366569336/Cohesity-Veritas-combine-as-new-data-protection-company">Cohesity and Veritas</a> are merging, and Veeam's purchase of Cirrus from CT4 late last year eventually became the Veeam Data Cloud. Earlier this year, Veeam also acquired <a href="https://www.techtarget.com/searchdatabackup/news/366581913/Veeam-acquires-Coveware-for-incident-response-capabilities">incident response vendor Coveware</a>.</p> <p>Veeam did not release terms of its latest acquisition.</p> <p>"Veeam is not historically an often-acquiring organization, but that has clearly changed in the last few years," said Rick Vanover, Veeam's vice president of product strategy. "I don't see that behavior stopping."</p> <section class="section main-article-chapter" data-menu-title="Alcion at Veeam's service"> <h2 class="section-title"><i class="icon" data-icon="1"></i>Alcion at Veeam's service</h2> <p>The latest Veeam acquisition brings additional expertise in fast-growing areas, with as a service at the top, Vanover said.</p> <div class="imagecaption alignLeft"> <img src="https://cdn.ttgtmedia.com/rms/onlineImages/tolia_niraj.jpg" alt="Niraj Tolia, CTO, Veeam">Niraj Tolia </div> <p>Alcion's team of fewer than 50 employees joins Veeam, including Tolia as the new CTO and Kamra as vice president of technology. Tolia will lead product strategy and engineering for the Veeam Data Cloud. He succeeds Danny Allan, who left his CTO position to take on the same role at cybersecurity provider Snyk.</p> <p>A year ago, when asked if Alcion was open to an acquisition by Veeam, Tolia said the company's focus was on an independent path, growth, product enhancements and customer value. Now it's about going to the next level for Alcion and Veeam.</p> <p>"For our customers, it's just a much larger stage, getting to a much larger scale, much faster," Tolia said.</p> <p>Alcion claims to have hundreds of customers. Those customers will receive an offer to transition to Veeam Data Cloud, said Brandt Urban, Veeam's senior vice president of worldwide cloud sales. Veeam also offers standalone Backup for Microsoft Azure and Backup for Microsoft 365 products.</p> <p>Veeam, however, has not made a final decision about the end result for the <a href="https://www.techtarget.com/searchdatabackup/news/366538893/Alcion-applies-AI-security-focus-to-Microsoft-365-backup">Alcion product</a> and does not have a timetable for the integration yet.</p> <p>"Having us infuse this fastest-growing product in Veeam's history with the talent and the thought leadership that's coming in with Niraj, and the development capabilities across the Alcion team, is going to let us enhance the product faster, add more capabilities and then add more workloads," Urban said of Veeam Data Cloud.</p> <p>Bertrand said he is hoping to see Veeam cover additional SaaS workloads with its data protection, pointing to other collaboration and DevOps platforms as examples. In addition to Microsoft 365 protection, Veeam also offers Backup for Salesforce.</p> <p><b>"</b>I look at the strategy that <a href="https://www.techtarget.com/searchdatabackup/news/365530034/HYCU-R-Cloud-expands-data-protection-for-SaaS">HYCU has adopted</a>, which is different. They cover many, many platforms through a different mechanism than Veeam does," Bertrand said, referring to HYCU's R-Cloud. "It feels a little bit like an arms race has started. In the next few months, we'll know exactly which ones will be the most successful."</p> </section> <section class="section main-article-chapter" data-menu-title="AI and security and AI, oh my!"> <h2 class="section-title"><i class="icon" data-icon="1"></i>AI and security and AI, oh my!</h2> <p>The AI in Alcion's product helps administrators perform intelligent backups and detect malware. Krista Case, an analyst at The Futurum Group, said she sees Alcion using AI strategically, for example to&nbsp;adapt <a target="_blank" href="https://docs.alcion.ai/concepts/backup-and-restore/#backup" rel="noopener">backup schedules</a> based on data modification patterns, to trigger a backup based on identification of potentially malicious activity, and to suggest the best recovery point.</p> <p>"When we talk with practitioners&nbsp;about&nbsp;cyber resilience, they are most concerned about minimizing data loss and downtime -- and these Alcion capabilities directly address that requirement," Case said.</p> <blockquote class="main-article-pullquote"> <div class="main-article-pullquote-inner"> <figure> AI is actually a positive as long as it is not just hype. </figure> <figcaption> <strong>Christophe Bertrand</strong>Analyst, TheCube Research </figcaption> <i class="icon" data-icon="z"></i> </div> </blockquote> <p>End users and partners want vendors like Veeam to use AI in a smart and governed way, for example to help fight ransomware, Bertrand said. Veeam's AI capabilities include inline malware detection, its Intelligent Diagnostics service and a forthcoming <a href="https://www.techtarget.com/searchdatabackup/news/366587408/Veeam-backup-and-security-updates-include-cloud-vault-copilot">Copilot for its Backup for Microsoft 365</a>.</p> <p>"AI is actually a positive as long as it is not just hype," Bertrand said.</p> <p>And there is a lot of hype, with seemingly every tech vendor touting AI-focused features and products.</p> <p>"The focus [for users] is less on having a chatbot, and it is more about the outcome that AI can enable," Case said. "This might include detecting attacks that otherwise would have fallen through the cracks, for example."</p> <p>Veeam executives noted the importance of clear AI benefits for users.</p> <p>"We keep that top of mind because otherwise it's a really expensive experiment," Vanover said.</p> <p><i>Paul Crocetti is an executive editor at TechTarget Editorial. Since 2015, he has worked on TechTarget's Storage, Data Backup and Disaster Recovery sites.</i></p> </section> Veeam embarks on the acquisition path again with its purchase of Alcion. The deal brings several employees who specialize in AI and as-a-service knowledge. https://cdn.ttgtmedia.com/visuals/searchBusinessAnalytics/BI_management/businessanalytics_article_001.jpg https://www.techtarget.com/searchdatabackup/news/366612117/Veeam-acquisition-of-Alcion-supports-push-into-as-a-service-AI Fri, 27 Sep 2024 10:11:00 GMT <section class="section main-article-chapter" data-menu-title="What is BERT?"> <h2 class="section-title"><i class="icon" data-icon="1"></i>What is BERT?</h2> <p>BERT language model is an open source machine learning framework for natural language processing (<a href="https://www.techtarget.com/searchenterpriseai/definition/natural-language-processing-NLP">NLP</a>). BERT is designed to help computers understand the meaning of ambiguous language in text by using surrounding text to establish context. The BERT framework was pretrained using text from Wikipedia and can be fine-tuned with question-and-answer data sets.</p> <p>BERT, which stands for Bidirectional Encoder Representations from Transformers, is based on <a href="https://www.techtarget.com/searchenterpriseai/definition/transformer-model">transformers</a>, a <a href="https://www.techtarget.com/searchenterpriseai/definition/deep-learning-deep-neural-network">deep learning</a> model in which every output element is connected to every input element, and the weightings between them are dynamically calculated based upon their connection.</p> <p>Historically, language models could only read input text sequentially -- either left-to-right or right-to-left -- but couldn't do both at the same time. BERT is different because it's designed to read in both directions at once. The introduction of transformer models enabled this capability, which is known as bidirectionality. Using bidirectionality, BERT is pretrained on two different but related NLP tasks: masked language modeling (<a href="https://www.techtarget.com/searchenterpriseai/definition/masked-language-models-MLMs">MLM</a>) and next sentence prediction (NSP).</p> <p>The objective of MLM training is to hide a word in a sentence and then have the program predict what word has been hidden based on the hidden word's context. The objective of NSP training is to have the program predict whether two given sentences have a logical, sequential connection or whether their relationship is simply random.</p> <div class="youtube-iframe-container"> <iframe id="ytplayer-0" src="https://www.youtube.com/embed/tUT9PD5ttMo?si=ummGM_MO4rnGzGr4?autoplay=0&amp;modestbranding=1&amp;rel=0&amp;widget_referrer=null&amp;enablejsapi=1&amp;origin=https://www.techtarget.com" type="text/html" height="360" width="640" frameborder="0"></iframe> </div> </section> <section class="section main-article-chapter" data-menu-title="Background and history of BERT"> <h2 class="section-title"><i class="icon" data-icon="1"></i>Background and history of BERT</h2> <p>Google first introduced the transformer model in 2017. At that time, language models primarily used recurrent neural networks (<a href="https://www.techtarget.com/searchenterpriseai/definition/recurrent-neural-networks">RNN</a>) and convolutional neural networks (<a href="https://www.techtarget.com/searchenterpriseai/definition/convolutional-neural-network">CNN</a>) to handle NLP tasks.</p> <p><a href="https://www.techtarget.com/searchenterpriseai/feature/CNN-vs-RNN-How-they-differ-and-where-they-overlap">CNNs and RNNs</a> are competent models, however, they require sequences of data to be processed in a fixed order. Transformer models are considered a significant improvement because they don't require data sequences to be processed in any fixed order.</p> <p>Because transformers can process data in any order, they enable training on larger amounts of data than was possible before their existence. This facilitated the creation of pretrained models like BERT, which was trained on massive amounts of language data prior to its release.</p> <p>In 2018, Google introduced and open sourced BERT. In its research stages, the framework achieved state-of-the-art results in 11 natural language understanding (<a href="https://www.techtarget.com/searchenterpriseai/definition/natural-language-understanding-NLU">NLU</a>) tasks, including <a href="https://www.techtarget.com/searchbusinessanalytics/definition/opinion-mining-sentiment-mining">sentiment analysis</a>, semantic role labeling, text classification and the <a href="https://www.techtarget.com/searchdatamanagement/definition/disambiguation">disambiguation</a> of words with multiple meanings. Researchers at Google AI Language published a report that same year explaining these results.</p> <p>Completing these tasks distinguished BERT from previous language models, such as word2vec and GloVe. Those models were limited when interpreting context and polysemous words, or words with multiple meanings. BERT effectively addresses ambiguity, which is the greatest challenge to NLU, according to research scientists in the field. It's capable of parsing language with a relatively human-like common sense.</p> <p>In October 2019, Google announced that it would begin applying BERT to its U.S.-based production <a href="https://www.techtarget.com/whatis/feature/Google-algorithms-explained-Everything-you-need-to-know">search algorithms</a>.</p> <p>It is <a target="_blank" href="https://blog.google/products/search/search-language-understanding-bert/" rel="noopener">estimated</a> that BERT enhances Google's understanding of approximately 10% of U.S.-based English language Google search queries. Google recommends that organizations not try to optimize content for BERT, as BERT aims to provide a natural-feeling search experience. Users are advised to keep queries and content focused on the natural subject matter and natural user experience.</p> <p>BY December 2019, BERT had been applied to more than 70 different languages. The model has had a large impact on voice search as well as text-based search, which prior to 2018 had been error-prone with Google's NLP techniques. Once BERT was applied to many languages, it improved search engine optimization; its proficiency in understanding context helps it interpret patterns that different languages share without having to completely understand the language.</p> <p>BERT went on to influence many <a href="https://www.techtarget.com/searchenterpriseai/definition/AI-Artificial-Intelligence">artificial intelligence</a> systems. Various lighter versions of BERT and similar training methods have been applied to models from GPT-2 to <a href="https://www.techtarget.com/whatis/definition/ChatGPT">ChatGPT</a>.</p> </section> <section class="section main-article-chapter" data-menu-title="How BERT works"> <h2 class="section-title"><i class="icon" data-icon="1"></i>How BERT works</h2> <p>The goal of any given NLP technique is to understand human language as it is spoken naturally. In BERT's case, this means predicting a word in a blank. To do this, models typically train using a large repository of specialized, labeled training data. This process involves linguists doing laborious manual <a href="https://www.techtarget.com/whatis/definition/data-labeling">data labeling</a>.</p> <p>BERT, however, was pretrained using only a collection of unlabeled, plain text, namely the entirety of English Wikipedia and the Brown Corpus. It continues to learn through <a href="https://www.techtarget.com/searchenterpriseai/definition/unsupervised-learning">unsupervised learning</a> from unlabeled text and improves even as it's being used in practical applications such as Google search.</p> <p>BERT's pretraining serves as a base layer of knowledge from which it can build its responses. From there, BERT can adapt to the ever-growing body of searchable content and queries, and it can be fine-tuned to a user's specifications. This process is known as <a href="https://www.techtarget.com/searchcio/definition/transfer-learning">transfer learning</a>. Aside from this pretraining process, BERT has multiple other aspects it relies on to function as intended, including the following:</p> <h3>Transformers</h3> <p>Google's work on transformers made BERT possible. The transformer is the part of the model that gives BERT its increased capacity for understanding context and ambiguity in language. The transformer processes any given word in relation to all other words in a sentence, rather than processing them one at a time. By looking at all surrounding words, the transformer enables BERT to understand the full context of the word and therefore better understand searcher intent.</p> <p>This is contrasted against the traditional method of language processing, known as word embedding. This approach was used in models such as GloVe and word2vec. It would map every single word to a <a href="https://www.techtarget.com/whatis/definition/vector">vector</a>, which represented only one dimension of that word's meaning.</p> <h3>Masked language modeling</h3> <p>Word embedding models require large data sets of <a href="https://www.techtarget.com/whatis/definition/structured-data">structured data</a>. While they are adept at many general NLP tasks, they fail at the context-heavy, predictive nature of question answering because all words are in some sense fixed to a vector or meaning.</p> <p>BERT uses an MLM method to keep the word in focus from seeing itself, or having a fixed meaning independent of its context. BERT is forced to identify the masked word based on context alone. In BERT, words are defined by their surroundings, not by a prefixed identity.</p> <h3>Self-attention mechanisms</h3> <p>BERT also relies on a self-attention mechanism that captures and understands relationships among words in a sentence. The bidirectional transformers at the center of BERT's design make this possible. This is significant because often, a word may change meaning as a sentence develops. Each word added augments the overall meaning of the word the NLP algorithm is focusing on. The more words that are present in each sentence or phrase, the more ambiguous the word in focus becomes. BERT accounts for the augmented meaning by reading bidirectionally, accounting for the effect of all other words in a sentence on the focus word and eliminating the left-to-right momentum that biases words towards a certain meaning as a sentence progresses.</p> <figure class="main-article-image half-col" data-img-fullsize="https://www.techtarget.com/rms/onlineImages/whatis-bert-h.png"> <img data-src="https://www.techtarget.com/rms/onlineImages/whatis-bert-h_half_column_mobile.png" class="lazy" data-srcset="https://www.techtarget.com/rms/onlineImages/whatis-bert-h_half_column_mobile.png 960w,https://www.techtarget.com/rms/onlineImages/whatis-bert-h.png 1280w" alt="Diagram showing how BERT identifies meanings of words" height="387" width="279"> <figcaption> <i class="icon pictures" data-icon="z"></i>BERT examines individual words in context to determine the meaning of ambiguous language. </figcaption> <div class="main-article-image-enlarge"> <i class="icon" data-icon="w"></i> </div> </figure> <p></p> <p>For example, in the image above, BERT is determining which prior word in the sentence the word "it" refers to, and then using the self-attention mechanism to weigh the options. The word with the highest calculated score is deemed the correct association. In this example, "it" refers to "animal", not "street". If this phrase was a search <a href="https://www.techtarget.com/searchdatamanagement/definition/query">query</a>, the results would reflect this subtler, more precise understanding BERT reached.</p> <h3>Next sentence prediction</h3> <p>NSP is a training technique that teaches BERT to predict whether a certain sentence follows a previous sentence to test its knowledge of relationships between sentences. Specifically, BERT is given both sentence pairs that are correctly paired and pairs that are wrongly paired so it gets better at understanding the difference. Over time, BERT gets better at predicting next sentences accurately. Typically, both NSP and MLM techniques are used simultaneously.</p> </section> <section class="section main-article-chapter" data-menu-title="What is BERT used for?"> <h2 class="section-title"><i class="icon" data-icon="1"></i>What is BERT used for?</h2> <p>Google uses BERT to optimize the interpretation of user search queries. BERT excels at functions that make this possible, including the following:</p> <ul class="default-list"> <li>Sequence-to-sequence language generation tasks such as:</li> </ul> <ul class="default-list"> <li style="list-style-type: none;"> <ul style="list-style-type: circle;" class="default-list"> <li>Question answering.</li> <li>Abstract summarization.</li> <li>Sentence prediction.</li> <li>Conversational response generation.</li> </ul> </li> </ul> <ul class="default-list"> <li>NLU tasks such as:</li> </ul> <ul class="default-list"> <li style="list-style-type: none;"> <ul style="list-style-type: circle;" class="default-list"> <li>Polysemy and coreference resolution. Coreference means words that sound or look the same but have different meanings.</li> <li>Word sense disambiguation.</li> <li>Natural language inference.</li> <li>Sentiment classification.</li> </ul> </li> </ul> <p>BERT is open source, meaning anyone can use it. Google claims that users can train a state-of-the-art question-and-answer system in just 30 minutes on a cloud tensor processing unit, and in a few hours using a <a href="https://www.techtarget.com/searchvirtualdesktop/definition/GPU-graphics-processing-unit">graphic processing unit</a>. Many other organizations, research groups and separate factions of Google are fine-tuning the model's architecture with supervised training to either optimize it for efficiency or specialize it for specific tasks by pretraining BERT with certain contextual representations. Examples include the following:</p> <ul class="default-list"> <li><b>PatentBERT.</b> This BERT model is fine-tuned to perform patent classification tasks.</li> <li><b>DocBERT.</b> This model is fine-tuned for document classification tasks.</li> <li><b>BioBERT.</b> This biomedical language representation model is for biomedical <a href="https://www.techtarget.com/searchbusinessanalytics/definition/text-mining">text mining</a>.</li> <li><b>VideoBERT.</b> This joint visual-linguistic model is used in unsupervised learning of unlabeled data on YouTube.</li> <li><b>SciBERT.</b> This model is for scientific text.</li> <li><b>G-BERT.</b> This pretrained BERT model uses medical codes with hierarchical representations through <a href="https://www.techtarget.com/searchenterpriseai/definition/graph-neural-networks-GNNs">graph neural networks</a> and then fine-tuned for making medical recommendations.</li> <li><b>TinyBERT by Huawei.</b> This smaller, "student" BERT learns from the original "teacher" BERT, performing transformer distillation to improve efficiency. TinyBERT produced promising results in comparison to BERT-base while being 7.5 times smaller and 9.4 times faster at inference.</li> <li><b>DistilBERT by Hugging Face.</b> This smaller, faster and cheaper version of BERT is trained from BERT, then certain architectural aspects are removed to improve efficiency.</li> <li><b>ALBERT.</b> This lighter version of BERT lowers memory consumption and increases the speed with which the model is trained.</li> <li><b>SpanBERT.</b> This model improved BERT's ability to predict spans of text.</li> <li><b>RoBERTa. </b>Through more advanced training methods, this model was trained on a bigger data set for a longer time to improve performance.</li> <li><b>ELECTRA. </b>This version has been tailored to generate high-quality representations of text.</li> </ul> <figure class="main-article-image full-col" data-img-fullsize="https://www.techtarget.com/rms/onlineimages/the_trend_toward_smaller_language_models-f.png"> <img data-src="https://www.techtarget.com/rms/onlineimages/the_trend_toward_smaller_language_models-f_mobile.png" class="lazy" data-srcset="https://www.techtarget.com/rms/onlineimages/the_trend_toward_smaller_language_models-f_mobile.png 960w,https://www.techtarget.com/rms/onlineimages/the_trend_toward_smaller_language_models-f.png 1280w" alt="Quotes from AI experts on the rise of smaller language models." height="314" width="560"> <figcaption> <i class="icon pictures" data-icon="z"></i>Smaller language models, like the more optimized versions of BERT, are becoming more commonplace. </figcaption> <div class="main-article-image-enlarge"> <i class="icon" data-icon="w"></i> </div> </figure> </section> <section class="section main-article-chapter" data-menu-title="BERT vs. generative pre-trained transformers (GPT)"> <h2 class="section-title"><i class="icon" data-icon="1"></i>BERT vs. generative pre-trained transformers (GPT)</h2> <p>While BERT and GPT models are among the <a href="https://www.techtarget.com/whatis/feature/12-of-the-best-large-language-models">best language models</a>, they exist for different reasons. The initial <a href="https://www.techtarget.com/searchenterpriseai/definition/GPT-3">GPT-3 model</a>, along with OpenAI's subsequent more advanced GPT models, are also language models trained on massive data sets. While they share this in common with BERT, BERT differs in multiple ways.</p> <h3>BERT</h3> <p>Google developed BERT to serve as a bidirectional transformer model that examines words within text by considering both left-to-right and right-to-left contexts. It helps computer systems understand text as opposed to creating text, which GPT models are made to do. BERT excels at NLU tasks as well as performing sentiment analysis. It's ideal for Google searches and customer feedback.</p> <h3>GPT</h3> <p>GPT models differ from BERT in both their objectives and their use cases. GPT models are forms of generative AI that generate original text and other forms of content. They're also well-suited for summarizing long pieces of text and text that's hard to interpret.</p> <p><i>BERT and other language models differ not only in scope and applications but also in architecture. Learn more about </i><a href="https://www.techtarget.com/searchenterpriseai/feature/Exploring-GPT-3-architecture"><i>GPT-3's architecture</i></a><i> and how it's different from BERT.</i></p> </section> BERT language model is an open source machine learning framework for natural language processing (NLP). https://cdn.ttgtmedia.com/visuals/digdeeper/3.jpg https://www.techtarget.com/searchenterpriseai/definition/BERT-language-model Thu, 15 Feb 2024 18:14:00 GMT <section class="section main-article-chapter" data-menu-title="What is cognitive search?"> <h2 class="section-title"><i class="icon" data-icon="1"></i>What is cognitive search?</h2> <p>Cognitive search represents a new generation of enterprise search that uses artificial intelligence (<a href="https://www.techtarget.com/searchenterpriseai/definition/AI-Artificial-Intelligence">AI</a>) technologies to improve users' search queries and extract relevant information from multiple diverse data sets. Cognitive search capabilities extend beyond those of a classic search engine to bring numerous data sources together while also providing <a href="https://www.techtarget.com/searchcontentmanagement/tip/AI-in-content-management-supports-tagging-search">automated tagging</a> and personalization. It has the potential to greatly improve how an organization's employees discover and access information that's relevant and necessary to their work context.</p> <p>Cognitive search differs from previously available search products because it combines indexing technology with powerful AI technologies -- such as natural language processing (<a href="https://www.techtarget.com/searchbusinessanalytics/definition/natural-language-processing-NLP">NLP</a>) capabilities and <a href="https://www.techtarget.com/whatis/definition/algorithm">algorithms</a> -- to scale a variety of data sources and types. Additionally, developers can build search applications that can be embedded into business process applications, such as pharmaceutical research tools and customer portals.</p> <p>The primary benefits that organizations can reap from cognitive search include its impact on <a href="https://www.techtarget.com/searchenterpriseai/feature/KDD-in-data-mining-assists-data-prep-for-machine-learning">knowledge discovery</a> -- a user's ability to extract useful information from data. For example, cognitive search improves the relevance of extracted information and increases the efficiency of query responses, allowing employees to boost their productivity and provide better service.</p> </section> <section class="section main-article-chapter" data-menu-title="Importance and benefits of cognitive search"> <h2 class="section-title"><i class="icon" data-icon="1"></i>Importance and benefits of cognitive search</h2> <p>Keyword-based search and traditional enterprise search have become inadequate due to the increasing variety and amount of data used within organizations. The two methodologies impair search processes and employee productivity by returning irrelevant or incomplete results that users must sort through to find the information they need.</p> <p>With cognitive search, the AI technologies that are introduced enable enterprise search to extract advanced meaning from content as well as learn from users' searches to provide increasingly relevant and complete results. Some overall benefits of cognitive search include:</p> <ul class="default-list"> <li><strong>Maximized productivity.</strong> A single search functionality removes the necessity of switching between apps and eliminates time wasted on tasks like re-entering credentials multiple times. Furthermore, the unification of data tools allows organizations to streamline their business processes.</li> <li><strong>Improved employee experience and engagement. </strong>Employee loyalty is promoted through the elimination of wasted time and the increase in productivity. Machine learning (<a href="https://www.techtarget.com/searchenterpriseai/definition/machine-learning-ML">ML</a>) algorithms that provide personalized suggestions help users find relevant data more quickly and the flexibility of cognitive search creates an improved user experience through personalization. Since an employee's search experience is improved, they're more likely to use the tools consistently.</li> <li><strong>Lower operational costs. </strong>Maximized productivity decreases an organization's operational costs since less time and resources are needed for gathering information and knowledge discovery. This is especially beneficial to industries such as healthcare and legal services that work with massive amounts of data.</li> </ul> <p>As companies grow and acquire new customers, the need to run and analyze large amounts of data increases as well. If a company is bringing in thousands of new customers every day, then their data growth is exponential, making it almost impossible to keep up with the new information. Cognitive search makes it feasible to decipher a consistently growing collection of data for use within different departments of the company.</p> </section> <section class="section main-article-chapter" data-menu-title="How does cognitive search work?"> <h2 class="section-title"><i class="icon" data-icon="1"></i>How does cognitive search work?</h2> <p>The design elements used in enterprise search form the foundation of cognitive search. This means that organizations do not need to entirely rebuild their information technology (IT) department when implementing cognitive search. AI technologies then build on top of this foundation to find relevant information across all available enterprise data sources.</p> <p>NLP is used to understand what unstructured data from emails, documents, market research, videos and recordings mean. ML algorithms continuously improve the relevancy of results. Some of the most common ML algorithms found in cognitive search include:</p> <ul class="default-list"> <li><strong>Clustering.</strong> This is an <a href="https://www.techtarget.com/whatis/definition/unsupervised-learning">unsupervised learning</a> algorithm that groups subsets of data based on similarities. Clustering is employed when users do not want to run a search through the entire search index. Its goal is to limit searches to specific groups of documents in each cluster.</li> <li><strong>Classification. </strong>This is a <a href="https://www.techtarget.com/searchenterpriseai/definition/supervised-learning">supervised learning</a> algorithm that creates a model to predict labels for new data using a training set comprised of pre-labeled data.</li> <li><strong>Regression. </strong>This is another supervised learning algorithm that uses the relationship between input and output variables to predict continuous numeric values from data.</li> <li><strong>Recommendation.</strong> This often combines various basic algorithms to generate a recommendation engine that offers potentially helpful content to users. Also called content-based recommendation, it offers personalized recommendations based on the relationship between a user's interest and the description and attributes of a document.</li> </ul> <p>In addition to these ML algorithms, a heavy computing process, referred to as <em>similarity</em>, builds a matrix that synthesizes the interactions between data samples.</p> </section> <section class="section main-article-chapter" data-menu-title="Cognitive search tools"> <h2 class="section-title"><i class="icon" data-icon="1"></i>Cognitive search tools</h2> <p>Cognitive search is still in its infancy, but multiple companies have jumped on the opportunity to create and market cognitive search tools. They include the following:</p> <ul class="default-list"> <li>IBM with Watson Explorer.</li> <li>Coveo.</li> <li>Attivio with their Cognitive Search and Insight Platform.</li> <li>Lucidworks.</li> <li>Mindbreeze.</li> <li>Sinequa with their Insight Platform.</li> <li>Microsoft Azure Cognitive Search.</li> <li>Algolia.</li> </ul> </section> <section class="section main-article-chapter" data-menu-title="Examples of cognitive search"> <h2 class="section-title"><i class="icon" data-icon="1"></i>Examples of cognitive search</h2> <p>Legal practices with international exposure are finding cognitive search useful by implementing legal industry-specific add-ons which help find experts in specific areas of law. These experts can then be organized into specialized teams across a firm's international offices.</p> <p>Cognitive search is also finding beneficial applications within customer service. Representatives can access multiple applications and widespread digital content simultaneously, including everything from shipping information to product details. This allows them to respond to customer requests and resolve problems more efficiently.</p> <p>Within IT operations, cognitive systems can consistently monitor <a href="https://www.techtarget.com/whatis/definition/log-log-file">log files</a> that indicate faulty builds or misuse of the network. <a href="https://www.techtarget.com/whatis/definition/telemetry">Telemetry</a> data can also be scanned to find irregular activity that could warn of a potential outage.</p> </section> Cognitive search represents a new generation of enterprise search that uses artificial intelligence (AI) technologies to improve users' search queries and extract relevant information from multiple diverse data sets. https://cdn.ttgtmedia.com/visuals/digdeeper/5.jpg https://www.techtarget.com/searchenterpriseai/definition/cognitive-search Thu, 11 May 2023 13:28:00 GMT 60 webmaster@techtarget.com