Browse Definitions :
Definition

black box AI

Black box AI is any artificial intelligence system whose inputs and operations are not visible to the user or another interested party. A black box, in a general sense, is an impenetrable system. 

Deep learning modeling is typically conducted through black box development: The algorithm takes millions of data points as inputs and correlates specific data features to produce an output. That process is largely self-directed and is generally difficult for data scientists, programmers and users to interpret.

When the workings of software used for important operations and processes within an organization cannot easily be viewed or understood, errors can go unnoticed until they cause problems so large that it becomes necessary to investigate and the damage caused may be expensive or even impossible to repair. 

AI bias, for example, can be introduced to algorithms as a reflection of conscious or unconscious prejudices on the part of the developers, or they can creep in through undetected errors. In any case, the results of a biased algorithm will be skewed, potentially in a way that is offensive to people who are affected. Bias in an algorithm may come from training data when details about the dataset are unrecognized. In one situation, AI used in a recruitment application relied upon historical data to make selections for IT professionals. However, because most IT staff historically were male, the algorithm displayed a bias toward male applicants. 

If such a situation arises from black box AI, it may persist long enough for the organization to incur damage to its reputation and, potentially, legal actions for discrimination. Similar issues could occur with bias against other groups as well, with the same effects.To prevent such harms, it's important for AI developers to build transparency into their algorithms and for organizations to commit to accountability for their effects. 

This was last updated in August 2019

Continue Reading About black box AI

Networking
  • Network as a Service (NaaS)

    Network as a service, or NaaS, is a business model for delivering enterprise WAN services virtually on a subscription basis.

  • network configuration management (NCM)

    Network configuration management is the process of organizing and maintaining information about all of the components in a ...

  • presentation layer

    The presentation layer resides at Layer 6 of the Open Systems Interconnection (OSI) communications model and ensures that ...

Security
  • backdoor (computing)

    A backdoor attack is a means to access a computer system or encrypted data that bypasses the system's customary security ...

  • Heartbleed

    Heartbleed was a vulnerability in some implementations of OpenSSL, an open source cryptographic library.

  • What is risk management and why is it important?

    Risk management is the process of identifying, assessing and controlling threats to an organization's capital and earnings.

CIO
HRSoftware
  • team collaboration

    Team collaboration is a communication and project management approach that emphasizes teamwork, innovative thinking and equal ...

  • employee self-service (ESS)

    Employee self-service (ESS) is a widely used human resources technology that enables employees to perform many job-related ...

  • learning experience platform (LXP)

    A learning experience platform (LXP) is an AI-driven peer learning experience platform delivered using software as a service (...

Customer Experience
  • headless commerce (headless e-commerce)

    Headless commerce, also called headless e-commerce, is a platform architecture that decouples the front end of an e-commerce ...

  • chief customer officer (CCO)

    A chief customer officer, or customer experience officer, is responsible for customer research, communicating with company ...

  • relationship marketing

    Relationship marketing is a facet of customer relationship management (CRM) that focuses on customer loyalty and long-term ...

Close