Browse Definitions :

data lifecycle

What is a data lifecycle?

A data lifecycle is the sequence of stages that a particular unit of data goes through from its initial generation or capture to its eventual archival and/or deletion at the end of its useful life.

Although specifics vary, data management experts often identify six or more stages in the data lifecycle, as noted in the following example and flowchart:

  1. Determine the need. Before generating new data or capturing existing data, ensure that the need for such data is established and confirmed.
  2. Generation or capture. In this phase, data comes into an organization, usually through data entry, acquisition from an external source or signal reception, such as transmitted sensor data.
  3. Data preparation for use. In this phase, data is processed to prepare it to be used. The data might be subjected to processes such as integration, scrubbing and extract, transform, and load (ETL). It might also be encrypted to ensure its security and privacy.
  4. Active use. In this phase, data is used to support the organization's objectives and operations.
  5. Management. During the management phase, data might be made available to the broader public or retained inside the organization. It can be stored in a short-term storage platform for continued availability while its security and privacy is constantly maintained.
  6. Long-term storage, retention or archiving. In this phase, data is removed from all active production environments and moved to an archive. It is no longer processed, used or published, but it is stored in case it is needed again in the future. The data might be stored on hard disk drives (HDDs), solid-state drives (SSDs) or tape in-house or with a cloud storage service.
  7. Purging and destruction. When the data becomes obsolete, every copy of data is deleted and destroyed as part of a removal process. The data might have been archived or it is data that is no longer needed or has been superseded by newer data. The data destruction process might also include the media on which the data resides.
Flowchart illustrating data lifecycle

In this data lifecycle flowchart, a step is added during which the need for the data is determined and validated. Considering how much data is generated every hour, this step is important to minimize the collection of too much data that may or may not be used in the organization (and may overload storage devices). Considering that archived data might be required again for legal and/or audit review or other purposes, a step has been added to identify the retrieval and reuse of archived data.

What is the importance of data lifecycle management?

Data lifecycle management (DLM) is becoming increasingly important because big data analytics have become mainstream and the ongoing development of the internet of things (IoT). It is also a key element in data governance.

Enormous volumes of data are being generated by an ever-increasing number of devices, so proper oversight of data throughout its lifecycle is essential to optimize its usefulness and minimize the potential for errors. Finally, archiving or deleting data at the end of its useful life ensures that it does not consume more resources than necessary.

Top automated data lifecycle parts
Augmented data lifecycle management employs AI and machine learning to bring self-configuring and self-turning data management.

DLM also ensures that data handling addresses the following security and privacy goals:

  • Confidentiality of the data. Data, especially sensitive data, is stored in secure environments so that unauthorized individuals cannot access it.
  • Integrity of the data. The data is managed and stored in a way that the data content is unchanged and protected from corruption, regardless of who uses the data or how many versions are generated.
  • Availability of the data. The data is available to only those individuals or entities who have a need to know and who have the appropriate level of security access.

DLM is also essential for establishing and maintaining compliance with key data security and privacy legislation, such as the General Data Protection Regulation (GDPR), Sarbanes-Oxley Act (SOX) and Health Insurance Portability and Accountability Act (HIPAA).

Achieving compliance with lifecycle management requirements

Considering the above regulations regarding data privacy, and the many others regarding data security (e.g., ISO 27001) at some point in time, especially during an IT audit, it will be necessary to demonstrate compliance with data lifecycle controls. Activities to perform in advance of audits and reviews include the following:

  • Ensure that policies regarding data management and lifecycle management are in place and up to date.
  • Examine data management procedures to ensure they are up to date and accurately reflect the current state of data management in the organization.
  • Generate copies of management reports that provide evidence of how data is managed, stored, archived and destroyed.
  • Prepare evidence of how/where data is stored and archived, as well as how it is destroyed, and how the security of the data is established and maintained; this can be organized with internal data management applications or with third-party solutions.

See how data governance and data management work together and what key roles a data management team should include.

This was last updated in February 2023

Continue Reading About data lifecycle

  • timing attack

    A timing attack is a type of side-channel attack that exploits the amount of time a computer process runs to gain knowledge about...

  • privileged identity management (PIM)

    Privileged identity management (PIM) is the monitoring and protection of superuser accounts that hold expanded access to an ...

  • possession factor

    The possession factor, in a security context, is a category of user authentication credentials based on items that the user has ...

  • Systems Modeling Language (SysML)

    Systems Modeling Language (SysML) helps teams design, develop, test and deploy complex physical systems.

  • business process reengineering (BPR)

    Business process reengineering (BPR) is a management practice in which business processes used are radically redesigned to ...

  • innovation management

    Innovation management involves the process of managing an organization's innovation procedure, starting at the initial stage of ...

  • employee resource group (ERG)

    An employee resource group is a workplace club or more formally realized affinity group organized around a shared interest or ...

  • employee training and development

    Employee training and development is a set of activities and programs designed to enhance the knowledge, skills and abilities of ...

  • employee sentiment analysis

    Employee sentiment analysis is the use of natural language processing and other AI techniques to automatically analyze employee ...

Customer Experience
  • customer profiling

    Customer profiling is the detailed and systematic process of constructing a clear portrait of a company's ideal customer by ...

  • customer insight (consumer insight)

    Customer insight, also known as consumer insight, is the understanding and interpretation of customer data, behaviors and ...

  • buyer persona

    A buyer persona is a composite representation of a specific type of customer in a market segment.