Big data management
Big data management is the organization, administration and governance of large volumes of both structured and unstructured data.. See complete definition.
Top Stories
-
Tip
17 Oct 2022
Make data usability a priority on data quality for big data
To help make big data analytics applications more effective, IT teams must augment conventional data quality processes with measures aimed at improving data usability for analysts. Continue Reading
-
News
02 Feb 2022
Onehouse emerges with managed Apache Hudi data lake service
One of the original creators of the Hudi project at Uber has launched a new company set to bring a managed service to market to operationalize cloud data lakes. Continue Reading
-
News
30 Mar 2018
Oracle Autonomous Database debut heralds new cloud hopes
It wasn't the first in the cloud computing space, but Oracle is all in now. Key to the effort is a new Autonomous Database service, initially available for data warehousing uses. Continue Reading
-
Podcast
29 Mar 2018
Kubernetes container orchestration gets big data star turn
The new thing in big data is Kubernetes container orchestration. While it's still early, there are signs of activity, which are cited in this edition of the Talking Data podcast. Continue Reading
-
Feature
21 Mar 2018
Kubernetes gains momentum in big data implementation process
Big data vendors and users are looking to Kubernetes-managed containers to help accelerate deployments and enable more flexible use of computing resources. Continue Reading
-
Definition
01 Mar 2018
U-SQL
U-SQL is a Microsoft query language that combines a declarative SQL-like syntax with C# programming, enabling it to be used to process both structured and unstructured data in big data environments. Continue Reading
-
Definition
21 Feb 2018
TensorFlow
TensorFlow is an open source framework developed by Google researchers to run machine learning, deep learning and other statistical and predictive analytics workloads. Continue Reading
-
News
16 Feb 2018
MongoDB 4.0 takes ACID transactions to multi-document level
MongoDB is taking a deeper step into SQL-style processing waters with a 4.0 update that brings increased support for ACID-compliant transactions to its NoSQL database. Continue Reading
-
Feature
06 Feb 2018
Data catalog software takes open data initiative to the streets
Data catalog software from Alation helped San Diego's chief data officer open up the city's data to residents, as well as city planners and other government workers. Continue Reading
-
Opinion
02 Feb 2018
Data managers should study up on GPU deep learning
As GPU deep learning becomes more common, data managers will have to navigate several new layers of complexity in their quest to build or buy suitable data infrastructure. Continue Reading
-
Tip
01 Feb 2018
Three ways to turn old files into Hadoop data sets in a data lake
Hadoop data lakes offer a new home for legacy data that still has analytical value. But there are different ways to convert the data for use in Hadoop depending on your analytics needs. Continue Reading
-
Podcast
23 Jan 2018
How AI and IoT will influence data management in 2018
AI and IoT will alter the data management landscape in 2018, according to analyst James Kobielus. AI will need regular updates, and DevOps will become more prevalent as a result. Continue Reading
-
Feature
15 Jan 2018
Machine learning models require DevOps-style workflows
Big data is driving the use of AI and machine learning. But teams must be swift to embrace DevOps and re-evaluate models, according to Wikibon's James Kobielus. Continue Reading
-
News
03 Jan 2018
Apache Hadoop 3.0 goes GA, adds hooks for cloud and GPUs
Is this the post-Hadoop era? Not in the eyes of Hadoop 3.0 backers, who see the latest update to the big data framework succeeding in machine learning applications and cloud systems. Continue Reading
-
Feature
19 Oct 2017
NewSQL databases rise anew -- MemSQL, Spanner among contenders
The NewSQL database was almost hidden when Hadoop and NoSQL arose. Now, as more big data teams move toward production uses, MemSQL, Cloud Spanner and similar products may get a second look. Continue Reading
-
Feature
21 Aug 2017
Data management processes take on a new tenor in analyst's view
Changes in data management processes -- including self-service data preparation, data lakes and real-time analytics -- create a new landscape in companies, according to analyst Matt Aslett. Continue Reading
-
Feature
10 Aug 2017
Find your niche in big data: Consulting, software, platforms
Channel partners looking for a niche in big data may find opportunities in consulting, predictive analytics system deployment and software implementation. Continue Reading
-
News
24 Jul 2017
Chief data officer jobs call for nurturing data ethos in companies
In many organizations, chief data officer jobs centered on defense against risk are giving way to ones emphasizing innovation. To do so, CDOs must nurture a data culture, MIT panelists said. Continue Reading
-
Quiz
11 Jul 2017
Quiz: What's your IQ on data lake implementation and management?
A data lake can provide lots of analytics value to an organization. This quiz can tell you if you know what it takes to deploy and manage one; it also contains info to help boost your knowledge. Continue Reading
-
News
02 Jun 2017
Doug Cutting, ‘father’ of Hadoop, talks about big data tech evolution
In a Q&A interview, the “father” of Hadoop, Doug Cutting, talks about the cyber-security applications of the stack, as well as Hadoop’s evolution Continue Reading
-
Feature
13 Mar 2017
Silicon Valley startups aim to make big data capture and prep slicker
A group of California-based startup and early-stage data analytics and management companies are bidding to make big data, including sensor data, more tractable for analysis Continue Reading
-
News
15 Sep 2016
HPE sells Vertica analytics, thanks to the growth of open source software
HPE is paring down its software holdings, including analytical database software in the Vertica line and other big data tools. A sale to Micro Focus is due to close next year, leaving users in some limbo for now. Continue Reading
-
Tip
19 Jul 2016
Salesforce rallies in retail with purchase of Demandware e-commerce
Now that Salesforce has purchased Demandware, it's positioned to compete with vendors like Oracle and even Amazon for market share. Continue Reading
-
Feature
26 Apr 2016
Inside the MapR Hadoop distribution for managing big data
The MapR Hadoop distribution replaces HDFS with its proprietary file system, MapR-FS, which is designed to provide more efficient management of data, reliability and ease of use. Continue Reading
-
Feature
22 Apr 2016
CW@50: Data management – Five decades of prospecting for business value
Data management and business intelligence have been at the heart of business value creation for decades. Read about how Computer Weekly has tracked their promise and tribulations Continue Reading
-
Feature
29 Oct 2015
Three ways to build a big data system
In a book excerpt, author Dale Neef outlines and compares different approaches organizations can take when trying to bring a big data system into their IT environments. Continue Reading
-
Podcast
25 Aug 2015
The latest trends in the Hadoop project
In this podcast, Tim Hall of Hortonworks discusses the Apache Hadoop project and how it has changed since its inception. Continue Reading
-
Feature
14 Aug 2015
Types of NoSQL databases and key criteria for choosing them
In a book excerpt, consultant Dan Sullivan offers insights into how to select the right type of NoSQL database for the right application in your organization. Continue Reading
-
Feature
20 Jul 2015
Business cases emerge from growing pains of Hadoop 1.0
The hype around the early generation of Hadoop is giving way to the reality of business programmes based on Hadoop 2 and Apache Spark Continue Reading
-
Quiz
11 Jun 2015
Apache Spark architecture quiz: Do you know Spark?
Apache Spark is an open source processing engine designed for use in big data applications. Take this quiz to see if you're up to speed on Spark's history and features. Continue Reading
-
Feature
11 Jun 2015
Which in-memory DBMS best fits your company's needs?
Database expert Craig S. Mullins examines some top in-memory DBMS software offerings to help you determine which is the best fit for your organization. Continue Reading
-
E-Zine
07 May 2015
Sort through types of database management systems before you buy
The database management system lies at the center of any organization's business activities. It houses the data that pumps life into operations and analytics applications and ensures that it's managed and manipulated securely. But the types of database management systems, or DBMSes, are numerous -- and organizations starting the evaluation process need to first examine the data they need to process, says data management consultant Craig S. Mullins in this special edition of Business Information. Is it traditional, structured data? Or is it a vast pool full of audio and video streams, social media data and other unstructured information?
First, Mullins surveys the database product landscape, detailing the advantages of the types of database management systems on the market, breaking them into three main categories: relational database management systems, NoSQL databases and in-memory databases. Next Mullins takes a close look at relational systems. Any organization shopping for a database system should begin here, he says, because relational databases are good for a wide range of uses -- traditional data management and analytical applications, batch processing and more. Finally, Mullins explores the relatively new world of NoSQL database systems, known for their able handling of big data. NoSQL has its own types of database management systems, falling into four categories: key-value, document, column and graph. Each is suitable for particular uses, so companies may want to consider mixing and matching, depending on their data needs. Continue Reading -
Opinion
01 May 2015
With database management tools, never settle
They're the heart of it all, pumping data to applications and business processes. But the sheer number of products on the market can boggle any mind. Continue Reading
-
Feature
30 Apr 2015
Which relational DBMS is best for your company?
Database expert Craig S. Mullins examines the top relational database management system software to determine which best fits your organization. Continue Reading
-
Feature
24 Apr 2015
Apache Spark speeds up big data decision-making
Spark, the open-source cluster computing framework from Apache, promises to complement Hadoop batch processing Continue Reading
-
Feature
17 Apr 2015
How to determine if an in-memory DBMS is right for your company
Database expert Craig S. Mullins examines the pros and cons of the in-memory database management system, and the criteria you should consider during the request-for-proposal and evaluation period. Continue Reading
-
Feature
09 Jan 2015
Big data and analytics advice from an operations pro
You don't need to employ special IT personnel for big data analytics. One company uses big data for developers to modify ads and find and adjust pain-points in a game application. Continue Reading
-
News
04 Jun 2014
Apache Spark goes 1.0, looks to improve on MapReduce performance
The Apache Software Foundation released Version 1.0 of Spark, a processing framework aimed at outpacing MapReduce for machine learning and other uses. Continue Reading
-
Opinion
30 May 2014
Anxieties about big data technology trends are valid
Criticism of big data technology trends may go too far, but proponents shouldn't dismiss the critiques. Continue Reading
-
Opinion
29 May 2014
Internet of Things applications put varied machine data to work
The Internet of Things is proving to be a hotbed of diverse data and new applications fueled by information from sensors and other machine sources. Continue Reading
-
Answer
29 May 2014
Seven best practices to boost big data governance efforts
Governing big data is much like governing other data, though businesses must adjust governance processes to accommodate larger, more varied data sets. Continue Reading
-
News
08 May 2014
Big data charge could turn data management profession on its head
Discussed at DAMA's EDW 2014 were more flexible approaches to the data management profession. That's in response to growing mountains of diverse data. Continue Reading
-
Opinion
28 Apr 2014
Big data cloud is changing familiar database tech tune
New cloud database applications that increasingly fall into the "big-data-intensive" category are shaking up the usual build-or-buy soliloquy. Continue Reading
-
Feature
17 Apr 2014
Data collection practices spark debate on big data ethics, privacy
The increasing emphasis on collecting and analyzing big data is driving a debate on whether tighter privacy rules are needed to protect customers. Continue Reading
-
Feature
14 Apr 2014
Supermarket co-op stocks up on big data platform to help spur sales
Co-operative Allegiance Retail Services is deploying a cloud-based big data stack to replace a homegrown system that fell short on analytics power. Continue Reading
-
Feature
31 Mar 2014
Selecting the right SQL-on-Hadoop engine to access big data
Rick van der Lans explains why it's important to evaluate the differences in the technologies that make it possible to access Hadoop data using SQL. Continue Reading
-
Feature
27 Feb 2014
Hadoop 2 opens new vistas beyond current UK data management practice
Hadoop 2 enables a greater range of data-crunching tasks, beyond Hadoop’s previous scope of batch processing, directly within the stack itself Continue Reading
-
News
09 Sep 2013
Survey: Execs buying into big data in a big way
TechTarget's Wayne Eckerson discusses survey results showing that two-thirds of respondents expect to spend more than $1 million on big data programs this year. Continue Reading
-
Feature
29 Aug 2013
Big data adds new complications for data stewards
In order to accommodate big data and the needs of data scientists, data stewards will need to adjust to a more short-term view of data stewardship. Continue Reading
-
Tip
22 Apr 2013
Real-time querying in Hadoop? Not so fast
Hadoop vendors are adding real-time query engines to the open source software. But the jury is out on that effort, writes Wayne Eckerson. Continue Reading
-
Feature
04 Feb 2013
Without analytics, it's 'big data, big schmada'
Data -- including big data -- has no value by itself, says TechTarget's Wayne Eckerson. It's what you do with data to gain business insights that counts. Continue Reading
-
Feature
15 Nov 2012
Hadoop World: Just like data warehousing's early days
The buzz-filled Hadoop World 2012 conference reminded TechTarget's Wayne Eckerson of a data warehousing event he attended in 1997. Still to be seen: Can Hadoop live up all to the hype? Continue Reading
-
Feature
05 Nov 2012
Unilever enterprise data warehouse locked to business programmes
Unilever is building a global enterprise data warehouse (EDW) locked into the critical paths of high-profile business programmes. Continue Reading
-
News
30 Oct 2012
Teradata brings big data into the SQL family
Teradata addresses the issue of combining unstructured, semi-structured and traditional row-and-column data warehousing in a single architecture. Continue Reading
-
Feature
08 Aug 2012
Mission impossible? Data governance process takes on 'big data'
Effective governance can help companies get the most out of their "big data" environments. But at this point, there's no formula for how to do that. Continue Reading
-
Tip
25 May 2012
Don't run from 'big data' tools, but approach with caution
There's nothing magic about "big data" technologies such as Hadoop and NoSQL databases, says BeyeNetwork expert Barry Devlin, who poses a series of questions to ask before deploying them. Continue Reading
-
Tip
24 May 2012
Why 'big data' is like seawater for text analytics
You can't just drink water from the ocean, writes BeyeNetwork expert Bill Inmon. Similarly, he says, unstructured text in "big data" pools must be refined before it's used in analytics applications. Continue Reading
-
News
26 Mar 2012
'Big data' breathes new life into CEP software
Complex event processing (CEP) technology is poised to move from the backwaters of data management to center stage, says TechTarget's Wayne Eckerson. The primary driver? "Big data." Continue Reading
-
News
06 Mar 2012
'Big data' hype and reality on full display at Strata
The Strata Conference was a showcase for Hadoop and NoSQL, which are gaining bigger footholds in corporate IT. But they're no panacea for managing data, says TechTarget's Wayne Eckerson. Continue Reading
-
News
27 Feb 2012
Categorizing 'big data' processing systems
In the past, analytics applications typically were powered by relational databases. Now the options are more varied and less straightforward, says TechTarget's Wayne Eckerson. Continue Reading
-
News
20 Feb 2012
New analytical ecosystem: Making way for 'big data'
The "big data" revolution has arrived and it's transforming long-established data warehousing architectures into multifaceted analytical ecosystems, says TechTarget's Wayne Eckerson. Continue Reading
-
Photo Story
16 Feb 2012
Top six data management software stories of 2011
From the rise of "big data" to office politics hampering MDM projects to Hewlett-Packard giving Neoview the ax, and more, there were plenty of notable data management stories in 2011. Continue Reading
-
News
27 Jan 2012
'Big data' liberation theology -- and reality
Data warehouses can do just about anything that "big data" technologies can do, says TechTarget's Wayne Eckerson. The big data movement's real weapon, he contends, is open source software. Continue Reading