Hadoop framework
Top Stories
-
Feature
06 Jan 2023
18 top big data tools and technologies to know about in 2023
Numerous tools are available to use in big data applications. Here's a look at 18 popular open source technologies, plus additional information on NoSQL databases. Continue Reading
-
Feature
17 Feb 2022
Hadoop vs. Spark: An in-depth big data framework comparison
Hadoop and Spark are widely used big data frameworks. Here's a look at their features and capabilities and the key differences between the two technologies. Continue Reading
-
Guide
21 Aug 2019
GDPR, AI intensify privacy and data protection compliance demands
This guide covers the challenges data management teams face on data protection and privacy, particularly with the rise of GDPR and similar laws and the growing use of AI tools. Continue Reading
-
News
21 Aug 2019
SnapLogic boosts its data integration technology with AI
SnapLogic's latest update for its Intelligent Integration Platform aims to make it easier for users to benefit from data integration. Continue Reading
-
News
20 Aug 2019
DigitalOcean Managed Databases add MySQL, Redis support
Expanding its database management system offerings, cloud computing vendor DigitalOcean added Managed Databases support for MySQL and Redis. Continue Reading
-
Feature
20 Aug 2019
Enterprise data marketplace aims to ease self-service chaos
Self-service data preparation can duplicate work and slow down analytics. One possible fix: an internal marketplace where users can 'shop' for data assets. Continue Reading
-
News
15 Aug 2019
AWS Lake Formation goes GA, as cloud data lake market grows
First released in late 2018, AWS Lake Formation is now generally available and it could be the force that helps to accelerate adoption in the cloud data lake market. Continue Reading
-
News
13 Aug 2019
Apollo data graph brings managed federation to enterprises
The new Apollo update is intended to enable organizations to federate multiple enterprise data sets more easily and use APIs to tap into disparate data sources. Continue Reading
-
Feature
13 Aug 2019
Data management roles: Data architect vs. data engineer, others
Veteran data pro Michael Bowers differentiates between key data management positions, including their salaries and which ones can add the most business value. Continue Reading
-
News
08 Aug 2019
Elastic Stack 7.3 adds new features to expand data analysis
Elastic Stack, in a new update, adds data frames capabilities to enable new forms of analysis, as well as integrating more data sources that can be searched and analyzed. Continue Reading
-
News
06 Aug 2019
HPE buys MapR assets to fuel AI applications
Longtime independent big data vendor MapR goes out of business, selling technology and intellectual property to HPE. The move marks the continuing decline of the Hadoop market. Continue Reading
-
News
01 Aug 2019
Startup Dgraph Labs growing graph database technology
Dgraph Labs looks expand its graph database platform, which offers the promise of reducing isolated data sources and speeding up queries. Continue Reading
-
News
31 Jul 2019
Naveego launches tool for analyzing data quality and health
Naveego Accelerator enables users to check for inaccurate data, incomplete data or improperly formatted data to prevent inconsistencies that could cost businesses money. Continue Reading
-
News
31 Jul 2019
NuoDB 4.0 improves cloud-native database capabilities
Support for multiple clouds, tighter integration with Kubernetes and indexing enhancements that accelerate database performance highlight the new NuoDB release. Continue Reading
-
Feature
30 Jul 2019
Information Builders exec talks data management ethics
In this Q&A, James Cotton, director of the Data Management Centre of Excellence at Information Builders, offers advice on how organizations can carry out ethical data management. Continue Reading
-
Tip
30 Jul 2019
SQL Server database design best practices and tips for DBAs
Good database design is a must to meet processing needs in SQL Server systems. In a webinar, consultant Koen Verbeeck offered advice on how to make that happen. Continue Reading
-
News
30 Jul 2019
Hitachi Vantara updates Pentaho 8.3 to expand DataOps vision
Hitachi Vantara's new Pentaho update brings DataOps capabilities for data management to help organizations derive better data insights. Continue Reading
-
Tip
29 Jul 2019
SQL Server in Azure database choices and what they offer users
SQL Server databases can be moved to the Azure cloud in several different ways. Here's what you'll get from each of the options for migrating SQL Server to Azure. Continue Reading
-
News
26 Jul 2019
Cloudera open source route seeks to keep big data alive
Inspired by the IBM-Red Hat model, Cloudera goes the open source route to broaden its market as demand for Hadoop weakens and the vendor takes on big competitors like AWS. Continue Reading
-
Feature
24 Jul 2019
Key components of an effective data virtualization architecture
Experts break down which elements -- both technical and nontechnical -- are most crucial to successfully deploying and managing a data virtualization architecture. Continue Reading
-
Tip
23 Jul 2019
10 cloud database migration mistakes to avoid
Database expert Chris Foot lists the top 10 oversights IT teams commonly make when undertaking a cloud database migration and offers tips on how to avoid them. Continue Reading
-
Feature
19 Jul 2019
11 real-time data streaming roadblocks and how to overcome them
Experts detail common challenges that IT teams encounter when deploying and managing real-time data streaming platforms and offer advice on how to address them. Continue Reading
-
News
03 Jul 2019
Snowflake product VP expounds on data warehouse for the cloud
In this Q&A, Christian Kleinerman, vice president of product development at cloud data warehouse vendor Snowflake, talks about the fast-moving company's plans for the next year. Continue Reading
-
News
03 Jul 2019
RavenDB Cloud automates database management
RavenDB's managed cloud database service, RavenDB Cloud, intends to automate security processes, reduce overhead and cater to businesses of all sizes. Continue Reading
-
Feature
02 Jul 2019
Container technologies promise more agility for big data apps
Along with the ability to provide greater agility and flexibility for big data applications, containers can play a role in IT strategy that drives real-time decision-making. Continue Reading
-
Feature
02 Jul 2019
Using a LEFT OUTER JOIN vs. RIGHT OUTER JOIN in SQL
In this book excerpt, you'll learn LEFT OUTER JOIN vs. RIGHT OUTER JOIN techniques and find various examples for creating SQL queries that incorporate OUTER JOINs. Continue Reading
-
News
28 Jun 2019
PostgreSQL database specialist EnterpriseDB gets new backing
EnterpriseDB is looking to push its database further with help from new financial backers. The deal sees Postgres originator Michael Stonebraker coming onboard as technical adviser. Continue Reading
-
Feature
27 Jun 2019
Explore data integration products for your organization
Browse through the current data integration products currently available to help you determine which tool best suits your organization's needs. Continue Reading
-
Feature
27 Jun 2019
Data modeling techniques to overcome common business challenges
In this interview, author and data modeling instructor Steve Hoberman discusses techniques for dealing with challenges that may arise in the data modeling process. Continue Reading
-
Feature
26 Jun 2019
How 4 organizations are breaking down data silos
Siloed data continues to inhibit enterprise efficiency. Here, IT professionals discuss problems their organizations are facing around data silos -- and how they're solving them. Continue Reading
-
News
26 Jun 2019
Immuta Automated Data Governance Platform comes to AWS
Immuta's Automated Data Governance Platform, providing data visibility, accessibility and protection, is now available as a managed service on AWS Marketplace. Continue Reading
-
Tip
20 Jun 2019
Building leaner, meaner BI data sources
As business intelligence analysis and reporting platforms become increasingly important in the enterprise, so does the data that feeds them. Are your BI data sources up to par? Continue Reading
-
Feature
19 Jun 2019
A comparison of open source, real-time data streaming platforms
With so many real-time data streaming tools, how do you know which is right for your organization? Experts compare Spark Streaming, Kafka Streams, Flink and others. Continue Reading
-
Tip
18 Jun 2019
Data virtualization benefits seen in unified views, IT agility
Through in-place integration, data virtualization platforms can provide wider access to data and simplify security and governance. But they come with some limitations. Continue Reading
-
News
18 Jun 2019
MongoDB Atlas cloud service adds data lake, touts multi-cloud
MongoDB released an S3-compatible data lake its developer legions can quickly query. But, word of MongoDB Atlas use on Google's cloud shows there are clouds to sow beyond AWS. Continue Reading
-
Feature
14 Jun 2019
Data virtualization use cases cover more integration tasks
In a Q&A, Gartner analyst Mark Beyer discusses user adoption of data virtualization software and the wider data integration uses that the technology is now handling. Continue Reading
-
Feature
13 Jun 2019
Microservices and big data start to get closer
Microservices are riding a wave of user interest, leading to changes in IT operations. ThoughtWorks expert Zhamak Dehghani discusses what that means for big data. Continue Reading
-
News
05 Jun 2019
AtScale updates its data warehouse virtualization platform
AtScale 2019.1 comes with many new capabilities, including extended database support, increased security and advanced analytical query capabilities. Continue Reading
-
News
04 Jun 2019
IBM Db2 update aims to simplify use with AI
IBM's new Db2 release adds a host of AI-powered enhancements, including a range of automated error reporting capabilities and tools to more easily create AI algorithms. Continue Reading
-
News
04 Jun 2019
Vendor unveils Snowflake Data Exchange, Google Cloud integration
Snowflake Computing's data exchange marketplace aims to make data easily shared between users and providers, enable users to innovate and give providers an opportunity to monetize data. Continue Reading
-
News
03 Jun 2019
Lyftron launches universal data access platform
Lyftron launched a data access platform that uses a data hub, data insights, enterprise-wide data catalogs and lineage, and hybrid cloud management and migration to unify data. Continue Reading
-
News
31 May 2019
MapR's future in jeopardy, layoffs loom
It's right there in a MapR letter to California's labor department: A leader in the Hadoop market is desperately seeking funding after poor sales of its promising data platform. Continue Reading
-
Tip
31 May 2019
Key features to create a SQL Server audit trail in databases
SQL Server offers a set of built-in auditing tools that can help make the process of tracking logins and other database activities easier for database administrators. Continue Reading
-
Tip
28 May 2019
The evolution of the data preparation process and market
Organizations have long struggled with inconsistent data and other issues. Expert Andy Hayler explores how that has led to the rise of the data preparation tools market. Continue Reading
-
Opinion
24 May 2019
GDPR privacy concerns still brewing on law's first birthday
The first year of the much-debated EU data protection rule was subdued. High-profile fines for privacy breaches have yet to come, but regulators are starting to take action. Continue Reading
-
Feature
22 May 2019
The main picks for Hadoop distributions on the market
Check out the current top Hadoop distribution vendors in the market to help you determine which product is best for your company. Continue Reading
-
Feature
21 May 2019
Inside view of Tibco integration architecture planning
Tibco's acquisitions of well-regarded, small software specialists such as SnappyData are part of a drive toward what it calls 'connected intelligence.' CTO Nelson Petracek provides background. Continue Reading
-
Tip
20 May 2019
Check SQL Server Query Store performance impact before using
Many IT teams hesitate to use SQL Server Query Store due to performance concerns. Consultant Andy Warren offers tips on how to test and get started with Query Store. Continue Reading
-
News
20 May 2019
Lumina launches Radiance, a data risk management platform
In an effort to prevent data loss, Lumina launched Radiance, a SaaS data risk management platform. It collects and analyzes data to help prevent risks and threats. Continue Reading
-
Feature
14 May 2019
Advice on enterprise data cleansing from an SAP VP
SAP's Kristin McMahon details data cleansing best practices and explains why a good data cleanse needs continual communication, collaboration and oversight. Continue Reading
-
Feature
10 May 2019
Data virtualization layer feeds logical data warehouse, Agile BI
Indiana University is using data virtualization to combine data from various source systems for analysis, as part of an initiative to improve strategic decision-making. Continue Reading
-
Feature
09 May 2019
Data modeling software tackles glut of new data sources
Data modeling platforms are starting to incorporate features to automate data-handling processes, but IT must still address entity resolution, data normalization and governance. Continue Reading
-
News
07 May 2019
ProvenDB brings blockchain applications to world of MongoDB
Blockchain is intriguing technology, but carries with it high system overhead. ProvenDB adds blockchain to MongoDB in an effort to gain acceptable performance. Continue Reading
-
News
01 May 2019
Snowflake CEO Bob Muglia replaced by former ServiceNow CEO
Frank Slootman, who led ServiceNow and Data Domain through successful IPOs, is the new chairman and CEO of cloud data warehouse vendor Snowflake, replacing former CEO Bob Muglia. Continue Reading
-
Feature
01 May 2019
Find the best data integration tools for your organization
Read analysis and comparisons of data integration tools to help you select the right platform from the leading commercial and open source products currently on the market. Continue Reading
-
Feature
30 Apr 2019
Data model design tips to help standardize business data
Data models should be understandable to business users and kept to a reasonable scope, say the leaders of a data modeling initiative at England's Environment Agency. Continue Reading
-
News
30 Apr 2019
Snowflake CEO Bob Muglia talks cloud data warehouse evolution
In this Q&A, now-former Snowflake CEO Bob Muglia discusses the vendor's decision to embrace cloud data warehousing and how the industry is changing as more data moves to the cloud. Continue Reading
-
Feature
29 Apr 2019
Wayfair charts open source components course to growth
Teams at Wayfair mix new open source tools to power customer-facing apps. In such shops, tech leaders like Ben Clark must deftly maneuver an obstacle course of data components. Continue Reading
-
News
25 Apr 2019
MongoDB buys Realm database to boost mobile chops
MongoDB is buying Realm, the maker of an open source database for cross-platform mobile applications, to boost its Atlas cloud platform and Stitch backend as a service. Continue Reading
-
Tip
22 Apr 2019
SQL Server performance tuning best practices for DBAs
Tuning database performance is a complex process, but consultant Joey D'Antoni details a list of SQL Server performance tuning best practices that can make it easier. Continue Reading
-
News
17 Apr 2019
Google takes a run at enterprise cloud data management
New Google Cloud boss Thomas Kurian is putting databases and data management at the forefront at Google. The vendor has forged key data deals, showing a more mature Google Cloud. Continue Reading
-
Feature
17 Apr 2019
4 factors to consider in a Hadoop distributions comparison
Examine the key characteristics necessary to evaluate in a Hadoop distribution comparison, focusing on enterprise features, subscription options and deployment models. Continue Reading
-
News
15 Apr 2019
Kafka at center of new event processing infrastructure
Events are as important as data in emerging applications underlying many e-commerce efforts. Streams of events tell a company what motivates customers to use online products. Continue Reading
-
Tip
09 Apr 2019
Pros and cons of using SQL Server audit triggers for DBAs
Using triggers to capture audit information in SQL Server can be instrumental in keeping track of database use and changes. But they aren't a perfect fit for all cases. Continue Reading
-
Feature
05 Apr 2019
USAA adds data engineering skills to speed data science work
When the United Services Automobile Association's data science team wasn't getting data in the right format, the team lead realized the USAA needed more data engineers. Continue Reading
-
Feature
04 Apr 2019
How data staging helped Walgreens transform its supply chain
Walgreens built a centralized data warehouse to give supply chain partners a better view into its data -- but analytics were slow. That's where a data staging tier came in. Continue Reading
-
News
04 Apr 2019
Tools manage performance for big data cloud applications
Tools such as Unravel and Pepperdata offer a way to measure performance of big data cloud applications, which may aid companies with on-premises configuration issues. Continue Reading
-
Feature
03 Apr 2019
Evaluate the features of data integration tools and software
Before you evaluate and select data integration tools and software, assess which must-have, should-have and nice-to-have features match your organization's needs. Continue Reading
-
Feature
02 Apr 2019
Data integration platforms take users beyond ETL software
Discover how commercial data integration platforms help organizations manage and simplify the process of combining and sharing their increasing volumes of data. Continue Reading
-
Tip
29 Mar 2019
5 things to know about deploying big data systems in data containers
Planning for security and container APIs, and watching out for infrastructure sprawls are some issues to be aware of before deploying big data in containers. Continue Reading
-
Tip
29 Mar 2019
Google Cloud Spanner overview: 4 features to consider
Cloud Spanner's ability to offer data consistency and horizontal scalability has helped the relational database service gain traction. Learn more about its architecture. Continue Reading
-
Tip
27 Mar 2019
How to resolve and avoid deadlocks in SQL Server databases
Deadlocks are a real hindrance to SQL Server users, but database administrators can avoid them by taking steps to limit them and stop them from recurring. Continue Reading
-
News
25 Mar 2019
Facebook alumni forge own paths to big data analytics tools
Startups Interana and Rockset differ in their approaches to providing new query capabilities on fast-arriving big data. Both are led by technologists who started at Facebook. Continue Reading
-
News
25 Mar 2019
Building a business glossary enhances data governance
Experts say data professionals should work to create a common vocabulary in organizations to help boost data governance and compliance with laws like GDPR. Continue Reading
-
Tip
22 Mar 2019
Why data silos matter: Settling ownership of data issues
Data management is often still seen as an IT task, but that can lead to data silos. Find out why the business should be in charge of its data as part of a governance process. Continue Reading
-
News
21 Mar 2019
Machine learning approaches gain critical mass for data pros
Machine learning will bring change to analytics and data management, said data luminary Michael Stonebraker. Others agree managing such change will take special effort. Continue Reading
-
News
15 Mar 2019
Big data management practices gain attention amid risks
Many data professionals have yet to solidify traditional data management practices, but they have a new set of challenges to overcome to ensure data privacy and avoid misuse. Continue Reading
-
News
12 Mar 2019
Aerospike database garners Spark, Kafka connectors
Apache Kafka and Apache Spark connectors ease use of the Aerospike NoSQL data store in high-speed applications such as analytics that are becoming more broadly supported. Continue Reading
-
News
11 Mar 2019
Data catalog software takes on data lakes, privacy laws
Data catalogs form a hub for managing enterprise data. New products focus on machine learning and AI add-ons that help automate aspects of data governance. Continue Reading
-
Feature
06 Mar 2019
Augmented data management draws more enterprise interest
With AI and machine learning, organizations are starting to augment their data management. This is changing the way enterprise users capture, govern and integrate data. Continue Reading
-
Feature
27 Feb 2019
How to navigate the challenges of the data modeling process
Data modeling and curation can help businesses more efficiently use data they've collected. There are challenges, however -- beginning with ensuring data quality. Continue Reading
-
Feature
27 Feb 2019
8 tips to improve the data curation process
A data curation and modeling strategy can ensure accuracy and enhance governance. Experts offer eight best practices for curating data. First, start at the source. Continue Reading
-
News
26 Feb 2019
Hazelcast grid tunes for data scalability tradeoffs
An in-memory data grid (IMDG) from Hazelcast lets designers tune subsystems to support consistency over availability, or the reverse, depending on what designers want. Continue Reading
-
Feature
25 Feb 2019
Explore Hadoop distributions to manage big data
Discover the uses of Hadoop distributions and the first steps in evaluating these products, as well as how the merger of rivals Cloudera and Hortonworks affects the market. Continue Reading
-
Feature
21 Feb 2019
DataOps is more than DevOps for data, Delphix CTO says
Data operations is young compared to DevOps, but it is increasingly used as part of projects that put data at the center of development. Here, Delphix CTO Eric Schrock makes observations about the trend. Continue Reading
-
Tip
21 Feb 2019
SQL Server auditing best practices: 3 key questions for DBAs
Acing a SQL Server database audit starts with careful monitoring of how sensitive data is accessed and used so you can answer the top questions that auditors ask. Continue Reading
-
News
14 Feb 2019
Originators form group to boost Presto SQL query engine
The Presto engine arose as an alternative to Hive for big data queries. Now, the Presto Software Foundation has formed to promote the SQL query software's virtues. Continue Reading
-
Feature
13 Feb 2019
5 best practices for managing real-time data integration
Real-time data integration isn't like traditional data integration -- "it's moving, it's dirty and it's temporal," cautions one data pro. Experts offer up some best practices. Continue Reading
-
News
11 Feb 2019
Web integration platform eases way to machine learning models
StoryFit data scientists employ machine learning algorithms to gauge film script scenarios' prospects. They use Import.io tools to make data preparation easier. Continue Reading
-
News
01 Feb 2019
Open source cloud databases battle software 'strip mining'
Cloud giants like AWS have adopted open source databases, causing Confluent, MongoDB and others to guard their assets the best way they know how: licensing. Continue Reading
-
News
01 Feb 2019
Cloud data management, security top of mind for government
Federal government data officers grapple with cloud data management, weighing lower cost and efficiencies against security threats and vendor lock-in. Continue Reading
-
Feature
01 Feb 2019
Cloud data warehouse makes inroads as users spurn admin tasks
Overlooked in the run-up to Hadoop, data warehouses have found new life off premises. Cloud-based data warehouses find favor with teams that want to reduce warehouse administration. Continue Reading
-
Feature
01 Feb 2019
5 trends for SQL Server environments as SQL Server 2019 looms
SQL Server is undergoing new changes, as Microsoft prepares to release the 2019 version of the database software. Other changes are also on tap for SQL Server users. Continue Reading
-
Feature
29 Jan 2019
5 data management infrastructure technologies to evaluate
It's the start of a new year -- is your organization ready to face the data management challenges ahead? Here are five technologies to consider adopting. Continue Reading
-
Feature
23 Jan 2019
Advantages of graph databases: Easier data modeling, analytics
Graph databases are finding a place in analytics applications at organizations that need to be able to map and understand the connections in large and varied data sets. Continue Reading
-
News
22 Jan 2019
New Teradata CEO pursues cloud-based architecture
Cloud architecture, analytics and AI data processing are top innovation priorities for new Teradata CEO Oliver Ratzesberger. He talks about his goals in this Q&A. Continue Reading
-
News
21 Jan 2019
Data.gov shutdown shows limits of open data
The Data.gov shutdown shows that, as open data can be turned off, data professionals may need to consider alternative sources for the kinds of data the government offers. Continue Reading
-
News
15 Jan 2019
Cloudera and Hortonworks combo to push CDP, machine learning
Two wunderkinds of Hadoop have formalized their merger. Cloudera and Hortonworks say they will place special focus on AI as they chart the stand-alone vendor's future. Continue Reading
-
Tip
11 Jan 2019
Why organizations need a solid data governance strategy
The flood of data flowing into data warehouses, data lakes and other systems makes effective data governance a must for successful business analytics initiatives. Continue Reading
-
News
10 Jan 2019
IBM Weather Channel location data use spurs privacy concerns
IBM CEO Ginni Rometty pushed real-time weather data mining for travel and other uses, even as the Los Angeles city attorney is suing IBM's Weather Company unit over its sharing of location data with business partners. Continue Reading
-
News
28 Dec 2018
Data management trends for 2019: Governance, DataOps, cloud
Better data governance, increased cloud use and wider DataOps adoption head the list of trends for data management teams to plan for in 2019, IT analysts say. Continue Reading
-
Podcast
19 Dec 2018
Open source support was central to 2018 data deals
Mergers and acquisitions unsettled the big data status quo in 2018. Open source support made these couplings a bit different than those of the past, Talking Data podcasters said. Continue Reading