Big data analytics
Big data analytics is the often complex process of examining big data to uncover information -- such as hidden patterns, correlations, market trends and customer preferences -- that can help organizations make informed business decisions.. See complete definition.
Top Stories
-
Feature
21 Nov 2022
What makes up an analytics pipeline?
Analytics pipelines were traditionally hidden away, but they're changing as more organizations focus on agility for their data. Learn what makes up a successful analytics pipeline. Continue Reading
-
Tip
17 Oct 2022
Make data usability a priority on data quality for big data
To help make big data analytics applications more effective, IT teams must augment conventional data quality processes with measures aimed at improving data usability for analysts. Continue Reading
-
News
23 Mar 2018
Cambridge Analytica-Facebook case shows need for ethical data mining
Ethical data collection practices are becoming even more important, as cases like Cambridge Analytica's misuse of Facebook data challenge consumer trust in enterprise analytics. Continue Reading
-
Feature
21 Mar 2018
Kubernetes gains momentum in big data implementation process
Big data vendors and users are looking to Kubernetes-managed containers to help accelerate deployments and enable more flexible use of computing resources. Continue Reading
-
Podcast
21 Mar 2018
IT, others focus efforts as GDPR implementation date looms
The EU's General Data Protection Regulation is meant to bring better data privacy to bear in the age of big data. How GDPR implementation will proceed remains to be seen. Continue Reading
-
Feature
16 Mar 2018
Silicon Valley startups home in on pressing value from data
A slew of California-based startup and early-stage data analytics companies are promising CIOs better ways to get value from their investments in databases and analytics Continue Reading
-
News
15 Mar 2018
Streaming tool from StreamSets eyes data in motion for GDPR
StreamSets software for inspecting big data brings governance to data in motion. Such capabilities may find more use as the European Union's GDPR deadline looms. Continue Reading
-
Feature
15 Mar 2018
Rethinking analytics processes spurs enterprise innovation
By taking a fresh look at the makeup of their analytics organizations, enterprises can innovate their business models and take advantage of digital disruption. Continue Reading
-
Feature
14 Mar 2018
Diversified data sets for analytics deliver top results
Analytics teams should focus on data diversity to ensure that their projects deliver the most meaningful insights -- but they must be wary of some stumbling blocks. Continue Reading
-
Definition
01 Mar 2018
U-SQL
U-SQL is a Microsoft query language that combines a declarative SQL-like syntax with C# programming, enabling it to be used to process both structured and unstructured data in big data environments. Continue Reading
-
Podcast
26 Feb 2018
Newest AI technology set to disrupt tech sector, job markets
Emerging AI technology will lead to the creation of influential new software companies and disrupt job markets, says a Goldman Sachs researcher -- predictions that are discussed here. Continue Reading
-
Podcast
22 Feb 2018
Google TPUs open up on cloud; LinkedIn intros Hadoop Dynamometer
In big data news, we find Google TPUs, or Tensor Processing Units, offered as a cloud service, while LinkedIn is open sourcing a Hadoop test simulator called Dynamometer. Continue Reading
-
Definition
21 Feb 2018
TensorFlow
TensorFlow is an open source framework developed by Google researchers to run machine learning, deep learning and other statistical and predictive analytics workloads. Continue Reading
-
News
16 Feb 2018
MongoDB 4.0 takes ACID transactions to multi-document level
MongoDB is taking a deeper step into SQL-style processing waters with a 4.0 update that brings increased support for ACID-compliant transactions to its NoSQL database. Continue Reading
-
Feature
16 Feb 2018
Data Lab CEO Gillian Docherty on how data can boost Scottish economy
Gillian Docherty, CEO of The Data Lab, outlines the organisation’s mission to turn analytics and artificial intelligence to benefit Scotland’s economy Continue Reading
-
E-Zine
15 Feb 2018
Slow to gain traction, AI apps on the verge of explosion
From chatbots ("Can I help you?") to killer bots ("I'll be back."), artificial intelligence runs the gamut of applications and emotions like no other technology. It's been nearly 70 years since AI first came into consciousness with humankind, yet only recently has it made serious inroads into business and consumer markets, mainly riding the coattails of its offshoot, machine learning.
AI's other offspring -- deep learning, cognitive computing, image recognition and natural language processing -- show plenty of promise but have barely left the womb. Yet market research and industry foot soldiers tell us AI is on the cusp of explosive growth in practically every major industry.
Along those lines, the February issue of Business Information opens with our editor's note, which provides a good dose of common sense and advice for businesses not to throw caution to the whirlwind of hyperbole surrounding AI. Companies anxious to apply AI, particularly machine learning, to their operations for fear they'll fall behind the competition, must first do their homework and separate fact from fiction, all the while keeping in mind that implementing AI apps is difficult and requires time, expense and the right kind of data.
Our cover story takes the issue of good data one step further and examines the very real dangers that bias in machine learning data sets can create. Data scientists have their work cut out for them identifying and pinpointing bias, especially since nearly all data inherently contains bias. In another feature, we look at the usefulness of today's AI tools, which, except for some niche AI apps, are not very useful, but they come with a whole lot of promise as a soon-to-be transformative force in business operations.
Also in this issue, a major components manufacturer uses AI apps to transform its assembly line process, cognitive computing can help improve a doctor's bedside manner, the benefits of AI and machine learning entice healthcare organizations to move to the cloud and humankind's range of reactions to AI's unlimited potential parallel those of the atomic age.
Continue Reading -
Guide
15 Feb 2018
Guide to using advanced analytics and AI in business applications
AI hype is becoming reality as the technology gains ground in businesses. Get tips on machine learning, cognitive computing and other advanced analytics initiatives in this guide. Continue Reading
-
Feature
15 Feb 2018
Big data throws bias in machine learning data sets
AI holds massive potential for good, but it also amplifies negative outcomes if data scientists don't recognize data biases and correct them in machine learning data sets. Continue Reading
-
Opinion
15 Feb 2018
Cognitive computing in healthcare mends doctor-patient gaps
Cognitive computing and healthcare data aggregation may prove to be important bedside companions to doctors treating their patients in the era of medical specialization. Continue Reading
-
News
13 Feb 2018
IT analysts look for clarity on IBM analytics tools
IBM's analytics software is strong technically but disjointed and hard to understand, say analysts who think simplifying things should be a priority at the company's Think 2018 conference. Continue Reading
-
Feature
12 Feb 2018
Insurer's machine learning use case: Changing driver behavior
Machine learning tools can be put to use for more than targeted marketing and product recommendations. Auto insurer HiRoad is using them to help create safer drivers. Continue Reading
-
Feature
06 Feb 2018
Data catalog software takes open data initiative to the streets
Data catalog software from Alation helped San Diego's chief data officer open up the city's data to residents, as well as city planners and other government workers. Continue Reading
-
Opinion
02 Feb 2018
Data managers should study up on GPU deep learning
As GPU deep learning becomes more common, data managers will have to navigate several new layers of complexity in their quest to build or buy suitable data infrastructure. Continue Reading
-
Tip
01 Feb 2018
Three ways to turn old files into Hadoop data sets in a data lake
Hadoop data lakes offer a new home for legacy data that still has analytical value. But there are different ways to convert the data for use in Hadoop depending on your analytics needs. Continue Reading
-
Tip
29 Jan 2018
Big data in cloud computing demands an IT skill set change
Enterprises continue to shift big data workloads to the cloud. And while the move doesn't require a total overhaul of IT skills, it does demand some change for admin and dev teams. Continue Reading
-
Podcast
26 Jan 2018
Talking Data: Is there a difference between AI and analytics?
Any AI definition should include things like predictive analytics, according to some users, even though those practices are straightforward and have been around for decades. Continue Reading
-
Podcast
23 Jan 2018
How AI and IoT will influence data management in 2018
AI and IoT will alter the data management landscape in 2018, according to analyst James Kobielus. AI will need regular updates, and DevOps will become more prevalent as a result. Continue Reading
-
Feature
22 Jan 2018
Infographic: The evolution of the chief data officer role
The CDO role, which has never been rigidly defined, is undergoing a face-lift as emerging technologies present new opportunities to enterprises. Continue Reading
- Definition 17 Jan 2018
-
Tip
17 Jan 2018
How to use machine learning to build a predictive algorithm
Machine learning is an invaluable tool for solving business problems, but don't jump into it for predictive analytics without understanding these important factors. Continue Reading
-
Feature
15 Jan 2018
Machine learning models require DevOps-style workflows
Big data is driving the use of AI and machine learning. But teams must be swift to embrace DevOps and re-evaluate models, according to Wikibon's James Kobielus. Continue Reading
-
Feature
12 Jan 2018
Comparing the leading big data analytics software options
This big data analytics tools comparison examines products from the leading vendors and provides insight into how you can differentiate between the various offerings. Continue Reading
-
Definition
10 Jan 2018
pseudoscience
Pseudoscience is a proposition, a finding or a system of explanation that is presented as science but that lacks the essential rigor of the scientific method. Pseudoscience can also be the result of an inquiry that is based on faulty premises or data. Continue Reading
-
Feature
09 Jan 2018
Insights into buying big data analytics tools
Before purchasing big data analytics software, companies must first identify their specific needs and then evaluate how the product features address those needs. Continue Reading
-
News
03 Jan 2018
Apache Hadoop 3.0 goes GA, adds hooks for cloud and GPUs
Is this the post-Hadoop era? Not in the eyes of Hadoop 3.0 backers, who see the latest update to the big data framework succeeding in machine learning applications and cloud systems. Continue Reading
-
Feature
15 Dec 2017
The benefits of using big data analytics software
Tools for big data analytics have a lot to offer, and they come in many varieties. Get to know some of the ways business users and data scientists can use the software. Continue Reading
-
Tip
21 Nov 2017
Hiring vs. training data scientists: The case for each approach
Hiring data scientists is easier said than done -- so should you try to train current employees in data science skills? That depends on your company's needs, writes one analytics expert. Continue Reading
-
Feature
19 Oct 2017
NewSQL databases rise anew -- MemSQL, Spanner among contenders
The NewSQL database was almost hidden when Hadoop and NoSQL arose. Now, as more big data teams move toward production uses, MemSQL, Cloud Spanner and similar products may get a second look. Continue Reading
-
Feature
26 Sep 2017
Bank sees ETL processes evolve from IT to front-line users
ETL jobs -- once the sole province of IT -- take on a new form as data wrangling and self-service gain greater traction with business users of analytics. Continue Reading
-
Tip
22 Sep 2017
Video conferencing usage gets new analytics to gauge ROI
Instead of assuming video ROI success, enterprises can now see a correlation between video conferencing usage and productivity with the help of new analytics tools. Continue Reading
-
Definition
31 Aug 2017
statistical noise
Statistical noise is unexplained variability within a data sample. The term noise, in this context, came from signal processing where it was used to refer to unwanted electrical or electromagnetic energy that degrades the quality of signals and data. Continue Reading
-
Guide
29 Aug 2017
Enterprise data analytics strategy: A guide for CIOs
This Essential Guide explores enterprise data analytics strategies and how to select the right infrastructure, management tactics and technologies for your organization. Continue Reading
-
Feature
21 Aug 2017
Data management processes take on a new tenor in analyst's view
Changes in data management processes -- including self-service data preparation, data lakes and real-time analytics -- create a new landscape in companies, according to analyst Matt Aslett. Continue Reading
-
Feature
11 Aug 2017
Big data application development: An introduction to Hadoop
Getting into the big data market? Hadoop offers a platform for dealing with very large data sets and the technology's vendors offer training and support for channel partners. Continue Reading
-
Tip
03 Aug 2017
Ten steps to start using predictive analytics algorithms effectively
A successful predictive analytics program involves more than deploying software and running algorithms to analyze data. This set of steps can help you put a solid analytics foundation in place. Continue Reading
-
Feature
31 Jul 2017
How SAS Enterprise Miner simplifies the data mining process
The SAS Enterprise Miner data mining tool helps users develop descriptive and predictive models, including components for predictive modeling and in-database scoring. Continue Reading
-
News
25 Jul 2017
AA Ireland turns to predictive analytics in fraud crackdown
Roadside assistance company invests in analytics software to fight fraud and introduce dynamic pricing after its split with the AA Continue Reading
-
News
24 Jul 2017
Chief data officer jobs call for nurturing data ethos in companies
In many organizations, chief data officer jobs centered on defense against risk are giving way to ones emphasizing innovation. To do so, CDOs must nurture a data culture, MIT panelists said. Continue Reading
-
Quiz
11 Jul 2017
Quiz: What's your IQ on data lake implementation and management?
A data lake can provide lots of analytics value to an organization. This quiz can tell you if you know what it takes to deploy and manage one; it also contains info to help boost your knowledge. Continue Reading
-
Feature
26 Jun 2017
Examining the Knime Analytics Platform for big data analytics
Knime offers open source data analytics, reporting and integration tools, as well as commercial software that can help build more efficient workflows. Continue Reading
-
Feature
23 Jun 2017
Machine learning platforms comparison: Amazon, Azure, Google, IBM
The platform war over machine learning tools is heating up. Use our features comparison chart to see how four top vendors stack up and help you decide which is right for your enterprise. Continue Reading
-
Feature
23 Jun 2017
MicroStrategy CEO Michael Saylor speaks about ‘expanding universe’ of BI
Co-founder and CEO of MicroStrategy takes the long view on business intelligence and sees its universe expanding with mobile technologies Continue Reading
-
Definition
11 Jun 2017
clinical data analyst
A clinical data analyst is a healthcare information professional who verifies the validity of scientific experiments and gathered data from research. Continue Reading
-
News
05 Jun 2017
How Facebook uses deep learning models to engage users
At Facebook it's all about user engagement, and to accomplish this, the company relies heavily on deep learning algorithms to tailor its products to the interests of individuals. Continue Reading
-
Feature
31 May 2017
A look at the IBM SPSS Modeler and IBM SPSS Statistics analytics tools
The IBM SPSS Modeler and IBM SPSS Statistics predictive analytics tools help users and data analysts build predictive models and manage the analytics process. Continue Reading
-
Definition
30 Mar 2017
R programming language
The R programming language is an open source scripting language for predictive analytics and data visualization. Continue Reading
-
News
11 Jan 2017
McKinsey finds hard work to do in Big Data Revisited report
The McKinsey Global Institute has revisited its path-breaking 2011 big data report, finding a new salience for machine learning, and most companies under-capitalising their data Continue Reading
-
Podcast
03 Jan 2017
Three key 2017 technology trends to know
Predictive analytics, voice-activated devices and artificial intelligence will take shape as significant technology trends in 2017. Some still await further maturity. Continue Reading
-
Opinion
06 Oct 2016
Sports data analytics isn't always a slam dunk
The growing trend of analytics in sports has caused much debate, and even though data-driven decisions can provide some benefits, they're not right for all occasions. Continue Reading
-
News
15 Sep 2016
HPE sells Vertica analytics, thanks to the growth of open source software
HPE is paring down its software holdings, including analytical database software in the Vertica line and other big data tools. A sale to Micro Focus is due to close next year, leaving users in some limbo for now. Continue Reading
-
Feature
31 Aug 2016
Why physicists are a good fit for data science jobs
With data scientists in short supply, physicists and other academic researchers from hard-science disciplines are increasingly finding places on data science teams -- and even leading big data analytics projects. Continue Reading
-
News
02 Aug 2016
Panama Papers technology shows data journalism at its best
The open source data analytics and visualisation software of the Panama Papers project empowers a new kind of journalism, says the International Consortium of Investigative Journalists Continue Reading
-
Tip
29 Apr 2016
Hadoop data analysis: Common concerns with the HDFS platform
Using HDFS technology as a data analysis platform may be insufficient for your storage needs. Explore the challenges storage administrators may encounter and find out how to address them. Continue Reading
-
Feature
26 Apr 2016
Inside the MapR Hadoop distribution for managing big data
The MapR Hadoop distribution replaces HDFS with its proprietary file system, MapR-FS, which is designed to provide more efficient management of data, reliability and ease of use. Continue Reading
-
News
12 Feb 2016
Spark a natural fit for IoT data analytics
IoT big data analytics is needed to make sense of the mountains of data coming from industrial equipment and Spark may be up to the task. Continue Reading
-
Feature
07 Dec 2015
IoT data analytics spurred on by big data's expansion
Data from the Internet of Things creates opportunities to analyze equipment performance and track the activities of drivers and users of wearable devices. But IoT data analytics requires significant IT provisions. Continue Reading
-
Feature
29 Oct 2015
Three ways to build a big data system
In a book excerpt, author Dale Neef outlines and compares different approaches organizations can take when trying to bring a big data system into their IT environments. Continue Reading
-
Feature
13 Oct 2015
Location-tracking system improves business efficiency
For organizations in industries such as healthcare and trucking, analyzing data on the location and movements of employees is helping to streamline operations. Continue Reading
-
Definition
30 Sep 2015
Lambda architecture
Lambda architecture is an approach to big data management that provides access to batch processing and near real-time processing with a hybrid approach. Continue Reading
-
News
07 Jul 2015
Manchester City group joins forces with SAP to further global goals
The City Football Group, which includes Manchester City, has teamed up with SAP to further its global development goals Continue Reading
-
Quiz
11 Jun 2015
Apache Spark architecture quiz: Do you know Spark?
Apache Spark is an open source processing engine designed for use in big data applications. Take this quiz to see if you're up to speed on Spark's history and features. Continue Reading
-
News
28 May 2015
Salesforce ties Wave analytics tool to big data platforms
Salesforce is getting more involved in big data analytics, with new partnerships that open up its cloud-based Wave analytics technology to Hadoop and other big data systems. Continue Reading
-
Definition
14 Apr 2015
revenue attribution
Revenue attribution is the process of matching customer sales to specific advertisements in order to understand where revenue is coming from and optimize how advertising budgets are spent in the future. Continue Reading
-
Feature
05 Mar 2015
Use of analytics in sports media becomes contentious
Analytics are gaining a high profile in sports media, but some fans and commentators say the stats just aren't what they're looking for. Continue Reading
-
Feature
09 Jan 2015
Big data and analytics advice from an operations pro
You don't need to employ special IT personnel for big data analytics. One company uses big data for developers to modify ads and find and adjust pain-points in a game application. Continue Reading
-
News
07 Jan 2015
Case study: UniCredit uses Fico to apply prescriptive analytics to risk management
Italian bank UniCredit is using Fico software to apply prescriptive analytics to risk management Continue Reading
-
Feature
12 Dec 2014
Customer analysis a winning play for 'Candy Crush Saga' maker
King Digital Entertainment was losing players of "Candy Crush Saga" at an early stage in the online game -- a problem it solved with help from a customer data analytics project. Continue Reading
-
News
10 Oct 2014
SAP teams up with Birst to offer cloud analytics on Hana
SAP is working with cloud BI supplier Birst to provide “instant analytics in the cloud” on the Hana in-memory database platform Continue Reading
-
Feature
08 Sep 2014
Sweden’s Klarna and Spotify design big data architectures to aid business growth
Many organisations still don't know how to capitalise on big data. Swedish firms Klarna and Spotify are making use of its possibilities Continue Reading
-
Feature
04 Jul 2014
Analytics scours World Cups for greatest footballer of all time
Sports data company Opta’s analytics of every World Cup finals match ever played since 1966 adds to the debate on who is football’s greatest of all time Continue Reading
-
News
04 Jun 2014
Apache Spark goes 1.0, looks to improve on MapReduce performance
The Apache Software Foundation released Version 1.0 of Spark, a processing framework aimed at outpacing MapReduce for machine learning and other uses. Continue Reading
-
Opinion
30 May 2014
Anxieties about big data technology trends are valid
Criticism of big data technology trends may go too far, but proponents shouldn't dismiss the critiques. Continue Reading
-
Feature
30 May 2014
Conversation with Monash: On horses for courses and parallelism
Data industry viewer Curt Monash discusses parallelism and more as new databases gain attention. Part 1 in a series. Continue Reading
-
Opinion
29 May 2014
Internet of Things applications put varied machine data to work
The Internet of Things is proving to be a hotbed of diverse data and new applications fueled by information from sensors and other machine sources. Continue Reading
-
Opinion
28 Apr 2014
Big data cloud is changing familiar database tech tune
New cloud database applications that increasingly fall into the "big-data-intensive" category are shaking up the usual build-or-buy soliloquy. Continue Reading
-
Feature
18 Apr 2014
Big data vendors should stop dissing data warehouse systems
Wayne Eckerson examines the analytics roles of data warehouses and big data systems and says he's tired of data warehouse bashing by big data vendors. Continue Reading
-
Feature
18 Apr 2014
Laws leave gray area between big data and privacy
Laws on data collection and use remain foggy, leaving businesses to feel their way through big data and privacy laws. Continue Reading
-
Feature
17 Apr 2014
Data collection practices spark debate on big data ethics, privacy
The increasing emphasis on collecting and analyzing big data is driving a debate on whether tighter privacy rules are needed to protect customers. Continue Reading
-
Feature
14 Apr 2014
Supermarket co-op stocks up on big data platform to help spur sales
Co-operative Allegiance Retail Services is deploying a cloud-based big data stack to replace a homegrown system that fell short on analytics power. Continue Reading
-
Podcast
11 Apr 2014
IDC analysts see more Internet of Things projects ahead
At IDC Directions, analysts predict more Internet of Things projects in the near future. Continue Reading
-
Feature
31 Mar 2014
Selecting the right SQL-on-Hadoop engine to access big data
Rick van der Lans explains why it's important to evaluate the differences in the technologies that make it possible to access Hadoop data using SQL. Continue Reading
-
Quiz
16 Oct 2013
Quiz: Creating effective predictive analytics programs
Take this brief quiz to test your knowledge of predictive analytics and what is needed to implement successful predictive modeling processes. Continue Reading
-
News
09 Sep 2013
Survey: Execs buying into big data in a big way
TechTarget's Wayne Eckerson discusses survey results showing that two-thirds of respondents expect to spend more than $1 million on big data programs this year. Continue Reading
-
Feature
29 Aug 2013
Big data adds new complications for data stewards
In order to accommodate big data and the needs of data scientists, data stewards will need to adjust to a more short-term view of data stewardship. Continue Reading
-
News
28 Aug 2013
eHarmony finds MongoDB the perfect match for data store
Online dating site eHarmony has used open source NoSQL database MongoDB for its data store, to speed up delivery of matches between users Continue Reading
-
News
16 Aug 2013
Online retailing means the industrialisation of analytics
Figleaves founder and eCommera chief scientist Michael Ross argues that online retailing requires a new type of action-oriented business intelligence Continue Reading
-
News
08 Jul 2013
Davenport: Travel industry data pioneer has lost way but can be saved
Tom Davenport sees the travel industry as a data pioneer that has lost its way, but can be redeemed Continue Reading
-
News
03 Jul 2013
Ocado delivers with in-house software development
Online grocery retailer Ocado built its own software for its own business – and now it delivers for Morrisons Continue Reading
-
News
22 May 2013
McLaren uses high-speed data analytics to gain Formula 1 edge
The McLaren Group uses high-speed data analytics as it seeks a competitive edge for its Formula 1 racing team engineers Continue Reading
-
News
10 May 2013
Lean analytics to change business decision making
Attending to the “one metric that really matters” is one principle of a “lean analytics” approach Continue Reading
-
Tip
22 Apr 2013
Real-time querying in Hadoop? Not so fast
Hadoop vendors are adding real-time query engines to the open source software. But the jury is out on that effort, writes Wayne Eckerson. Continue Reading
-
News
28 Feb 2013
Six things Hadoop brings to the BI and analytics table
TechTarget's Wayne Eckerson outlines what is unique about Hadoop and related technologies versus traditional BI and data warehousing tools. Continue Reading