Machine to machine process will exchange of data between billions of devices
Infrastructure and sensors will createhugelog data in real-time.
Online gaming systems will support millions of concurrent users;each one will produce multiple inputs per second.
3rd character of big data
Big data is not just numbers, dates, and strings. Big data is also geospatial data, 3D data, audio and video, and unstructured text, which includes log files and social media
Traditional databases systems were formed to reportsmaller volumes of structured data, fewer updates, or a predictable, consistent data structure.
types of BIG DATA ANALYSIS
BIG DATA ANALYSIS includes various types of data
Storing Big data
Selecting Big data
Processing big data
SELECTING BIG DATA STORES
Choosing the correct data stores on the basis of your data characteristics
Moving code to data
Execution of polyglot data store solutions
Aligning business goals to the suitable data store.
STORING BIG DATA
Analyzing your data characteristics
Selection of data sources
Removing redundant data
Establishing the role of NoSQL
Overview of Big data stores
Data models: Key-value, graph, document, column family.
Hadoop distributed file system
PROCESSING BIG DATA
Integrating disparate data stores
Mapping data to the programming outline
Connecting and removing data from storage
Transforming data for processing
Dividingdata in preparation for Hadoop MapReduce
Using Hadoop MapReduce
Making the components of Hadoop MapReduce
ImplementingHadoop MapReduce jobs
Observingthe progress of job flows
The structure of BIG DATA
HISTORY OF BIG DATA
The word 'Big Data has been in usagesince the time of 1990s. But it is unknown who first identified and utilized this term Big Data.
Some groups of people believe thatJohn R. Mashey has used this term first.
On the basis of its true principle, Big Data is not totally new, because people have been trying to utilize data analysis and analytics methodologiesfor supportingtheir decision-making process.
The ancient Egyptians about300BC now tried to capture all existing data in the library of Alexandria.
In the last two decades, the volume and speed of the data have generated and change.
The total number of data in the world was 4.4 zettabytes in 2013.
To demonstratethis development, the Big Data evolution is now divided into ‘3’ main phases. Each phase has its own features and abilities. To understand the context of big data today, it is essential to recognizehow each phase contributed to the modern meaning of big data.
BIG DATA PHASE
BIG DATA PHASE 3.0
Thoughweb-based unstructured content is still the main focus for many organizations in data analysis, data analytics, and big data, the current possibilities to recovervaluable information are emerging out of mobile devices.
BIG DATA PHASE 2.0
Since the early 2000s, the internet and website have started to provide unique data collections and data analysis chances. With the expansion of web traffic and online stores, companies such as Yahoo, Amazon, eBay started to analyze the behavior of customers by analyzing click-rates, IP-specific location data, and search logs. This opened a completely new world of opportunities.
BIG DATA PHASE 1.0
Data analytics and big data, both terms were originated from the longstanding domain of database management. It depends onthe storage, extraction, and optimization methodologies that are common in data that will bestored in relation database management systems.
Applications of Big data analysis:
Used in smarter healthcare
Involves in Homeland security
Disadvantages of Big data:
Despite the applications and advantages of big data, it has several disadvantages as well.
Need for talented scientists and experts and highly paid workers in the IT field.
Incompatible tools are needed. Hadoop is a highly useful tool in Big data, but the standard version of Hadoop is currently available to handle real-time analysis.
Correlation errors- A common methodology that isutilized to analyze Big data. And that is available to draw correlations by connecting one variable to another.
Security and privacy concerns: Big Data analytics will allowyou to detect fraudulent attempts.
HIGS – An inspiring PhD research assistance company with the idea of becoming “STAND DIFFERENT THAN REST”.
We are a “BIG TEAM” with more than 50 employees. HIGS teamed up with research-oriented, skilled, doctoral fellows who are here to work with you.
HIGS will help from the beginning or any stage of your research journey.