Introduction to Big Data
Class 12 · Artificial Intelligence
5.1 Introduction to Big Data
In today's digital world, enormous amounts of data are generated every second through online transactions, mobile devices, sensors, social media platforms and many other sources.
Traditional computer programs and conventional database systems may not be able to efficiently store, process and analyse such enormous and complex datasets. This type of data is commonly referred to as Big Data.
Big Data refers to extremely large and complex datasets that cannot be efficiently handled by traditional computer programs and conventional database systems.
Small Data vs Big Data
Before understanding Big Data, it is useful to understand Small Data.
Small Data refers to datasets that are relatively manageable, easily accessible and actionable for individuals or small organisations.
| Feature | Small Data | Big Data |
|---|---|---|
| Size | Relatively small datasets. | Extremely large datasets, ranging from terabytes to much larger scales. |
| Complexity | Generally easier to manage and analyse. | Highly complex and may contain different types and formats of data. |
| Processing | Can usually be processed using traditional software and databases. | Often requires specialised technologies and distributed computing systems. |
| Example | A local shop maintaining its daily sales records. | Millions of online transactions generated by a large e-commerce platform. |
Sources of Big Data
Big Data is generated from many different sources. Three important sources are:
| Source | Meaning | Examples |
|---|---|---|
| Transactional Data | Data generated when people or organisations perform transactions. | Online purchases, banking transactions, ticket bookings and payment records. |
| Machine Data | Data automatically generated by machines, devices and sensors. | IoT sensors, GPS devices, industrial machines and smart devices. |
| Social Data | Data generated through interactions and activities on social platforms. | Social media posts, comments, likes, shares, photographs and videos. |
Why is Big Data Important?
Big Data itself has limited value unless it can be processed and analysed effectively. Organisations use specialised technologies to discover useful information, trends and patterns hidden within large datasets.
- Helps organisations make data-driven decisions.
- Helps identify hidden patterns and trends.
- Supports better understanding of customer behaviour.
- Helps organisations improve processes and efficiency.
- Supports innovation and the development of new products and services.
A large school may generate huge amounts of data through student attendance, examination results, ERP transactions, website visits, learning platforms, transport systems and digital applications.
When such data becomes extremely large, diverse and continuously generated, specialised Big Data technologies can be used to store, process and analyse it.
Big Data in Everyday Life
Big Data is generated continuously through activities performed in everyday life.
- Searching information on the Internet.
- Watching videos and using streaming platforms.
- Making online purchases.
- Using social media applications.
- Using smartphones and GPS-based applications.
- Using smart devices and IoT sensors.
Competency-Based Question
An online shopping platform receives millions of customer searches, product views, purchases, payment records and customer reviews every day.
Explain why this information can be considered Big Data and identify any three major sources from which such data may be generated.
Click to View Answer
The information can be considered Big Data because it is extremely large, complex and continuously generated. It may also contain different types and formats of data.
Three major sources are transactional data such as purchases and payments, machine data such as server or sensor data, and social data such as customer reviews and social media interactions.
Common Beginner Mistakes
- Assuming that Big Data means only large-sized files.
- Thinking that Big Data contains only structured data.
- Confusing Big Data with ordinary databases.
- Assuming that simply collecting large amounts of data automatically produces useful information.
- Forgetting that Big Data can be generated from transactional, machine and social sources.
- Thinking that traditional software can always process extremely large and complex datasets efficiently.
Quick Revision
- Big Data: Extremely large and complex datasets.
- Traditional programs may not efficiently handle Big Data.
- Small Data: Relatively manageable data that can be handled using conventional tools.
- Transactional Data: Generated through transactions such as online purchases.
- Machine Data: Generated automatically by machines, devices and sensors.
- Social Data: Generated through social media activities and interactions.
- Big Data is analysed to discover patterns, trends and useful insights.
Memory Trick
B → TMS
Remember:
- B → Big Data
- T → Transactional Data
- M → Machine Data
- S → Social Data
Big Data comes from Transactions, Machines and Social activities.
Exam Tips
- Learn the definition of Big Data clearly.
- Remember the three important sources: Transactional, Machine and Social Data.
- Be able to differentiate between Small Data and Big Data.
- In competency-based questions, explain why traditional systems may not be sufficient for extremely large and complex datasets.
- Use practical examples such as online shopping, social media, IoT sensors and school ERP systems wherever appropriate.
Frequently Asked Questions (FAQs)
1. What is Big Data?
Big Data refers to extremely large and complex datasets that cannot be efficiently handled by traditional computer programs and conventional database systems.
2. What is Small Data?
Small Data refers to relatively manageable datasets that are easily accessible and actionable for individuals or small organisations.
3. What are the major sources of Big Data?
Three major sources are Transactional Data, Machine Data and Social Data.
4. Give an example of Transactional Data.
Online purchase records, banking transactions and payment information are examples of Transactional Data.
5. What is Machine Data?
Machine Data is data automatically generated by machines, devices and sensors, such as IoT sensor readings and GPS data.
6. What is Social Data?
Social Data is generated through interactions on social platforms, including posts, comments, likes, shares, images and videos.
7. Why do organisations use Big Data?
Organisations use Big Data to discover useful patterns and trends, improve decision-making, understand customers and support innovation.
8. Can Big Data be processed using traditional databases?
Extremely large and complex Big Data may not be efficiently handled by traditional computer programs and conventional database systems. Specialised technologies are therefore used for storing, processing and analysing it.
Summary
- Big Data refers to extremely large and complex datasets.
- Traditional computer programs and conventional databases may not efficiently handle such datasets.
- Small Data is comparatively manageable and can generally be processed using conventional tools.
- Major sources of Big Data include Transactional Data, Machine Data and Social Data.
- Big Data becomes valuable when organisations can discover meaningful patterns, trends and insights from it.
- Big Data supports better decision-making, improved efficiency, customer understanding and innovation.
Next Topic: Types of Big Data