The data rush: How information about you is 21st century 'gold' —
Big data is everywhere. It's created on farms, factories and phone lines. It's gathered in homes, hospitals and warehouses. You're even contributing to data pools by reading this article on your mobile phone, tablet or computer.Tech geeks have long discussed the potential of capturing the data we create to drive new efficiencies, target services more directly and provide the complex calculations that can underpin exciting new technologies.
According to author Rick Smolan, big data is opening up "a completely new dimension" as information from mobile phones, smart devices, internet browsers, ATM cards, medical equipment, desktop devices in the workplace and much more is gathered and analyzed. This is affecting all manner of industries including, health, transportation and entertainment and is "like watching the planet develop a nervous system," Smolan says.
Yet the sheer scale and complexity of these developments can often be lost in a never-ending well of digits and decimal places.
Did you know, for example, that the amount of data captured globally is estimated to reach 40 zettabytes* (more on which later) by 2020? Sure, that sounds like a big number, but can you visualize what it actually means.
No? It's hardly surprising.
Never fear, we've put together seven simple graphs to highlight the monumental scale of data being created, saved and processed and how this new field will come to define much of our lives in the future.
Not convinced about the rise of big data? This stat ought to change your mind.
According to research cited in the World Economic Forum's Global Information Technology Report 2014, 90% of the world's data was generated in the last two years. The remaining 10% was produced throughout the rest of human history.
Obviously, advancing technologies like the rise of mobile phone apps and social media platforms have a large part to play in this sudden explosion of data but the falling cost of storing data in recent years has also had a profound impact.
According to research cited in the World Economic Forum's Global Information Technology Report 2014, 90% of the world's data was generated in the last two years. The remaining 10% was produced throughout the rest of human history.
Obviously, advancing technologies like the rise of mobile phone apps and social media platforms have a large part to play in this sudden explosion of data but the falling cost of storing data in recent years has also had a profound impact.
Unsurprisingly, the most developed regions of the world have raced ahead with the amount information stored and collected.
According to the data in this graph, collated in the Big Data: Next Big Thing report from global IT trade organizations Nasscom, CRISIL and GR&A, the U.S. collected and stored more than 3,000 petabytes of data in 2012.
To put that figure into context, one petabyte contains enough space to store the DNA of the entire population of the U.S. and then clone them twice, according to analysis from Computer Weekly.
Meanwhile, large internet firms like Google or Apple store as much as 100 petabytes of data on their own servers, according to the Tech Terms glossary.
As the report figures were captured two years ago, they are likely to have multiplied in the period since.
By 2020, the amount of data captured globally is expected to reach 40 zettabytes (there are a million petabytes in a zettabyte. A single zettabyte contains enough space to store 36 million hours of HD video), according to Nasscom.
According to the data in this graph, collated in the Big Data: Next Big Thing report from global IT trade organizations Nasscom, CRISIL and GR&A, the U.S. collected and stored more than 3,000 petabytes of data in 2012.
To put that figure into context, one petabyte contains enough space to store the DNA of the entire population of the U.S. and then clone them twice, according to analysis from Computer Weekly.
Meanwhile, large internet firms like Google or Apple store as much as 100 petabytes of data on their own servers, according to the Tech Terms glossary.
As the report figures were captured two years ago, they are likely to have multiplied in the period since.
By 2020, the amount of data captured globally is expected to reach 40 zettabytes (there are a million petabytes in a zettabyte. A single zettabyte contains enough space to store 36 million hours of HD video), according to Nasscom.
The WEF describes data as a new form of asset class equivalent to oil or gold. "Today we are seeing a data boom rivaling the Texas oil boom of the 20th century and the San Francisco gold rush of the 1800s," the Global Information Technology Report 2014 states.
Like these vital commodities, data is created, exchanged and traded all over the world.
As such, the growth in cross border data flows -- where information is exchanged online via email, instant messaging or on digital trading platforms that bring buyers and sellers together -- is moving at a rapid pace. In fact, it is growing at a far faster rate than conventionally measured trade in goods and services.
The demand for international bandwidth increased at a compound annual rate of 49% between 2008 and 2012, according to consulting firm TeleGeography, which tracks international data flows, in a 2013 report from the Progressive Policy Institute.
By comparison, the volume of global trade in goods and services rose at an average rate of 2.4% over the same period.
While the world's recent economic difficulties may be reflected in the goods and services figures, the ever more valuable nature of data is clearly demonstrated in the demand for greater bandwidth.
PPI also points to research from the Brookings Institute which states that increased data flows can lead to increased "productivity, innovation and growth" in entirely new hi-tech industries.
The technological framework to move and store this data, across international borders if required, is hence vital.
Like these vital commodities, data is created, exchanged and traded all over the world.
As such, the growth in cross border data flows -- where information is exchanged online via email, instant messaging or on digital trading platforms that bring buyers and sellers together -- is moving at a rapid pace. In fact, it is growing at a far faster rate than conventionally measured trade in goods and services.
The demand for international bandwidth increased at a compound annual rate of 49% between 2008 and 2012, according to consulting firm TeleGeography, which tracks international data flows, in a 2013 report from the Progressive Policy Institute.
By comparison, the volume of global trade in goods and services rose at an average rate of 2.4% over the same period.
While the world's recent economic difficulties may be reflected in the goods and services figures, the ever more valuable nature of data is clearly demonstrated in the demand for greater bandwidth.
PPI also points to research from the Brookings Institute which states that increased data flows can lead to increased "productivity, innovation and growth" in entirely new hi-tech industries.
The technological framework to move and store this data, across international borders if required, is hence vital.
The market for big data products -- defined as revenue derived from sales of related hardware, software and services to transmit and capture data -- totaled $18.6 billion in 2013, according to the Wikibon Big Data Market and Revenue Forecast 2013 - 2017.
That's more than the annual GDP of some small countries.
It also represents a growth rate of 58% over the previous year, emphasizing the growing focus and capacity of companies, governments and institutions to store the data we create.
And as this graph shows, the industry of data collection and storage that has sprung up in recent years has plenty of room for growth as more of our data is captured.
That's more than the annual GDP of some small countries.
It also represents a growth rate of 58% over the previous year, emphasizing the growing focus and capacity of companies, governments and institutions to store the data we create.
And as this graph shows, the industry of data collection and storage that has sprung up in recent years has plenty of room for growth as more of our data is captured.
Smartphones have made us all contributors to the big data economy. Every social media post, video, picture and website visit can tell businesses something about their customers.
This graph from the WEF's Global Information Technology Report 2014 which analyzed data from Cisco, which shows the sharp growth of mobile data traffic looking ahead to 2015.
True, the method of calculation chosen may not be the most reader friendly, but think of it this way: If the level of data created in 2010 equaled 100 bytes, then that figure is set to multiply by a factor of almost 20 by 2015.
For large tech companies such as Google, Apple, Facebook and Twitter this boom will offer a plethora of consumer data that they can store, analyze and use to tailor services to customers and advertisers.
Africa alone is expected to hit one billion mobile phone subscriptions by 2015, according to Informa Telecoms.
But alongside these opportunities, there are also issues surrounding privacy and security.
Concerns over how data has been shared or can be exploited by intelligence and security agencies like the NSA and MI5 has been an issue of major public debate in recent years, thanks in part to the disclosures of whistleblower Edward Snowden.
The head of Britain's GCHQ intelligence service, however, recently described social media as "the command and control networks of choice for terrorists and criminals."
This graph from the WEF's Global Information Technology Report 2014 which analyzed data from Cisco, which shows the sharp growth of mobile data traffic looking ahead to 2015.
True, the method of calculation chosen may not be the most reader friendly, but think of it this way: If the level of data created in 2010 equaled 100 bytes, then that figure is set to multiply by a factor of almost 20 by 2015.
For large tech companies such as Google, Apple, Facebook and Twitter this boom will offer a plethora of consumer data that they can store, analyze and use to tailor services to customers and advertisers.
Africa alone is expected to hit one billion mobile phone subscriptions by 2015, according to Informa Telecoms.
But alongside these opportunities, there are also issues surrounding privacy and security.
Concerns over how data has been shared or can be exploited by intelligence and security agencies like the NSA and MI5 has been an issue of major public debate in recent years, thanks in part to the disclosures of whistleblower Edward Snowden.
The head of Britain's GCHQ intelligence service, however, recently described social media as "the command and control networks of choice for terrorists and criminals."
The rise of mobile has created an entire new industry on the back of the data our smartphones create.
For example the so-called "app economy" -- the business created by software applications like Instagram, Whatsapp or even sports performance widgets like the Nike+ training app running on smartphones -- has created hundreds of thousands of high-tech jobs, according to the WEF.
One 2012 study by the Progressive Policy Unit, meanwhile, calculated that American GDP growth for the first two quarters of 2012 rose by 0.6% when it was adjusted to include the impact of rise in data consumption (which remained unmeasured at that point).
For example the so-called "app economy" -- the business created by software applications like Instagram, Whatsapp or even sports performance widgets like the Nike+ training app running on smartphones -- has created hundreds of thousands of high-tech jobs, according to the WEF.
One 2012 study by the Progressive Policy Unit, meanwhile, calculated that American GDP growth for the first two quarters of 2012 rose by 0.6% when it was adjusted to include the impact of rise in data consumption (which remained unmeasured at that point).
Where research leads, commercial adaptations tend to follow.
This graph from Research Trends Magazine displays the increasing number of peer-reviewed research papers exploring the topic of big data.
Experts working in the field suggest that big data will help power exciting new systems like the internet of things, where any physical item -- even every day items like a chair or fridge -- can connect to the internet and make decisions based on their owner's data patterns.
But for these futuristic visions to happen as planned, there has to be a focus on training people with the requisite skills to make them a reality.
The 2012 Big Data: Next Big Thing report suggests that the U.S. alone will suffer from a shortage of 190,000 data scientists by 2018 if steps are not taken to encourage more people to learn the skills required to keep up with big data's booming growth.
This graph from Research Trends Magazine displays the increasing number of peer-reviewed research papers exploring the topic of big data.
Experts working in the field suggest that big data will help power exciting new systems like the internet of things, where any physical item -- even every day items like a chair or fridge -- can connect to the internet and make decisions based on their owner's data patterns.
But for these futuristic visions to happen as planned, there has to be a focus on training people with the requisite skills to make them a reality.
The 2012 Big Data: Next Big Thing report suggests that the U.S. alone will suffer from a shortage of 190,000 data scientists by 2018 if steps are not taken to encourage more people to learn the skills required to keep up with big data's booming growth.