Artificial intelligence and big data
Introducing artificial intelligence and big data
Researchers in computer science and statistics have developed advanced techniques to obtain insights from large disparate data sets. Data may be of different types, from different sources, and of different quality (structured and unstructured data). These techniques can leverage the ability of computers to perform tasks, such as recognising images and processing natural languages, by learning from experience. The application of computational tools to address tasks traditionally requiring human sophistication is broadly termed ‘artificial intelligence’ (AI).
As a field, AI has existed for many years. However, recent increases in computing power coupled with increases in the availability and quantity of data have resulted in a resurgence of interest in potential applications of artificial intelligence. These applications are already being used to diagnose diseases, translate languages, and drive cars; and they are increasingly being used in the financial sector as well.
AI is a broad field, of which ‘machine learning’ is a sub-category. Machine learning may be defined as a method of designing a sequence of actions to solve a problem, known as algorithms, which optimise automatically through experience and with limited or no human intervention. These techniques can be used to find patterns in large amounts of data (big data analytics) from increasingly diverse and innovative sources.
‘Big data’ is a term for which there is no single, consistent definition, but the term is used broadly to describe the storage and analysis of large and/or complicated data sets using a variety of techniques including AI. Analysis of such large and complicated datasets is often called ‘big data analytics.’ A key feature of the complexity relevant in big data sets analytics often relates to the amount of unstructured or semi-structured data contained in the datasets.