[Big] Data Management

Big data is seen as one of the most important enabling technologies for the IoT. In this chapter, we will talk about big (and not-so-big) data management for M2M and IoT applications. Because this is a complex topic and there is no one-size-fits-all solution, we will gradually construct the conversation by talking about scenarios of different complexity. But, before we get to that, we will set the stage by talking to one of the most respected figureheads of the big data movement: Mike Olson, Chief Strategy Officer and Founder of Cloudera.

IoT Data Types and Analytics Categories

As we discussed in the Plan/Build/Run section of Part II (“Ignite | IoT Methodology”), it is important to understand the distribution of and relationships between different data types within your distributed IoT application landscape. The figure below provides an overview of some of the most common data types found in an IoT environment. Data coming from remote assets typically includes meter data, sensor data, event data and multimedia data (such as video data). Data uploaded to assets typically includes configuration data, work instructions, decision context (weather forecast, for example), and various multimedia files. In the backend, data coming from remote assets is usually managed in a dedicated IoT or M2M data repository, which includes basic asset and device data, status data, and status change history, as well as time series data (such as groupings of meter readings). It’s worth noting that, as in any good, heterogeneous enterprise application landscape, there will most likely be other applications which also contain asset-related data – some of which might be redundant. For example, the ERP system is likely to contain the Asset Master Data, including all data relating to the product configuration at sales time, as well as customer contract data. One or more ticket management systems will contain customer queries related to specific assets. Integrating this data into one, holistic view of the asset is usually a challenging task. In addition, there will be a lot of related data in other systems, including customer transaction history, customer social data, product data, etc.

Building efficient applications based on this data is as challenging as in any heterogeneous enterprise application landscape, and can be tackled via well-established approaches like EAM (Enterprise Architecture Management), EAI (Enterprise Application Integration), SOA (Service Oriented Architecture), and BPM (Business Process Management) [EBPM]. This is something we will look at in the next chapter.

Making sense of IoT data requires efficient analytics – this does not just apply to big data, as we will see in the following. There are different categories of analytics. Some of them simply review past events, while some more advanced analytics categories attempt to use historical data to forecast future events and developments.

Basic analytics include:

• Descriptive analytics: “What is happening?” (Example: An engine stopped working)
• Diagnostic analytics: “Why did it happen?” (Example: Because of a faulty vault)

Advanced, forward-looking analytics include:

• Predictive analytics: “What is likely to happen?” (Example: When is an engine likely to stop working?)
• Prescriptive analytics: “What should be done to prevent this from happening?” (Example: Exchange the vault before it breaks)

Not all IoT solutions will leverage all of the above data types and analytics categories. In fact, it is extremely important for a project manager to understand how they can ensure that the data management architecture and selected tools are limited to what is really needed. As we will see in the following, there are good ways to combine different technologies, so starting with a Minimum Viable Product (MVP) philosophy often makes sense. Of course, choosing the right core data management technology – for example, relational versus NoSQL – must be based on careful analysis.

Four IoT Data Management Scenarios

In order to ensure that an IoT project manager using the Ignite | IoT Methodology can more easily identify the right architecture and technologies for data management, we have defined four basic scenarios. We will use these scenarios to gradually construct our discussion of different options for IoT data management architectures and technologies.

Scenario A is relatively straightforward, assuming a moderate number of assets in the field (a couple of thousand assets), with a very small number of events coming in per asset per hour. Analytics will be limited to basic reports and descriptive analytics. We call this scenario “Basic M2M.”

Scenario B assumes a real Enterprise IoT solution with hundreds of thousands of assets in the field, and/or higher volumes of data coming in from the assets. The initial focus for this project is on analyzing the data stream from the assets and being able to react in “real time” to critical events (meaning within a second, for example, not hard real-time). We call this scenario “Enterprise IoT and CEP,” where CEP stands for Complex Event Processing.

Scenario C builds on this scenario, adding the requirement to store field data over a longer period of time and on a large scale. Basic descriptive and, potentially, diagnostics analytics will be performed on this data.

Scenario D is an extension of Scenario C, plus more advanced analytics, like predictive and prescriptive.

A summary of all four scenarios can be found in the figure below.

Scenario A: Basic M2M

A typical application for Scenario A would be remote equipment monitoring, such as for a fleet of mobile industrial machines. A limited number of status updates and events would be expected per machine over time. Basic reporting will show machine status overview, machine utilization, etc. The architecture will usually be relatively straightforward for the backend. The initial challenge will most likely center on integration of assets and having the required device interfaces for assets (see our discussion of Technology Profile for IoT gateways). The figure below shows the AIA for Scenario A.

In terms of technology selection, two key decisions have to be made: which data repository should be used to manage asset and field data, and which interface technology should be used to integrate the remote asset.

For the data repository there are a number of different options, including RDBMS and NoSQL. Many traditional M2M application platforms are built on RDBMS technology, and this works perfectly well in most cases. One advantage is that one can build on the rich ecosystem of related tools, from reporting to backup management. Also, most organizations will have the required skills to build and operate RDBMS-based solutions.

However, if the scenario is likely to be extended over time; if slightly more complex applications have to be developed, for example, or the analytics requirements are likely to include some more advanced time series analysis in the future, then the solution architect might be well advised to consider a NoSQL database instead.

The second technology decision relates to the integration of the asset and the management of the events and other data streaming from it. If an M2M or IoT application platform is used, it will most likely have this functionality built in. If it is a custom development, there are multiple options. For very basic applications, an application server or a basic event processing technology like NodeJS could be used. There is no lack of established messaging and related technologies that could be used alternatively or in addition. Finally, some IoT-specific messaging solutions like MQTT have started to emerge, which should offer additional benefits due to their specific nature. The figure below provides an overview:

Scenario B: Enterprise IoT and CEP

Scenario B assumes that we are dealing with a much larger number of assets in the field, or that the assets are delivering a much higher data volume to the backend – or both.

In this scenario, close monitoring of data streams will often be required, as will the ability to react quickly to certain types of situations. In many cases it will also be necessary to analyze data streams for certain patterns in real time, for a steep increase in temperature for example.

In such cases, one technology that can be very useful is Complex Event Processing (CEP). CEP allows data from multiple data streams to be tracked and analyzed in order to infer patterns that have a specific meaning on the business level – for example, a traffic light switching to red and a pedestrian crossing the road at the same time. Consequently, one interesting use case of CEP is in the area of sensor data fusion (see Technology Profile for IoT Gateways).

The overall goal of CEP is to identify the business logic of related events, and be able to respond to them in (near) real-time.

Robin Smith, Director of Product Management at Oracle has summarized the basic event patterns in CEP for us as follows:

Filtering

• Data stream is filtered for specific criteria, such as temperature > 200 F

Correlation & Aggregation

• Scrolling, time-based window metrics, such as average heart pulse rate in the last 3 days

Pattern Matching

• Notification of detected event patterns, such as machine events A, B, and C occurred within 15-minute window

Geospatial, Predictive Modeling and Beyond

• Immediate recognition of geographical movement patterns, application of historical business intelligence models using data mining algorithms

The figure below shows the basic principle of CEP:

Example: Continuous Query Language (CQL)

One good example of a well-established method for analyzing event streams in CEP is the Continuous Query Language (CQL), an extension to the standard SQL language. There are many other approaches, including a number of tools that use visual modelers to support CEP. We will use CQL as our example as it will allow us to illustrate the basic principles.

Robin Smith, Director of Product Management at Oracle describes CQL as follows:

• CQL queries support filtering, partitioning, aggregation, correlation (across streams), and pattern matching on streaming and relational data
• CQL extends standard SQL by adding the notion of stream, operators for mapping between relations and streams, and extensions for pattern matching
• A window operator (RANGE 1 MINUTE for example) transforms the stream into a relation

The simple example below calculates the average temperature in the last minute for all temperature sensors submitting data to the selected stream:

SELECT AVG(temperature) AS avgTemp, tempSensorId

FROM temperatureInputStream[RANGE 1 MINUTE]

GROUP BY tempSensorId

CEP and IoT

In the context of the IoT, CEP can be deployed on the asset or in the backend, or both. The figure below shows the latter. Deploying CEP on the asset (shown here covering both gateway and on-asset business logic, because the boundaries can be blurred) has the advantage that many events can be pre-filtered or combined (sensor fusion), before being used locally (assets autonomy) or being forwarded to the backend (event forwarding). Deploying CEP in the backend has the advantage that multiple data streams from different assets can be combined for analysis. Also, context data (such as weather forecasts) can be added more easily to the decision-making process.

Case Study: Emerson

Emerson is an American multinational corporation that manufactures products and provides engineering services for a wide range of industrial, commercial, and consumer markets.

The Emerson Trellis™ Platform is a data center infrastructure management solution, which combines inventory management, site management, and power management. The solution uses a local gateway appliance to connect to storage units, servers, power systems, and cooling equipment. CEP is used in this solution for data aggregation, filtering, and processing to support real-time decision making in a very large-scale environment. The figure below provides an overview of the solution architecture:

 

Scenario C: Adding Big Data and Basic Analytics

Scenario C assumes that the solution needs to be able to perform long-term analysis of asset data on a very large, big data scale. Here, the boundaries are often not clear. Many CEP systems will be able to scale to very large data volumes. However, as we will see in the following, big data is about more than just volume.

Big Data

There are many different definitions of big data, which should generally be seen as a continuously evolving concept. Big data usually refers to very large amounts of structured, semi-structured, and unstructured data which is stored with the intention to mine it for information. It is hard to get specific numbers on how big data has to be to count as “big data,” but the general assumption is that it refers to petabytes, if not exabytes of data. Traditional, large enterprise applications such as ERP and CRM usually only reach terabyte level.

The driving forces behind big data have traditionally been large Internet companies, such as Google, Facebook, and the like. Google’s engineers, for example, elaborated the initial concept of MapReduce, which allowed data and queries to be spread across thousands of servers. Together with other companies like Yahoo, this concept was implemented as an open-source product called Hadoop, which is now supported by start-up companies such as Cloudera (see the interview at the beginning of this chapter) and Hortonworks. Google, arguably the leading-edge company in this field, has also introduced next generation technologies like its Cloud Dataflow, which is based on Flume and MillWheel.

Regardless of the technology used, big data has brought a lot of movement into the traditional RDBMS/OLAP market. One of the key changes is that big data is very well suited to manage very different types of data, from highly structured to completely unstructured.

Based on this infrastructure, for example, the emerging Data Lake concept is proposing to turn things upside down for enterprise; instead of defining a database structure first, and then populating it with data that fits into this structure, the Data Lake simply stores any and all kinds of data, and then makes this data available when it is needed, in whatever format is needed.

Doug Laney (now with Gartner) is credited with providing the first description of what many people see as the smallest common definition of big data – the three Vs: volume (dealing with large volumes of data), velocity (dealing with high-speed data streams in near real time), and variety (dealing with data in a variety of forms). Some people also add variability (or inconsistencies) and veracity (data quality) to the big data “Vs.”

Basic Analytics

Basic analysis of big data typically involves descriptive and diagnostic analytics. Descriptive statistics involve counts, sums, averages, percentages, minimum and maximum, and simple arithmetic, applied to groupings or filtered datasets.

Some people claim that 80% of business analytics are descriptive analytics, especially in social media analytics [LI1]. Examples include page views, number of posts, mentions, followers, average response time, etc.

Because of the explorative nature of many IoT projects, it can be assumed that many projects will initially also heavily rely on such basic, descriptive analytics (for things like analysis of statistics like MTBF (Mean Time Between Failure) of operational equipment).

Diagnostic analysis will also play an important role in the basic analysis of IoT data; in the event of a problem with a piece of operational equipment it will be vital to be able to identify the root cause as soon as possible in order to fix it quickly.

Adding NoSQL to the equation

In addition to unstructured big data repositories, document-oriented and NoSQL databases are also starting to play an important role in the context of the IoT. In the following, we discuss key aspects of IoT, NoSQL, and big data with Max Schireson, CEO of MongoDB at the time of the interview. MongoDB is a leading open-source database for NoSQL data management.

Building up the Big Data Infrastructure

Scenario C assumes that instead of (or in addition to) the real-time analysis of the asset data stream, we want to be able to keep this data for a longer time, in order to perform long-term pattern analysis on this “ephemeral” data. This scenario focuses on building up the required big data infrastructure in combination with stream processing, and introduces some basic algorithms that can be used to analyze this data.

Christopher Dziekan, Chief Product Officer at Pentaho explains: “Everyone is anxious to turn data (or big data) into big value and big profit, often pointing to the three Vs of big data – volume, variety and velocity. These Vs are driving an impetus for change architecturally; however, as companies build upon their big data infrastructures, more Vs come into play. A big data infrastructure is a necessary response to the first three Vs, but veracity and value also need to come into the equation.” To encompass all the Vs of big data, one must look to architectures and methodologies that enable large-scale production deployment.

To turn information into strategic advantage, a big data “orchestration” platform is needed that combines data integration with business analytics. Advanced analytic techniques are applied to the data generated by IoT devices, blending that data with both relational data and new data sources. The platform must fit into existing IT infrastructure and connect to business applications so that it is universally available at the point of impact, supporting line-of-business and operational decisions.

Examining the analytics needs of the different user communities that work with IoT systems, we can see that no single tool or repository perfectly fits all users’ needs. Some need traditional data marts and data warehouses. Others may need a “data lake” repository that can be queried on an ad-hoc basis, along with a data refinery. Others need to analyze device states, perform streaming queries, and broadcast notifications. Still others need ad-hoc data transformation and an array of special tools for their data scientists.

Given these different requirements, how do we build a data architecture to meet all needs? First of all we need a set of tools or a platform that will enable us to manage the entire architecture.

Lambda Architecture

One approach for combining some of these elements into a single engine is known as the Lambda Architecture, which was introduced by Nathan Marz. The Lambda Architecture consists of a “batch layer” that stores all of the historic data, a “speed layer” that processes data in real-time, and a “serving layer” that allows both of the other layers to be queried.

Since we want all the layers to perform well, the name “batch layer” is better thought of as a “data lake” – a term coined by James Dixon of Pentaho [FO1]. The speed layer should then be thought of as a “real-time layer.”

The Lambda Architecture solves some of the data architecture issues for an IoT system but it does not explicitly provide a way to store or query the state of one or all of the devices.

IoT Data Architecture

The Lambda Architecture can be used as the guiding principle to create an IoT system.

From the diagram you can see that the IoT data architecture contains the Real-Time Layer, the Data Lake/Historical Layer, and the Query Layer of the Lambda Architecture. It also has the State Layer, and the Blending/Transformation/Refinery/Data Mart layer.

James Dixon of Pentaho elaborated on the importance between the State Layer and the Data Lake in his “Union of the State” concept. In a blog post on this topic, Dixon recommends capturing the initial state of an application’s data and the changes to of all of the attributes, not just the main/traditional fields. He advocates applying this approach to multiple applications, each with its own Data Lake of state logs, storing every incremental change and event. This results in capturing the state of every field of (potentially) every business application in an enterprise across time, or, the “Union of the State.”

This approach can be applied in many situations. For example, an e-commerce vendor can find out, for any specified millisecond in the past, how many shopping carts were open, what was in them, which transactions were pending, which items were being boxed or were in transit, what was being returned, who was working, how many customer support calls were queued, and how many were in progress. By its very nature, this approach supports processes like trend analysis and compliance.

To date, there are no off-the-shelf applications or tools that offer a Lambda Architecture product or the IoT extensions to it. This is something that the IoT implementation team needs to consider when designing, in order to maximize the abilities, efficiency, and effectiveness of their IoT system.

Scenario D: Adding Advanced Analytics

Once the infrastructure for big data management and the IoT has been built up, the next step is to increase the level of sophistication with which the data is analyzed. This includes predictive and prescriptive analytics. Predictive analytics uses historical data to make forecasts about future events and developments, while prescriptive analytics attempts to guide actions to achieve an optimized outcome.

Many people actually see these scenarios as extremely important in the IoT. In particular, industrial IoT scenarios will greatly benefit from use cases like predictive maintenance.

Because of the complexity of these use cases, many companies are now looking at creating new job roles, like a data scientist who combines the required business, technical, and data analytics skills required to support such user cases.

Predictive analytics includes a number of different techniques, such as modeling, machine learning, and data mining.

Machine Learning

Dr. Tapio Torikka is a Product Manager for Condition Monitoring at Bosch Rexroth. In the following, he provides an explanation of machine learning as an important element of predictive analytics, based on [ML1] and [ML2].

Machine learning is a field of Artificial Intelligence in which an algorithm constructs a model based on input data in order to make predictions. No explicitly programmed instructions are used to create the model, which enables problems to be solved where little or no domain knowledge is available. The diagram below shows the process of training and applying a machine learning algorithm. After data is acquired from a selected source it has to be preprocessed, for example by scaling the data and imputing missing values. After preprocessing, a feature extraction step extracts any significant information from the data in order to improve the subsequent training step. During the training phase a learning algorithm is used to iteratively adjust the internal parameters of the machine learning model to improve itself. Once a desired accuracy or a preset training time has been reached, the model can be saved and later used to make predictions with unknown data.

The following types of learning can be identified:

• Supervised Learning
  • Labeled training data is used to build a classifier model
  • Example Application: Classification of emails between “normal” and “spam”
• Unsupervised Learning
  • Structure of the data is learned without labels
  • Example Application: Credit card fraud detection
• Semi-Supervised Learning
  • Small amount of labeled data, larger volume of unlabeled data
  • Example Application: Image recognition. A small number of manually labeled images available, a huge pool of unlabeled images available on the Internet

Supervised learning is preferred when labeled example data (desired outputs for presented input data) is available. When labeled data is rare or unavailable, semi-supervised or unsupervised learning can be used. A simple unsupervised training process is illustrated in the figure below. The algorithm tries to adjust its parameters in such a way that the predictions given by the model (light blue circles) match the input data (dark blue circles). This kind of model can be used for Anomaly Detection, i.e. the model will predict if a given piece of input data will exhibit the same structure as the training data.

Two distinct types of model can be built during the training phase of the machine learning process:

• Classification: Output of the machine learning model (prediction) is discrete
• Regression: Output of the model is continuous

A classifier separates the data into n categories as presented in the training data, for example, a classifier built to categorize emails might classify them as normal or spam based on the content. A regression model can be used to predict continuous signals, for example the future values of a defined target signal.

Case Study: Machine Learning for Condition Monitoring in Industrial Applications

The following case study is provided by Dr. Tapio Torikka, Product Manager for Condition Monitoring at Bosch Rexroth.

In condition monitoring, the health state of a machine is assessed based on collected sensor data. This allows the operator to optimize the maintenance activities of their assets and reduce the risk of unplanned downtime (see figure below). Typically, these systems are operated continuously (this is known as online condition monitoring).

Machines in industrial environments are very complex, so assessing their health state can be quite challenging. Many machines are purpose-built for a specific application and therefore exhibit a unique behavior. Additionally, environmental effects like temperature variations, noise, vibrations, etc. can influence sensor signals. To make matters worse, industrial machinery tends to produce very large volumes of data, partly due to the need for high-frequency measurements, as a result of the highly dynamic production cycles of the machinery. These characteristics mean that monitoring the maintenance status of machinery by manually inspecting individual sensor signals or simple rules is next to impossible.

Storing large amounts of data is no longer a problem and can be solved by implementing available big data solutions. The ability of machine-learning algorithms to generate models based on training data alone addresses the harder problem of making sense of this data. Data scientists are needed to create analytics pipelines for data preprocessing and learning algorithms. Furthermore, they are responsible for defining learning goals and determining action proposals based on the results, together with domain experts. Access to fast computers is important, as machine learning algorithms typically require a lot of computing power.

Anomaly detection is a typical approach to detect abnormal behavior in the data source. In an industrial context, this approach can be used to detect a deviation from normal machine behavior indicating a faulty condition. Anomaly detection requires training data from only one reference state (normal behavior, for example).

In a real-world example, a mining operator was interested in monitoring the health of the Rexroth drive system of a conveyor belt, which is used to transport ore from the mine. This is a critical asset for the operator and is constantly in operation, as downtime translates directly into reduced revenue. Sensors were installed to collect data from the conveyor belt drive. A remote big data system was used to store the data from the system, and machine-learning algorithms were used to generate an index representing the general health state of the machine (this is an example of predictive analytics). No failure data was available from the system and therefore an anomaly detection algorithm was used to indicate the deviation from normal behavior.

A web portal was used to view the data and to monitor the system. The screenshot below is taken from the visualization screen of the portal.

An electric motor failure occurred in this drive unit. The above image shows the result of the anomaly detection algorithm (in blue) and the electric motor current (in red) over time. The electric motor current suddenly drops to zero after several months of operation without a significant change in the trend. Manual analysis of other sensor signals did not indicate any failure in the system. As can be seen in the image, the blue curve (the result of the anomaly detection algorithm) begins to deviate significantly from normal levels (90-100) well before the motor fails. This represents the capability of machine-learning algorithms to make sense of complicated data. Analyzing the data manually would have been an impossible task due to the large volume of data and the complexity of evaluating all the available sensor signals simultaneously.

Case Study: Aircraft Engine Analytics

The following case study was provided by Ted Willke, Senior Principal Engineer and GM, Graph Analytics Operation at Intel Corporation.

Big data systems are frequently used to calculate statistics from logs and other machine-generated data. This information may be used by engineers and data scientists to monitor operations, identify failures, optimize performance, and improve future designs. Aircraft engines are among the most complex machines in the world, bringing together many aspects of physics and having to operate over an extreme range of conditions [LYYJL]. They are often monitored for diagnostics to identify potential failures early, before they lead to safety concerns and interfere with flights.

Traditionally, snapshots of engine statuses and fault codes are taken during a flight and sent to a ground station for logging and monitoring. But with the availability of IoT systems that accommodate both streaming data and scale using big data technologies (Category C systems), engine data can be streamed off-board to fleet-wide ground stations, monitored continuously, and analyzed for deeper insights [DSAR]. The same stream of data can be collected on the same engine under the same flight conditions (altitude, velocity, temperature, operating time, etc.) time and time again. Simple statistics can be calculated based on this time series data to determine if the engine is operating as expected (when compared to design specifications), and to detect abnormal conditions, such as problems with the turbine pressure, exit air temperature, fuel flow rate, etc.

While Category C systems are able to detect anomalies and failures, they may not be able to isolate faults and uncover their root cause with high confidence and accuracy [LYYJL]. Furthermore, they may not be able to detect weak signals (subtle anomalies) that may develop into more apparent anomalies and serious faults over time. This is where machine learning comes in (Category D systems). Machine learning can create a model of the system under study (in this case, the aircraft engine), and then use the model to predict the behavior of the system. Machine learning is a data-centric approach. A prototype (generic form) of the model is selected and “trained” to improve predictive accuracy. A common training process involves supplying pairs of inputs (what the prediction will be based on) and known outputs (what is being predicted) so that patterns can be learned by the model. After the patterns are learned, accurate predictions can be made. In the case of engine diagnostics, the inputs are the operating characteristics of the engine and the prediction may be one of several engine fault codes (“normal” being one such code).

In one study [LYYJL], 49,900 flights’ worth of data was analyzed to determine the ability of machine learning to predict 1 of 9 engine fault conditions from 8 engine measurements. The technique was shown to be able to detect the existence of a fault early on and to attribute the fault to a specific fault code.

Conclusions

As we saw in this chapter, there are many different levels of sophistication when it comes to data management in the IoT. The four scenarios that we have outlined should help a product manager to decide which solution architecture and technology is right for their particular need. In the following section, we shall provide some conclusions, first from the perspective of industrial applications, and then from the technological perspective.

Industry 4.0 Perspective

Tobias Conz is a project manager in the industry team at Bosch Software Innovations. He is working at the intersection of advanced IT concepts and the harsh reality of today´s manufacturing IT environments. He describes his experience as follows:

“Data management in the area of manufacturing and Industry 4.0 is particularly challenging because many manufacturers are looking at investment cycles of 10-15 years, meaning that change doesn’t come easily in existing environments. The good thing is that many machines are actually collecting a tremendous amount of data already, which is needed for machine control. However, before Industry 4.0 there were few requirements to actually make this data available to the outside world. This means that we are looking at a lot of very basic data integration scenarios that have to deal with the usual problems like heterogeneous interfaces, data types and protocols, software in different versions, etc.

Because most machine data was not integrated at a higher level, we are finding a much higher degree of heterogeneity in this space, compared, for example, to banks and insurance companies, who had to deal with integration much earlier.

New data management concepts like data mining, stream processing, etc. are of course extremely interesting in the context of manufacturing and Industry 4.0. Given the specificities of this space, we recommend a dual strategy: Individual machine data is a treasure trove for data mining which should be used wherever applicable. However, the integration of all machines for this purpose is nearly impossible. This is why we are also developing a near-real-time stream processing solution which will allow us to validate the process parameters of multiple machines at the same time. We are doing this one machine type at a time, to see which machine types this actually works best for. The biggest challenge we are facing is to find a time- and cost-efficient method for integration.”

General Recommendations

As the CEO of MongoDB, Max Schireson was able to gain great insights into the deployment of NoSQL and big data technologies in general, as well as in the emerging IoT space. In this part of the interview, he provides some valuable recommendations for project managers: