John Kane & Michael Cohen, CA Technologies
Management and Monitoring of Big Data Infrastructure (MongoDB & Cassandra)
Big Data Architecture and Analytics | October 29, 2014 | Morning Session
MongoDB, Cassandra, Elastic Search and other technologies, are highly capable and widely used NoSQL database technologies used in advanced Big Data application environments. These innovative technologies provide capabilities that simplify and accelerate the handling, processing and retrieval of massive amounts of data, with each of them introducing new concepts, terminologies, and semantics.
Even more impressive, are the capabilities derived from using various combinations
of these systems with other technologies, such as Hadoop, as part of a Hybrid Big
Data Environment (HBDE). HBDE’s being so highly capable that they have quickly become
the typical way these technologies are used, replacing previous approaches where
these technologies were used in isolation.
Interestingly, while these technologies are typically not used in isolation, those charged with monitoring and managing todays HBDE’s are compelled to monitor and manage those systems in isolation as the monitoring/management tools and interfaces available for these systems are typically platform and technology specific. The monitoring/management tools have simply not kept up with how these systems are actually used.
Experience demonstrates that monitoring/managing using platform and technology specific tools increases complexity, cost and confusion, leaving blind spots that can seriously affect performance, availability and result in increased risk. That same experience has also clearly demonstrated that managing/monitoring holistically, using a platform and technology independent paradigm to be the superior alternative.
The objective of the Montauk project, is to provide monitoring and management services that:
- Address the specific monitoring and management requirements of the underlying Big Data technologies in a comprehensive manner
- Enable monitoring and management of these technologies in a holistic, platform and technology independent, normalized fashion
- Provide intuitive interfaces that provide instant and automatic access to relevant information and services