Skip to main content
No. of Recommendations: 101
An Elastic technical review.

1 - Overview
2 - Elastic Overview
3 - Compare to MDB

4 - Strengths, in Haiku

PART 3 <<<
5 - Use Cases
6 - Final Takeaways


Companies need a scalable database to handle search and analytics over a LOT of data, including ever-growing datasets like metrics. There are lots of reasons to integrate Elastic Stack into your infrastructure. Elastic Stack excels at search & analytics over:

- Full text data (ie articles, blog posts, tweets, comments)
- Terms text data (ie tags, usernames, locations)
- System logs & real-time metrics (ie systems, network devices)
- Application logs & real-time metrics (ie server-side apps, databases, APIs, microservices)
- Security/Audit logs (ie firewall logs, system audit logs)
- Numerical data (ie financial analytics, fraud detection)
- Time-series data (ie metrics, events, devices, IoT sensors)
- Geospatial data (lat/long points, geo-regions, location beacons)
- IP data (network traffic, routing logs)

These data types combine into multiple use cases they market against - but these are just the tip of the iceberg. Expect the use cases to continue to expand from here as they expand the product line as well as address more verticals specifically.

Use Case: Needing Better Search & Analytics Capabilities

This is where they started - creating a database stack that helps software development companies provide search capabilities within their architecture. It began with a focus on full-text search, but Lucene could also be utilized for indexing any type of field, and over unstructured data. Lucene caught up over the years to all those use cases, as the performance of numeric searching over time-series and geospatial data greatly improved. This made Elasticsearch and Solr more and more relevant to more and more use cases in custom development efforts. As seen in the popularity of their open-source repositories, Elasticsearch won the battle. Multiple SaaS services depend on Elasticsearch under the hood.

Some examples:
* The Library of Congress is digitizing their archives in Elastic Stack.
* Uber built a demand prediction system over Elastic Stack for UberEats service.
* Goldman Sachs built multiple internal tools, including a contract tracking system and a trade life-cycle tracking system, over Elastic Stack.
"Elastic has been one of the most interesting open source products that we've seen in the last couple years," said Don Duet, global co-head of the Goldman Sachs technology division, in an interview with InformationWeek. "What's impressive about it is how much value it can create in organizations."

Use Case: Infrastructure Monitoring

- Logs
- Metrics
- Application Performance Monitoring (APM)
- Uptime

Elastic is really pushing a wide variety of time-series and geospatial use cases around monitoring; IoT, sensor, app, network and infrastructure monitoring are all major use cases of Elastic Stack.

Elastic is really going after do-it-yourself infrastructure monitoring. There are 3 overlapping angles to using Elastic Stack for monitoring your infrastructure:
* It can ingest and search over log files output from your systems and server apps, like syslog and database logs.
* It can ingest and search over real-time metrics from your systems (like cpu/memory/disk/network usage) as well as your server applications.
* Then it can utilize APM, where you ingest your metrics straight from your apps themselves. It ties into your code directly via an APM library, available across a wide variety of software languages (Java, Javascript, Go, Python, Ruby). This becomes particularly needed if you have a distributed code base or use microservices strategy, where you really need to monitor the flow of communication and data between all your modules.

Same for networking and security monitoring. You can pull in logs from routers, firewalls and other networking equipment. Then use ML module to isolate anomalies, or view hot-spots on regional maps. So Elastic Stack allows and organization to watch their own infrastructure, networks and app stacks. This enables companies to do-it-themselves, for a fraction of the long-term cost of Splunk, New Relic and Datadog. I see those services as major competitors, where Elastic has to convince companies to do it themselves with Elastic Stack.

Use Case: Search services

- Site Search
- App Search
- Enterprise Search

It's a search engine at the core, so if you need search within your enterprise, on your web site, or within your mobile app, you are in the right place to Do-It-Yourself and embed Elasticsearch into your stack. And as discussed in depth before, Elastic is making moves here with SaaS services that provide these search capabilities directly to enterprises, without the need to host, manage or interact with the Elastic Stack themselves. But customers could always do these items themselves in the Elastic Stack with custom development.

Use Case: Analytics

- Security/Audit Analytics
- Business Analytics
- Mapping

Once you have your data flowing into Elastic, you can leverage the analytics capabilities for security and audit purposes. Utilize the ML module, or pipe it into your own analytical package (Spark, Hadoop, AWS EMR). You can use any kind of geospatial data in Elastic Search, to view traffic flows or group data into hot-spots within maps. You can use geo-fenced search filters, to search only in specific regions or overlapped geo-shapes.


Amazon is a competitor to hosting Elasticsearch. Unlike MDB, Elastic isn't combating it via licensing, but instead are combating it with a richer feature set from the X-Pack modules and other services. MDB and Elastic have different licensing battles for the same purpose - combat the cloud-vendor alternatives. MDB is trying to prevent them from using MongoDB altogether, while Elastic is trying to have differentiated features.

What Elastic says they have over AWS managed hosting:
- premium modules for ML, Security
- free modules for alerting, monitoring, SQL, Canvas
- Monitoring dashboards and APM UI
- Index curation & roll-up features (Hot/Warm/Frozen indexes)
- Elastic Map Service
- Logstash/Beats mgmt UI

Beyond hosting, I think the major competition isn't alternative open-source engines, it is their competitors in their use cases. SaaS infrastructure monitoring companies like Splunk, New Relic, Datadog, and the like are losing customers tired of the high monthly charges, who can build it themselves on Elastic Stack for a fraction of the cost. Elastic Stack is for DO-IT-YOURSELFERS and those on a budget, compared to tying into those SaaS tools where ever-growing datasets means ever-growing monthly expenses.


Elastic knew early on that they needed a complete ecosystem. Kibana is a data visualization dashboard, but also provides the interface to manage the cluster and the data within. Logstash and Beats both enable monitoring use case, and with Kibana, allow using Elastic Stack without coding. Elastic has a major focus on ML over the data, for things like anomaly and threat detection. In comparison, MDB has been catching up on ecosystem tools like Charts, but has nothing around analytics or ML tied in.

Yes, MDB has a much wider use case. But for search and analytics, there is really no alternative to Elasticsearch outside the way-less-used Solr. The choice for a company is really, does a search engine apply to our use case? If so, you go Elastic Stack. So the question of competition is really if you use Elastic Cloud or have AWS host your managed cluster. MongoDB is solely used by software development companies. Elastic Stack can be used without code! That means that, unlike MongoDB, it's not just for software developer companies -- any company can benefit. IT departments are using it just as a standalone Elastic Stack, directly integrating monitoring capabilities without needing any custom development effort. Kibana is a very easy-to-use visualization dashboard tool. IT can install Beats onto infrastructure, and suddenly it is all feeding into your cluster for DO-IT-YOURSELF monitoring.

I have spoken about Elastic before, as I attended and wrote up their ElasticON developer conference last October: Go back and re-read that now that you know what the hey they do now! Their main focus at the conference was for 2 main customer use cases: use it to monitor everything (logging + metrics + APM), and use it to help secure your network & infrastructure by building a Security Event Information Management system (SEIM) around it. I dove into more details about those use cases on that post. One highlight I continue to focus on was how Oak Ridge National Labs IT team brought their SEIM system from Splunk to in-house, and costs went from "$$$$$" to "$$" - showing they were cut by more than half. Simply put, companies with large infrastructure can save big bucks by taking a DO-IT-YOURSELF attitude with monitoring and security. Elastic directly competes with Splunk and New Relic here.

Elastic & MDB are similar companies with similar products. They are NoSQL databases that compete, and have closely matching business product strategies (both heavily focused on cloud-neutral managed hosting). Both have tried to differentiate their managed cloud-hosting service from AWS's. At a minimum, both are the authors of the database, so are absolutely the best resource to host that database for you and help you with it. But beyond that, MDB offers Stitch and Charts, and Elastic Cloud offers many add-on modules. AWS is starting to fill in the gaps with their "Open Distro for Elasticsearch", but they only cover a few of the basic X-Pack plugins so far (security and alerting). They aren't going to catch up to Elastic Stack's feature set like this. Elastic is more than happy to highlight what AWS Elasticsearch cannot offer in their marketing.

So the licensing battles are just a strategy difference on how to fend off competition from using their open-source core in a competing hosting services. MDB is fighting via their core licensing. Elastic is using their ecosystem of modules to differentiate their platform. Google and Microsoft are choosing to partner with Elastic for managed Elastic Stack hosting on their platform, instead of building a competing service. AWS is fighting it to the point of branching their own "Open Distro of Elasticsearch" that doesn't include the alternatively-licensed modules, instead having to write their own open-source security, alerting and SQL modules. AWS doesn't typically contribute to open-source. They aren't doing this out the kindness of their heart - they cannot sell managed Elasticsearch clusters without these features being present. Expect Elastic to continue expanding features to differentiate themselves. I can't believe AWS gets away with this competitive behavior, but I guess that behavior is par for the course for Amazon the retailer. If they see a way to capture a few more points of margin, they take it and cut out the middleman. When MDB changed their license, the press sold it at the time as combating Asian cloud providers, but in reality the first front was AWS.

I am going to take it a step further -- "Open Distro of Elasticsearch" shows me that AWS cannot compete against Elastic Cloud with just the core Elasticsearch, as they had to find a way to use the proprietary features that they couldn't include under the "Elastic License" they are under. Different license game than MDB, but I think it's working just as well. AWS has to find another way to differentiate their service from Elastic Cloud (besides price -- yes, AWS is cheaper). I think they are already starting to market it differently, as recently I saw a blurb touting AWS Kinesis as a data stream platform that can easily integrate directly into AWS Elasticsearch.

[Side bar to the whole "open-source database company doing cloud hosting" part: Confluent, maker of Apache Kafka, is one to watch for going forward. Kafka is a data streaming platform on a high-availability cluster. Not a database, per say, but damn close (more a persisted, high-availability message queue). Disclaimer, I am a database developer that uses Kafka a lot. Confluent hosts Confluent Cloud managed hosting service, and so if & when it becomes public, I would consider its numbers and put it up with MDB and Elastic as an extremely sticky platform for software development companies. AWS runs a competing AWS Kinesis service, but now also runs its own AWS Managed Kafka service, as that platform has a lot of momentum. Confluent has taken the same route Elastic has in changing the licensing of other components in their ecosystem, not the core database.]

Creating a managed cloud-neutral hosting service over the core platform is clearly a big money maker for these open-source companies. That's the current big revenue growth coming in. But Elastic is adding the next wave of growth -- creating their own enterprise-focused SaaS services around search and analytics. This is the two fold nature of Elastic's acquisitional prowess. It first bolted on tool sets around it's core, to build an ecosystem around itself. But the recent acquisitions are altogether different. In Swifttype and, it found companies that built themselves on Elasticsearch (as they are allowed to, by the permissive Apache 2.0 license!) for their SaaS search service for enterprises. Such a superb direction for Elastic; they can leverage their expertise plus provide an alternate path for their customers! There may be risks in this direction, but I think this has already been addressed by Elastic -- the marketing is taking a great tack in saying you can use the SaaS Service or do it on Elastic Stack yourself. Elastic is also keeping Swiftype an independent division. It's such a good idea -- find companies building on the Elastic Stack, and acquire ones that align with Elastic's use cases. They can leverage all their knowledge about the core Elastic Stack platform it is built on, but focus these SaaS services toward highly-honed enterprise solutions around search and analytics.

Very exciting, and this just seems like the start. The recent acquisition of really has me intrigued. They have a developer-focused SaaS service that integrates into code tracking services (Github, Bitbucket) and provides intelligence and search capabilities over your code base. This puts them into same developer-focused SaaS market as Atlassian. Which, as it so happens, is a market that has extreme cross-selling potential to their existing Elastic Stack customers who are using it for application development. They could combine Insight's service with app monitoring (especially APM modules) and make it a very focused SaaS service. Or it could combine with the new Elastic Enterprise Search as a differentiator over Google's offering. Whatever is coming down the pipeline from this, Elastic is going to be competing in an all new market (software development SaaS tooling). They are already in the enterprise SaaS tooling market now, and against some big names -- Swiftype directly competes with Google! With these new SaaS directions, each potentially opens up all new markets! Perhaps cutting in to Elastic Stack potential market for do-it-yourself solutions, but they capture that customer regardless.

Elastic has a land-and-expand philosophy with Elastic Stack customers; if they can get a new customer to use it for one use case, then they will find all their other use cases for it and start expanding their use from there. If on Elastic Cloud, managed hosting fees will likely increase over time. If self-hosted, customers may rise up the support tiers as their dependency increases. This is all easily seen in their $NER >130%. The new SaaS services may cut into this a bit, as this provides an alternate path for new customers to take, where they won't get into the core Elastic Stack and find other use cases. However, those SaaS services will have their own growing customer base and expanding use (as customers have more traffic and more documents, so move up the pricing tiers), so difficult to know. Regardless, as you can probably tell, I am really enamored with the dueling business strategies they appear to be navigating perfectly.

It all combines into some fascinating moves by Elastic. I walked into this research project thinking they were a MDB clone, but I now feel Elastic has a much richer story than MDB. This SaaS tooling direction took me completely by surprise. [I had here-to-fore under-estimated the Swiftype acquisition more than I should have, and I had never even heard of before this research.] I was mightily impressed by Okta after my tech review of them; I am moreso of Elastic. Perhaps due to my closeness to their product, I corralled my thoughts around this company incorrectly. Today, I cannot deny I have a new excitement around the potential here. And, if my financial bet is correct, it's just the beginning of their SaaS moves. TAM potential is completely unknown. Tomorrow they could create or acquire a New Relic or PagerDuty or Everbridge clone by combining Elastic Stack with Twilio notifications. Any SaaS monitoring service is competition to a do-it-yourself solution on Elastic Stack, but they clearly have their sights on SaaS services built on that for companies that just don't want the additional hassle of maintaining or interfacing with an Elastic Stack cluster (who'd rather avoid the do-it-yourself).

I think MDB and Elastic are such similar business models, that I'd really like to see a numbers to numbers comparison of MDB and Elastic. Nearly same revenue, nearly same growth [well, maybe... MDB just accelerated this recent Q!], nearly same market cap [then again, MDB just jumped 25%!]. I moved to have nearly equal allocations in them, and will start exiting one when it starts faltering in head-to-head comparison of their stats - I want to find the one executing better after a few Qs then move mostly to that one. Anyone in the collective want to start tracking and posting a head-to-head numbers comparison? (Please! I'm too busy pontificating here!)

In closing, I hope I convinced you, Saul and others here, to revisit your concerns about their open-source strategy. It's two sides of the same battle of keeping their cloud hosting services differentiated against the cloud vendors' offerings. MongoDB can't be hosted past v3.6, so all new features are protected from here. ELK is entirely open and free, but it's the integrated modules that require licensing, and all new features can be protected from here. It feels that Elastic letting companies utilize and embed Elasticsearch into their own products, via that permissive Apache 2.0 license, is what fueled this next phase of Elastic's revenue growth in having these "side-car" SaaS services they've acquired.

Needless to say, I have increased my allocation prior to the publication of this massive missive. I hope you learned something. I sure did, which is why I love this kind of homework.

long ESTC (7%)
Print the post  


What was Your Dumbest Investment?
Share it with us -- and learn from others' stories of flubs.
When Life Gives You Lemons
We all have had hardships and made poor decisions. The important thing is how we respond and grow. Read the story of a Fool who started from nothing, and looks to gain everything.
Contact Us
Contact Customer Service and other Fool departments here.
Work for Fools?
Winner of the Washingtonian great places to work, and Glassdoor #1 Company to Work For 2015! Have access to all of TMF's online and email products for FREE, and be paid for your contributions to TMF! Click the link and start your Fool career.