Elastic’s “fork” of the Elasticsearch open-source software (OSS) codebase means you will be writing much larger checks to Elastic than you previously did. Paying for the X-pack was the first step to monetize the work they did to fix bugs and enhance the numerous plugins. Now, the second installment of “pay up” is coming to your doorstep.

We know many customers who are using the Elastic stack but complain that the costs surrounding this “free” open source technology are quite high. The cost of running Elastic can even be more than Splunk in some cases, which is well-known for being too expensive.

Elastic’s costs are about to get a lot higher. Moreover, if you are a Managed Service Provider (MSP), or another vendor of software that uses Elastic’s storage as the basis of your derivative and proprietary cloud service which you resell to your customers, then expect to get a hefty invoice from Elastic N.V. for the privilege of using their current code base as the basis of your cloud service. Alternatively, you can use the existing Elastic code base and work on your innovations, which means you will be using outdated code in a year.

Why?

Elastic recently announced that it was going to “fork” its codebase on Elasticsearch at version 7.11. The Kibana visualization engine is being forked as well. We believe the reason is the corporation that manages the open-source codebase was tired of Amazon taking Elastic’s work for free and reselling it to AWS customers for a profit. Elastic announced that the license for version 7.11 would become a Server Side Public License (SSPL) and no longer Apache 2.0 GPL.

Neither Elastic, AWS, nor anyone else makes money from the simple Elasticsearch binary. They generate revenue from the managed service they provide. As anyone who has tried to stand up an Elasticsearch stack knows, running and managing a distributed system like Elasticsearch is hard, and providing a managed service to less technically adept customers can solve that end user’s difficult technical problem. However, Elastic simply can’t compete against an enormous vendor like AWS because the customer base of AWS is more likely to take the path of least resistance as the AWS offering of Elasticsearch is integrated with other AWS services.

The Elastic announcement and subsequent comment didn’t come as a total surprise to people who work closely with the source code of these projects. Over the last couple of years, Elastic has slowed the pace of open-source projects and began moving away from investing in them. Instead, Elastic has driven the vast majority of innovation to its proprietary, commercial, “X-Pack” - which generates revenue to Elastic.

Elastic states,

"This change in source code licensing has no impact on the overwhelming majority of our user community who use our default distribution for free. It also has no impact on our cloud customers or self-managed software customers…Our license change is aimed at preventing companies from taking our Elasticsearch and Kibana products and providing them directly as a service without collaborating with us."

In plain language, Elastic is tired of those companies (specifically AWS) who are using the sweat and dollars of Elastic to resell them while not paying Elastic for the efforts of that labor. Those resellers benefited from all the updates and bug fixes that Elastic released while paying Elastic nothing. This is purely a move about getting a first count of the money and has nothing to do with “open source.”

MongoDB, an open source NoSQL database company came up with the SSPL license a few years ago as it faced the same problem: AWS was using their product, using all the updates, and then charging customers, but no revenue flowed back to MongoDB.

SSPL Contains a Poison Pill

Unfortunately, the SSPL has a major problem. AWS or any other cloud provider that offers Elasticsearch services under the SSPL must agree to the SPPL’s “poison pill.” If you offer such services under SSPL, you must open source all your software and services in your hosting cloud infrastructure. That includes your entire codebase. Unless, of course, Elastic has “blessed” it and made an exception.

The vast majority of AWS software is already open source, but AWS isn’t likely to ever agree to open source the rest of it. Neither is anyone else as it would be equivalent to taking a “scud missile,” enhancing it to make it into a “cruise missile,” and then saying to the world, “here you go, these are all the plans and schematics for all our innovations to turn that scud missile into a cruise missile.”

That’s not going to happen.

Elastic began the transition away from a free open-source product by adding proprietary features to its codebase in a package called “X-Pack.” Even the basic security plugin, as part of the X-Pack, requires a license. In February 2020, AWS “forked” the X-Pack offering, and introduced “Open Distro for Elasticsearch” It’s a value-added Elasticsearch enhancement that contains a series of AWS sponsored plugins added on top of the open-source software (OSS) version of Elasticsearch, and includes AWS versions of security, alerting, RBAC, Webhooks, and SQL support to try and bridge the gap with the X-Pack plugin. Before this change in the license, Elastic offered two different binary options for download: an Elastic-licensed version, and an Apache-2.0-licensed version. The Apache-2.0 licensed version doesn’t include their commercial X-Pack plugins. The AWS Open Distro for Elasticsearch uses the Apache-2.0-licensed binary, so when you download it, you’re getting the Apache-2.0-licensed stack. When the AWS security plugin was added for “free” as part of the AWS Open Distro package, Elastic had few options remaining to deal with the competitive threat of AWS.

Elasticsearch is also available on other major public clouds such as Elastic on Azure or Elastic on Google Cloud, but in those cases, Elastic has a business relationship with those cloud providers meaning that Elastic is paid by them. That’s not the case with Amazon AWS who created a managed version of Elasticsearch and began to build its customer base and generate revenue from those customers. Its also true that other vendors are providing Elasticsearch as the base of their proprietary service so that Elastic is not receiving any money in return.

We believe that all customers are rightly concerned about the implications of the SSPL on their own business.

No more freebies from Elastic

Even though Elastic claims that users of Elastic won’t be affected, Elastic’s cloud offering is far more expensive than Amazon’s offering. Furthermore, Elastic’s offering actively resists any integration with AWS, so the only option is using AWS marketplace which is effectively the same as setting up your cluster on EC2 and having the admin team look after it. As a result, it’s a major problem for Elastic users who have their systems in AWS.

IT staff might want to inquire with the lawyers in their respective legal departments (that would be fun to watch) to determine their business risk of using Elasticsearch as part of their proprietary ElasticSearch-based cloud service. Unfortunately, the “poison pill” remains.

Section 13 of the SSPL states:

"If you make the functionality of the Program or a modified version available to third parties as a service, you must make the Service Source Code available via network download to everyone at no charge, under the terms of this License. Making the functionality of the Program or modified version available to third parties as a service includes, without limitation, enabling third parties to interact with the functionality of the Program or modified version remotely through a computer network, offering a service the value of which entirely or primarily derives from the value of the Program or modified version, or offering a service that accomplishes for users the primary purpose of the Program or modified version."

The licensing terms of Section 13 get better. Paragraph 2:

"Service Source Code" means the Corresponding Source for the Program or the modified version, and the Corresponding Source for all programs that you use to make the Program or modified version available as a service, including, without limitation, management software, user interfaces, application program interfaces, automation software, monitoring software, backup software, storage software and other hosting software, all such that a user could run an instance of the service using the Service Source Code you make available.

In other words, everything that you create using new versions of Elasticsearch or Kibana as the base of your proprietary product must be made freely available to the community, and your competitors.

This phrase looks like legal fun:

"…enabling third parties to interact with the functionality..._"

A broad definition of that phrase would include a significant number of use-cases that define why organizations use Elasticsearch in the first place. The central argument in favor of the SSPL was that it didn’t outright prohibit competitors so that it could be classified as an “open-source” license. But the SSPL license has the direct effect of making SaaS competition unfeasible, so it’s not “open-source” in spirit. During an Open Source Initiative (OSI) review of the license, MongoDB submitted new versions of the license that clarified potentially unintended effects of the contentious clauses, but the license was never OSI-approved, and MongoDB is still using the most problematic version of the license (v1). At this point, Elastic is being careful to say "open" and not "open source", as they understand that the SSPL is not OSI open source approved.

Commercial Nightmare?

If you are using Elasticsearch as part of a cloud service, then must you give up that proprietary source code? Are you putting your company’s business “at-risk” by using Elasticsearch? For example, if you are offering dashboards for monitoring, that are driven by Elasticsearch and Elasticsearch is in the background, it’s not clear that you are free of SSPL. Risk exists since you are still providing Elasticsearch as part of your proprietary “service.”

Assume you are doing something seemingly innocuous in the cloud. You are a retailer using Elasticsearch to run a search bar on your website. Women’s shoes: Heels vs. flats, lowest price vs. highest price. You may have risk. Assume that you are running a managed Security Operation Center (SOC) service and offering Elasticsearch functionality as a service to run your service. Your risk is probably quite high that you are exposed to SSPL.

The stock market loved the “fork” announcement, as the stock price gapped up by 11% at the open on Jan 14, before settling down and closing up 6% for the day. Recently, as customers have expressed concern about the meaning of the fork to their respective businesses, Elastic’s stock price has drifted downwards.

The bottom line for commercial enterprises using Elastic as the base of their product is that it’s a potential nightmare: either pay fees to Elastic to continue using their services, or your cloud provider will make you switch to their homebrew ELK stack based on the software from before the new license took effect.

What this means is that your costs will go up since their costs will go up.

The Rescue

The LogZilla NEO platform utilizes a patented Deduplication algorithm which significantly reduces the volume of data sent downstream, operates at full fidelity in real-time, without any data loss so that the precise data stream can be rebuilt for audit purposes. The moment that the data is ingested, is the exact moment it is indexed, is the same moment it appears on your monitor. True real-time and full fidelity of the real-time data stream. LogZilla NEO’s deduplication maintains all your source data and your source-tracking to maintain that data integrity. Along with two additional patents on our database which significantly reduces the number of resources used downstream, any user can continue using their favorite SIEM or other cybersecurity platforms. Our backend architecture was designed to be completely and easily separable from the front end, so MSPs could easily write their UI if they choose. LogZilla NEO has also defined very wide bands of data volumes at very inexpensive rates to ensure that users can see all their data, not just the data they “can afford.”

Why do you care?

If you have to pay even more for Elastic and all the related services, then you need to control your downstream licensing and infrastructure costs. LogZilla NEO does that. Depending on the configuration of your IT Architecture, LogZilla NEO gets 40-60% dedup out of the box, and about 70% dedup with some tweaking to the unique characteristics of your IT Infrastructure.

Positioning LogZilla NEO in front of Elastic or any SIEM or cybersecurity platform means that the reduced data loads and reduced need for cloud resources will save you money. It’s that simple.

Deploy LogZilla in 30 seconds or schedule your 10-minute DEMO to see if LogZilla is right for you.

‍