APM系列-国外新兴厂商New Relic vs. AppDynamics

前：

New Relic的上市使得IT和资本界开始重新重视APM，当然跟传统APM相比，New Relic还是有相当的创新，另外还有一点是目前的创业潮导致的企业级需求增大。

In recent years, IT projects seem to have stopped asking “which APM solution should we buy”, and have started asking “should we buy New Relic or AppDynamics?” Given the speed at which these two companies are innovating, the many product comparisons available on the web quickly become outdated. This comparison is a little more “high level”, so hopefully it will help give people some context for further research for several years to come. Please read some (recent) technical comparisons too, and take advantage of the free trial periods to install and try both products.

A shared history

Both AppDynamics and New Relic have their roots in Wily Technology. Wily was sold to CA Technology in 2006, giving CA their APM suite (Introscope, CEM, etc.).

Lew Cirne founded Wily in 1998, and left CA in 2007 before starting New Relic in 2008 (“New Relic” is an anagram of “Lew Cirne”). Jyoti Bansal was a product architect at Wily in the last year before it was sold to CA, and continued in that role at CA for another 2 years before leaving to found AppDynamics in 2008.

Both New Relic and AppDynamics are “market disruptors” (from the Clayton Christensen Innovator’s Dilemma definition) in the APM space. They both both have similar products and are building on a fresh platform. Traditional APM vendors are now playing catch-up to their new, tightly-focussed, faster-moving competitors.

In December 2014 New Relic launched on the stock market, with shares rising to $33.82, giving them a valuation of ~$1.4 billion on the day of the IPO. AppDynamics has not yet had an IPO (but it is sure to happen soon). In July 2014 they took a funding round of $120M. This funding round gives them a theoretical valuation of ~$1B. This will be validated when they go public.

Note: I am currently working on a “History of New Relic” blog post, which will go into a little more detail.

A better way of buying Enterprise software

Both AppDynamics and New Relic are riding the “consumerization of the enterprise” trend. Customers at large companies are wanting to buy software without all the hassle and expense of a heavy-weight enterprise sales process.

The sales forces at traditional business software companies spend days, weeks, and even months installing, explaining, and training customers on their products. To flip this model, New Relic has borrowed from the consumer technology playbook and made a product meant to provide enough visual pop to attract customers on its own. It’s also designed to be easy enough to use that customers can install it without the help of a knowledgeable salesperson.
…business software makers used to rely on “good PowerPoint and sales teams” to make up for a product’s deficiencies. But that “just doesn’t work anymore.”
—- Business Week: Death of the Salesman

The old way of buying software	The new way of buying software
Product pricing is secret. You must have at least one meeting with a sales rep before they will tell you any pricing information. During the meeting they will try to figure out how much budget you have, and how important the problem is to you. Every customer gets a different price depending on what the sales rep thinks they will be willing to pay.	Product pricing is public. Look up the pricing information on the website. Email or call a sales rep if your situation doesn’t neatly fit the licensing model or if you want to buy a huuuge quantity (and think they might give you a discount)
Software is sold with a high upfront cost, followed by a yearly “support and maintenance” fee.	Software is sold under a subscription model(monthly or yearly). Pay for what you use. Don’t pay for “shelf-ware”.
A small number of customers, each paying a large amount of money. Some enterprise software products might only have a hundred customers, which means that they suffer from more bugs due to the combinatorial explosion of database versions, operating system platform levels, permission levels, etc. in the operating environment.	Sold cheaply to many. The incremental cost of software is very low. If the cost of sale can be kept low and the potential market for the product is large, then it becomes viable to have a volume-oriented business model. It seems counter-intuitive for the cheaper product to be better than the expensive product, but this is the case with this model.
Sold to C-level execs who will never use it themselves Because the software is so expensive, it should be sold at the highest organisational level (look for key words like “strategic”, “center of excellence”, or “company-wide initiative”). The main decision-maker (who controls the budget) is not the same person who will be actually using the product, so they tend to prioritise features that real users don’t care about. Compares competing products with feature checklists, rather than actually using the products (encouraging feature-bloat from vendors).	Cheap enough for a dev lead to pay for it with his company credit card. Projects that haven’t thought to budget for it can still afford to buy it. The person buying the product is probably going the be the same person who ends up using it.
A full-frontal assault by salesmen in expensive suits. Lots of meetings, lots of demos, forming “relationships” with stakeholders (playing golf, going drinking, remembering the names of their children), identifying internal champions and blockers, shepherding the sale through the internal procurement process.	A low-touch sales model. Give the customer the information they need to make a purchasing decision themselves. Available to answer questions/offer assistance if needed, but ultimately the sale is driven by the purchaser, not the vendor. The opposite of the “pushy salesman” scenario.

Most of what I have just said, was already said better by Joel Spolsky back in 2004:

…software is priced three ways: free, cheap, and dear.
1. Free. Open source, etc. Not relevant to the current discussion. Nothing to see here. Move along.
2. Cheap. $10 – $1000, sold to a very large number of people at a low price without a salesforce. Most shrinkwrapped consumer and small business software falls into this category.
3. Dear. $75,000 – $1,000,000, sold to a handful of rich big companies using a team of slick salespeople that do six months of intense PowerPoint just to get one goddamn sale. The Oracle model.
All three methods work fine.

Notice the gap? There’s no software priced between $1000 and $75,000. I’ll tell you why. The minute you charge more than $1000 you need to get serious corporate signoffs. You need a line item in their budget. You need purchasing managers and CEO approval and competitive bids and paperwork. So you need to send a salesperson out to the customer to do PowerPoint, with his airfare, golf course memberships, and $19.95 porn movies at the Ritz Carlton. And with all this, the cost of making one successful sale is going to average about $50,000. If you’re sending salespeople out to customers and charging less than $75,000, you’re losing money.

The joke of it is, big companies protect themselves so well against the risk of buying something expensive that they actually drive up the cost of the expensive stuff, from $1000 to $75000, which mostly goes towards the cost of jumping all the hurdles that they set up to insure that no purchase can possibly go wrong.
— Joel on Software: Camels and Rubber Duckies

If you imagine a continuum with a traditional high-touch enterprise sales process on one end and a no-touch sales process on the other (no sales reps, just a website, some Google ads and a signup form), New Relic and AppDynamics would be somewhere in the middle, with AppDynamics a little closer to the high-touch end.

Both companies have a similar number of employees, but AppDynamics seems to have a higher ratio of sales people to engineers. Personally, I think it is better to have more engineers working on making a product better, but I am an engineer who gets more excited by a good product than by a good sales pitch. AppDynamics’ bigger sales force allows them to go head-to-head with traditional APM vendors (like CA) at big banks, and to try the expensive top-down sales process, but still take advantage of the fact that there are customers who will buy with a low-touch sales model through their website.

APM stack vs APM stack

Both New Relic and AppDynamics have a very similar suite of APM products.

APM Component	AppDynamics	New Relic
Synthetic Monitoring	Integrates with Apica WebPerformance (3rd-party)	New Relic SYNTHETICS. Based on Selenium (web only).
End-User Monitoring (Browser-based)	AppDynamics Web End-User Experience Management (EUEM)	New Relic BROWSER
End-User Monitoring (Mobile Apps)	AppDynanics Mobile Application Monitoring	New Relic MOBILE
App Server Diagnostics	App Agents available for Java, .NET, PHP, Node.js, and C++	New Relic APM supports Java, .NET, Ruby, Python, PHP, and Node.js
Infrastructure Monitoring	AppDynamics Machine Agents	New Relic SERVERS New Relic PLATFORM
Database Monitoring	AppDynamics for Database. Gives actual execution plan of slow database queries.	Limited coverage with New Relic PLATFORM. Database metrics/counters only.
Data Analytics	Application Analytics	New Relic INSIGHTS

I think that New Relic wins in the Data Analytics space (New Relic INSIGHTS), and that AppDynamics wins in the Database Monitoring space – they acquired a company that had a good product for analysing traditional (RDBMS) databases, although this is something that every DBA will already have tools for.

Due to its larger userbase/developer community, New Relic has more user-contributed PLATFORM monitors, so you are more likely to find support for bleeding-edge NoSQL databases etc.

On-premises options

When you are trying to sell software to a large company, there can be many stakeholders (any of whom can spoil your chances of closing the sale), so there is pressure to try to make everyone happy – even if they are asking for a feature that you know they don’t need. Sometimes it is easier to just build the useless feature, rather than spend time trying to re-educate every client you are talking to.

When selling a SaaS product, a common concern is allowing data to be sent outside of the company firewall. You will hear words like “confidential”, “sensitive data”, and mutterings about “data sovereignty”. These objections ignore the facts that APM tools don’t touch any personally identifiable information, that the most sensitive data they store is the number of visits your website gets, and that the US government isn’t going to subpoena anyone so that they can look at your slow SQL queries.

In this case, AppDynamics has rolled over (and accepted the higher support costs this will incur) and offered an on-premises version of their software that you can install on your own server.
New Relic stood firm and only offers their own hosted version of their software.

If someone in your organisation forces you to go with an on-premises APM solution, you now have to worry about:

• Provisioning hardware (will you need to organise servers for a dev/test environment too?)
• Capacity Planning (How much storage? How much memory/CPU? Will this change over time?)
• Backups (and testing your backups)
• Architecture (Firewall ports, Failover/DR, Where should the system be located on the network?)
• Software upgrades for the underlying platform (OS patches, etc.)
• Software upgrades for the APM product (wait for the users to complain that the new feature that all the SaaS customers instantly have access to will not be available until your next scheduled upgrade cycle)
• Hardware upgrades (refreshing the underlying hardware every x years)
• Staff (Whose job will it be to maintain/manage the system? Whose budget does this come out of? Who do they report to?)

User community and Developer mindshare

One of the most important non-technical things about a piece of software is its user community. Users talking to each other, writing blog posts, tweeting, and asking questions in forums all helps to make improve the lives of end-users, and outcomes for the business. It also makes it easier to hire people who are already familiar with the product.

AppDynamics and New Relic have worked hard to develop a user community – both companies have a free version of their product in addition to the fully-featured paid versions, but New Relic really seems to have won the hearts and minds of the developer community. New Relic has significantly more users (both paid and free) than AppDynamics, and this gives them a strong “network effect”.

Doing a search for both companies on Stack Overflow shows 1652 results for New Relic and only 112 results for AppDynamics. Google Trends shows something similar; 4 or 5 times more people are searching Google for “New Relic” (shown in red) compared to “AppDynamics” (in blue).

When I was working for an AppDynamics partner and was helping to sell AppDynamics, I would often hit a snag early in the sales process after I identified a potential customer with an application that could really benefit from APM. About 50% of the time I would discover that someone from the dev team had already installed New Relic. This was a bit frustrating.

Whenever I talk to keen, switched on developers (as opposed to fossilised devs who stopped learning new things 10 years ago), they all seem to have heard about New Relic. I don’t meet many who have heard about AppDynamics. I think that New Relic has already won the battle for developer mindshare.

Cost

Imagine you are going shopping for a product…let’s say it’s a shirt…that you expect to pay $50 for. When you find two different shirts sitting on the shelf with price tags of $1 and $2, are you going to get hung up on the price difference, or are you going to buy the one that fits better irrespective of cost?

Last time I checked, AppDynamics was approximately twice the price of New Relic but I don’t think that this should be a major part of a buying decision. Both products have a much smaller TCO than the products from traditional APM vendors, and customers will only be buying APM solutions for web applications which are important enough to monitor – applications where downtime or poor performance will have a real business impact (guaranteed to be much greater than the cost of monitoring tools).

Both New Relic and AppDynamics are quick to install and configure. Don’t get caught in the trap of traditional APM tools that take a long time to set up. I know a couple of very expensive consultants who are 12 months into a 6 month implementation of CA APM for a small number of applications at a large bank. Spending half a million dollars on consultants on top of the software purchase cost is just adding insult to injury.

I like the fact that that both companies are quite open about their pricing (New Relic more so than AppDynamics), and that both companies have a subscription model (monthly/yearly), rather than a large upfront cost.

Conclusion and Recommendation

As a web performance consultant, I like working with the products from both New Relic and AppDynamics. Both companies have created APM solutions that I think are better than the APM products from the traditional vendors.

If I had to make a long-term bet on which company was ultimately going to win in the APM market, I would have to put my money on New Relic. They seem to be innovating faster, they have a much bigger user/developer community around their product, and they really seem to get “consumerization of the enterprise”.