A Review of Monthly Operating Costs for a Startup’s Engineering Infrastructure.

By: Kuba Fietkiewicz

Assuming that you have completed a successful training session at one of the awesome local startup accelerators (Muckerlab, StartEngine, Amplify.la, or Launchpad.la, or you get picked for YC); you have received some immediate traction with 100K users signing up to find out more about your idea; and your idea got funded with $XMM series A; you’ll need to start thinking about the services that you will need to get your product to market, and how much you will be paying for those services in the course of running your business.  Your startup is not only going to need human talent, which will undoubtedly be your largest expense, and some kick-butt office space for you to create your masterpiece (#2 expense), but also the infrastructure and tools to get your product built.  But what kind of infrastructure do you need at this time in your venture, and how much will it cost to get you to Alpha and beyond?

The major components that make up the infrastructure to release your product to customers revolve around content delivery, engineering productivity, and customer relationship management.  Your content delivery platform includes the actual infrastructure that your application will run on, including a chosen Platform-as-a-Service (PaaS), a content delivery network (CDN) for static content, and monitoring to make sure it’s all up and satisfying your users. Engineering productivity tools that are essential in building a digital product include a source repository, continuous integration, defect tracking, testing, project management, and team communication.  Finally, you need infrastructure to help you communicate with your customers.  This may include a customer relationship management (CRM) system with a service desk, and some sort of facility for communicating with your entire customer base.

The following is a brief analysis of some popular vendors providing each of these services, including why they were chosen, and the expected monthly costs based on some base assumptions, broad generalizations, extrapolations where there was not enough data, and downright lies. This is by far not a comprehensive list of tools or vendors, and I encourage constructive discussion on the utility of these, and whether I missed some important ones.

Content Delivery Infrastructure

Assuming that you have a popular web application expecting 1MM page views per month, I picked a fairly generic setup of 4*XL application servers, and 2*XL db servers in multiple availability zones, and a CDN for static content delivery of 5MB per page view.  One can design this kind of system a hundred different ways, using smaller instances and scaling horizontally, or larger ones and scaling both vertically and horizontally, adding caching layers, etc, etc.  This post assumes that we did our due diligence and that these are the right choices under the circumstances.

Application Platform

The platform is where you will deploy your code, it is what will serve your application to your users.  It should be easily scalable, and easy to manage. The choices for major PaaS providers that support the major development stacks include Amazon Elastic Beanstalk which supports Ruby, PHP, Node, .Net, Python, and Java ¹; Heroku which supports Ruby, Node, Java, Python, Closure, Scala ²; or Engine Yard which offers a platform that supports applications written in Ruby, Node, and PHP ³.  Amazon is the most flexible, and cheapest solution, as you can either use their PaaS offering (Beanstalk), or you can build your own a-la-carte platform from the complete set of infrastructure services that they offer.  With Beanstalk, Amazon combines the infrastructure services that you need to deploy a scalable app for you, removing the need to do it a-la-carte, and the beauty is that you also only pay for the underlying platforms that you are using ⁴. In contrast Heroku and EngineYard are platforms built on top of the Amazon stack ^5,6, and must therefore charge a premium to make it profitable.  Heroku, though roughly twice the cost of Amazon ⁷, is from my personal experience the easiest platform to work with, and IMHO is the clear engineer favorite for getting from zero to hero with the least amount of pain.

Winner: Amazon Elastic Beanstalk
Reason: Price & Flexibility
Cost: (based on assumptions)

App Servers: $1,800/mth app servers
Db Servers: $2,000/mth RDS db servers
Total: $3,800/mth ⁸

Close Second: Heroku
Reason: Ease of use
Cost: (based on assumptions)

App Servers: $3,600/mth
Db Servers: $3,200/mth
Total: $6,800/mth ⁹

CDN

A CDN offloads the delivery of static content from your app servers, and delivers it to your users from servers that are closest to your users.  This helps reduce costs and improve the performance of your app.  The number of CDN providers and resellers is pretty high, with the most popular being Amazon’s CloudFront, EdgeCast, and LimeLight ¹⁰.  Amazon is the clear winner, with big reach, and competitive pricing.

Assumptions: 1MM page views/mth * 5MB/page, US Only servers

Winner: Amazon CloudFront
Reason: Price & Flexibility
Cost: (based on assumptions)

Total: $607/mth ⁸

Monitoring

The monitoring of the health of your infrastructure is essential to ensure that your users can continue to use your site or buy your widgets.  If you use Amazon services, and especially Elastic Beanstalk, you can take advantage of CloudWatch.  Like the rest of Amazon’s services, not entirely intuitive or easy to use, but at least it’s there.  My personal favorite for monitoring is NewRelic, which gives you pretty decent insight into your application’s ability to serve requests, and send monitoring alerts when things go wrong.

Assumptions: we are monitoring 4 application servers.

Winner: NewRelic
Reason: Ease of use
Cost:

Total: $596/mth ¹¹

Outside of human capital and office space, your platform is the next highest monthly expense.  In total, the platform can easily run you a cool $5,000/month.  It is debatable whether the extra $3,000 expense/month for Heroku is worth it.  As an engineer I appreciate the simplicity of their tools and the time that they save me.

Engineering Productivity Tools

Source repository

Git has won the source revision control battle, with it being offered in the cloud by Github and Atlassian’s Bitbucket.  While it is free for open source projects on both sites¹², your super secret sauce will need a place where all of your engineers will collaborate in private.  It’s amazing how easy it is to create new repositories, passing Github’s limit of 20 repositories for $50/month before you know it. Of course you can always upgrade later.  You can alternatively use Bitbucket where the pricing model is much better for private repositories¹².

Assumptions: 20 private repositories, 5 engineers

Cost comparison:

Github: $50/mth
Bitbucket: $25/mth

Winner: Github
Reason: Personal preference
Cost:

Total: $50/mth

Continuous Integration

The choices for continuous integration have typically been to build your own instance of Hudson or Jenkins in house.  While many still do, companies like CircleCI have sprung up, claiming that they remove the 8% of a team’s engineering time that it takes to maintain your own CI server ¹³ .  Given that engineer time is the most costly part of your startup, this is not an insignificant savings, and a worthwhile thing to outsource.  The companies in this space include CircleCI, Codeship, and Solano Labs.

Cost comparison:

CircleCI: $49/mth –10 projects, unlimited builds, unlimited build time, 1 container ¹³
Codeship: $50/mth – 10 projects 1,000 builds, unlimited build time, 2 concurrent builds ¹⁴
Solano: $50/mth  – 4 workers, 40 worker hours ⁴⁷

Winner: CircleCI
Reason: Value for money

Total: $49/mth

Close Second: Codeship
Reason: Tight integration

Total: $50/mth

Defect Tracking

Defect tracking and resolution is an essential as part of your development process.  There are a number of options in this space, from basic open source projects such as Bugzilla or Mantis which you can either run yourself or have hosted, to the “Cadillac” solutions from Atlassian’s Jira and FogCreek’s FogBugz which also provide project management utilities^15,16.  Even Github is in this space with Github issues, however reporting isn’t quite there.

Assumptions: 10 person team

Cost comparison:

Github issues: free with paid github account ¹⁷
Hosted Mantis: $5/mth ¹⁸
Hosted Bugzilla: $15/mth ¹⁹
Hosted Jira: $50/mth (all features) ²⁰
FogBugz: $300/mth ²¹

Winner: Jira
Reason: Full featured, nice integration with other tools, excellent reporting
Cost:

Total: $50/mth

Testing

In a small group with a limited budget everyone should undertake the role of functional tester at all times.  You should also ultimately eat your own dog food, and through regular use of your app be able to know where it fails on a regular basis.  If you lack the time to do this, or feel the need for a more diverse testing group, services such as uTest and 99tests offer functional as well as usability testing services ²².  Though they claim to remove the need for at least 1 testing resource, the cost is pretty steep at $499/testing cycle, or if you use them for every sprint, you may be looking at a hefty bill if you have weekly sprints.

Winner: Do it yourself
Reason: Eat your own dog food
Cost:

Your hourly rate

Distant second: uTest
Cost:

$1,996/mth

Project Management

Project management tools allow you to define your project deliverables, track their status, and ultimately gives you insight into when features will be delivered and how much they will cost.  A number of different tools exist in this space.  While you can get away with not using cloud solutions for this, and use MS Project, or even just Excel, one gets a lot of value out of cloud-based solutions, especially when you work on a decentralized distributed team.  When you have remote engineers, or use staff supplementation, the ability to share, manage, and communicate project requirements and status from one place that everyone has access to is essential.

Assuming that you use some sort of Agile/Scrum process variant, you have many options.  FogCreek has the free ²³ Trello, there’s Pivotal Tracker²⁴, Atlassian has Jira Agile²⁵, and there are also Sprint.ly²⁶ among others.  They are all great, with the lesson being that if you’re not using one of these, your startup is probably going to fail.

Winner: Trello
Reason: Free, great mobile apps
Cost:

Total: Free ²³

Close Second: Pivotal Tracker
Reason: Excellent interface
Cost:

$50/mth²⁷

Close Third: Sprint.ly
Reason: Excellent interface
Cost:

Total: $140/mth (team of 10)²⁸

Communication

Speaking of decentralized teams and real-time asynchronous communication: you need it.  Email comes close, but is not real-time enough.  IRC is better at real-time, but not the best interface.  A great tool will act like IRC, will email you if you want, will allow you to post images or link to your source repository, or your CI, and will allow people to send you messages after you log off so that they will be there when you come back.  The offerings in this space are Hipchat from Atlassian ²⁹, and Campfire³⁰ from 37signals.

Cost comparison:

Campfire – $12/mth – 12 people, 1GB storage ³¹
Hipchat – $10/mth – 10 people ³²

Winner: Hipchat
Reason: More features³³
Cost:

Total: $10/mth

The total for all of these chosen engineering tools is only $160/month, not including testing.  This is incredibly cheap, and money well spent to keep your code organized, your communication flowing and your project on track.  You could spend as much as $500/month, though I’m not sure you would get that much more for the money.

Customer Relationship

CRM

Your customer relationship system should at the very least give you one view of all your interactions with a customer.  This gets complicated pretty fast when you have 100K+ customers so a good tool for this essential.  The basic customer CRMs include Zendesk³⁴, and Insightly³⁵, with SugarCRM³⁶ as one of the more widely used solutions³⁷.

Cost comparison:

Insightly – $50/15users/mth/ unlimited contacts ³⁸
Zendesk Pro – $59/user/mth ³⁹
Sugar Enterprise – $60/user/mth – many many features ⁴⁰

Winner: SugarCRM
Reason: Features
Cost:

Total: $60/mth

Email Communication

One of the most overlooked, and highest value re-engagement tools is your e-mail list.  Expect to send 4-10 emails to your users each month, depending on the relationship that you have with your users.  The major providers helping you manage that communication include Constant Contact⁴¹, MailChimp⁴², and Vertical Response⁴³.

Cost comparison:

MailChimp: $379/mth/100K users ⁴⁴
Vertical Response: $600/mth/100K users *Estimated as website pricing only up to 40,000 users⁴⁵
Constant Contact: $1,200/mth/100K users* Estimated as website pricing only up to 2500 users⁴⁶

Winner: MailChimp
Reason: Cost
Cost:

Total: $379/mth

The total for the best tools to work with your customers will cost you at least $440/month, and can run upwards of $1,200 for the most expensive list management tools.

Outside of engineering staff and office space, when you add up the platform, the engineering productivity tools, and the customer relationship tools, given the same choices your startup costs can be in the ballpark of $6,000/month, and can very easily reach $10,000 or beyond depending on your preferences of one toolset over another.  The bulk of this expense will be on the platform, then customer management, and finally tools to aid in your product’s development.  Because of this, optimization and validation of your platform spend seems like a worthwhile exercise for your fledgeling business.

Resources

1. http://aws.amazon.com/elasticbeanstalk/
2. https://www.heroku.com/features
3. https://support.cloud.engineyard.com/entries/21009842-engine-yard-technology-stack
4. http://aws.amazon.com/elasticbeanstalk/
5. http://www.aws-partner-directory.com/PartnerDirectory/PartnerDetail?id=1722
6. https://www.engineyard.com/products/technology
7. http://stackoverflow.com/questions/9802259/why-do-people-use-heroku-when-aws-is-present-whats-distinguishing-about-heroku
8. http://calculator.s3.amazonaws.com/calc5.html
9. https://www.heroku.com/pricing
10. http://blog.streamingmedia.com/2012/08/updated-list-of-vendors-in-the-content-delivery-and-transparent-caching-markets.html
11. http://newrelic.com/pricing
12. http://www.infoworld.com/d/application-development/bitbucket-vs-github-which-project-host-has-the-most-227061?page=0,1
13. https://circleci.com/pricing
14. https://www.codeship.io/#pricing
15. https://www.atlassian.com/software/jira
16. http://www.fogcreek.com/fogbugz/features/
17. https://github.com/plans
18. http://www.a2hosting.com/mantis-hosting
19. http://devzing.com/index.php
20. https://www.atlassian.com/software/jira/pricing?tab=ondemand
21. http://www.fogcreek.com/fogbugz/Pricing.html
22. http://www.benchmarkqa.com/wp-content/uploads/2012/04/Crowdsourced-Testing-Companies.pdf
23. https://trello.com/
24. http://www.pivotaltracker.com/
25. https://www.atlassian.com/software/jira-agile/overview
26. https://sprint.ly/
27. http://www.pivotaltracker.com/pricing
28. https://sprint.ly/#pricing
29. https://www.hipchat.com/
30. https://www.campfirenow.com/
31. https://www.campfirenow.com/signup
32. https://www.hipchat.com/pricing
33. https://www.hipchat.com/compare
34. http://www.zendesk.com/product/key-features
35. http://www.insightly.com/features
36. http://www.sugarcrm.com/
37. http://www.staff.com/blog/comparison-of-crm-software/
38. https://insightly.com/pricing
39. http://www.zendesk.com/product/pricing
40. http://d2owqhhe2x3j50.cloudfront.net/media.sugarcrm.com/datasheets/7/EditionsComparison_7_04-13-01-LR.pdf
41. http://www.constantcontact.com
42. http://mailchimp.com
43. http://www.verticalresponse.com
44. http://mailchimp.com/pricing/
45. http://www.verticalresponse.com/pricing
46. http://www.constantcontact.com/email-marketing/pricing
47. https://www.solanolabs.com/product