Amazon EC2 cluster costs

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

Amazon EC2 cluster costs

Jose Berardo
This post has NOT been accepted by the mailing list yet.
Sorry guys, but I tried to find out this question but I didn't get exactly what I need.

I'm developing an Android/iPhone app that will connect to a RabbitMQ cluster at Amazon's Ubuntu EC2.

I need to calculate how much it will be. I believe that it will have ~100k users.
I'm configuring the broker to have one user and one queue per mobile endpoint which will be connected over all the time (with basicConsume and manual ack - I believe that I wont need transactions).
These users and queues will be created by an unique PHP server using the REST RabbitMQ port.
The mobile endpoints will open another connection to basicPublish messages to one of my only 5 topic exchanges. This exchanges will need to deliver the message up to ~2k queues.
Each mobile endpoint will consume about 20 messages a day (produced by other endpoint, so I don't know exactly if it will be 20, 2 or 2k messages a day ;) .
Each message body will be only what the user usually types. I am using AMQP headers and parameters to all metadata.

I've read about resource usage at an old post here and I'm trying to calculate:
Just to keep one idle connection the server needs ~66kb and 8 processes (btw, I've changed heartbeat down to 60s), so:

100k connections * 66kb = 6.29Gb
100k connections * 8 processes = 800k processes (is it possible for a unique machine?)

I don't know yet if I will need to persist all the messages. At this time, I'm thinking don't persist.

Here in my madness I'm guessing that two "Linux on m3.xlarge EBS Optimized" (4 vCPU and 15Gb memory each, high I/O) in a cluster could be suitable to this project.

Am I wrong?

In this scenario, what would you use and how would you calculate? Any tips to figure out the best architecture that suits this projects or item to don't forget at Amazon's calculator will be gold for me.

Thanks in advance.