Help,question about rabbitmq+Haproxy,client java

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Help,question about rabbitmq+Haproxy,client java

Aaron
    I have let only one node  rabbitmq online in my company in order to decoupling messages.
    This node has not been down for almost one month fortunately?but some basic problem,such as the HA and Load balancing need to be considered,so that  can keep all messages complete no matter when the related server or app is down.Such like,another node will  work still and keep the produce、customer healthy when a node is down.
    So,i have considered a solution to protect from losting messages,it‘s rabbitmq(mirrored queue)+Haproxy.
    I have tried two different code,php  plays a role,both HA and load balancing.
    But,java code is not enough fortunately,LOL... Testing for several times,and I get a result,that is java code doesn‘t work at all in HA,but only load balancing....,during testing,when I down a random node,producer will throw out a related session error.
    Having troubled me for almost one week this problem,and looking  sincerely forward to ur reply.
    The environment will be shown below:
Haproxy:192.168.1.61:5670
rabbit@ubuntu :192.168.1.61:5672(3.1.5)
rabbit@ubuntu2:192.168.1.35:5672(3.1.5)

PHP code: 
 <?php

$n = 100000;
for($i=0;$i<$n;$i++){
echo '<br/>'.$i;
//连接RabbitMQ
$conn_args = array( 'host'=>'192.168.1.61' , 'port'=> '5670', 'login'=>'aaron' , 'password'=> '5555','vhost' =>'/');
$conn = new AMQPConnection($conn_args);
$conn->connect();

//创建exchange名称和类型
$channel = new AMQPChannel($conn);
$ex = new AMQPExchange($channel);
$ex->setName('direct_exchange_name');
$ex->setType(AMQP_EX_TYPE_DIRECT);
$ex->setFlags(AMQP_DURABLE | AMQP_AUTODELETE);
$ex->declare();

//创建queue名称,使用exchange,绑定routingkey
$q = new AMQPQueue($channel);
$q->setName('queue_name');
$q->setFlags(AMQP_DURABLE | AMQP_AUTODELETE);
$q->declare();

$q->bind('direct_exchange_name', 'routingkey_name');

//消息发布
$channel->startTransaction();
$message = json_encode(array('Hello World!','1','2','3','4','DIRECT'));
$ex->publish($message, 'routingkey_name');
$channel->commitTransaction();
$conn->disconnect();
}

?> 



JAVA code:
ConnectionFactory factory = new ConnectionFactory();// 创建链接工厂
factory.setHost("192.168.1.61");
factory.setUsername("aaron");
factory.setPassword("5555");
factory.setPort(“5670”);
factory.setAutomaticRecoveryEnabled(true);
Connection connection = factory.newConnection();// 创建链接
factory.setNetworkRecoveryInterval(10000);
for(int i = 0; i < 10000; i++) {
Channel channel = connection.createChannel();// 创建信息通道
channel.exchangeDeclare(EXCHANGE_NAME, "fanout", durable);// 创建交换机并生命持久化
String message = "Hello Wrold " + Math.random();
// 消息的持久化
channel.basicPublish(EXCHANGE_NAME, "",
MessageProperties.PERSISTENT_TEXT_PLAIN, message.getBytes());
System.out.println("[x] Sent '" + message + "'");
channel.close();
}
connection.close();

} 

_______________________________________________
rabbitmq-discuss mailing list has moved to https://groups.google.com/forum/#!forum/rabbitmq-users,
please subscribe to the new list!

[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Help,question aboutrabbitmq+Haproxy,client java

Michael Klishin-2


On 22 July 2014 at 12:20:46, Aaron ([hidden email]) wrote:
> > so that can keep all messages complete no matter when the related  
> server or app is down.Such like,another node will work still  
> and keep the produce、customer healthy when a node is down.
> So,i have considered a solution to protect from losting messages,it‘s  
> rabbitmq(mirrored queue)+Haproxy.

There are *many* things that may fail and HAproxy won't protect you from
many of them. See

http://www.rabbitmq.com/ha.html
http://www.rabbitmq.com/reliability.html
http://www.rabbitmq.com/partitions.html
http://www.rabbitmq.com/nettick.html

You have to use publisher confirms:

http://www.rabbitmq.com/confirms.html

and be ready to reconnect and re-try publishes. In case the client
itself may crash between re-publishes, you may also need to keep a write-ahead log
(WAL) for messages. Yes, 100% guaranteed delivery is much harder than just adding
HAproxy to the mix.

> But,java code is not enough fortunately,LOL... Testing for  
> several times,and I get a result,that is java code doesn‘t work  
> at all in HA,but only load balancing....,during testing,when  
> I down a random node,producer will throw out a related session  
> error.
> Having troubled me for almost one week this problem,and looking  
> sincerely forward to ur reply.

Sorry, can you explain what "a related session error" is? If you lose a node,
HAproxy will detect that a backend is down after a moment of time. RabbitMQ
connections have a certain amount of state (e.g. active consumers and delivery tag
counters), so you need to recover them.

This is much easier for consumers, though. In recent RabbitMQ versions, consumers
will be re-subscribed if the queue master node fails. In other words, if you are
not connected to the failed node, you won't have to basic.consume again, RabbitMQ
will do it for you.

Starting with RabbitMQ Java client 3.3, there's an automatic connection/channel/topology recovery
feature:

http://www.rabbitmq.com/api-guide.html

which is usually sufficient for recovering consumers. However, the client itself does not
perform internal queueing of outgoing messages, so in the period of time that the connection
is down, you need to accumulate messages in a collection/queue in memory
and re-send them with confirmation checking. 
--  
MK  

Staff Software Engineer, Pivotal/RabbitMQ
_______________________________________________
rabbitmq-discuss mailing list has moved to https://groups.google.com/forum/#!forum/rabbitmq-users,
please subscribe to the new list!

[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Loading...