shovel with authentication

classic Classic list List threaded Threaded
22 messages Options
12
Reply | Threaded
Open this post in threaded view
|

shovel with authentication

Tomas Tulka
This post was updated on .
Hi, my conf of a shovel with authentication doesn't work. RabbitMQ 3.1.5 standalone.
Without authentication everything is ok.

Java code:
        ConnectionFactory factory = new ConnectionFactory();
        factory.setHost("localhost");
        factory.setPort(5671);
        factory.setSaslConfig(DefaultSaslConfig.PLAIN);
        factory.setUsername("guest");
        factory.setPassword("guest");
        Connection conn = factory.newConnection();

Shovel def:
     {queue1_shovel,
      [{sources,      [{broker,"amqps://guest:guest@Server1"},
        {declarations, [
                {'exchange.declare', [
                    {exchange, <<"MyQ-exchange">>},
                    {type, <<"direct">>},
                    auto_delete
                    ]},
                {'queue.declare', [
                    {queue, <<"MyQ">>},
                    {arguments,[{<<"x-expires">>, long, 15000}]}
                ]},
                {'queue.bind', [
                    {exchange, <<"MyQ-exchange">>},
                    {queue,    <<"MyQ">>}
                ]}

        ]}]},
       {destinations, [{broker, "amqps://guest:guest@Server2"}]},
       {queue, <<"MyQ">>},
       {ack_mode, no_ack},
       {reconnect_delay, 5}
      ]}

The shovel seems to be running:
{queue1_shovel,
     {running,
         {source,
             {amqp_params_network,<<"guest">>,<<"guest">>,<<"/">>,"Server1",
                 undefined,0,0,0,infinity,[],
                 [#Fun<amqp_uri.7.32597394>,#Fun<amqp_uri.7.32597394>],
                 [],[]}},
         {destination,
             {amqp_params_network,<<"guest">>,<<"guest">>,<<"/">>,
                 "Server2",undefined,0,0,0,infinity,[],
                 [#Fun<amqp_uri.7.32597394>,#Fun<amqp_uri.7.32597394>],
                 [],[]}}},
     {{2014,1,28},{16,10,5}}},

I don't get any error from Java nor in log.

Sending a message on Server1's side is ok, but I will never receive on Server2's.

Getting "guest:guest" off from brokers definitions and removing setUsername and setPassword it starts working fine.

I tried it with a different user, but the same result...

Where can be a problem?

Thank you in advance!        
Reply | Threaded
Open this post in threaded view
|

Re: shovel with authentization

michaelklishin
2014-01-28 tt <[hidden email]>
Where can be a problem?

Have you tried specifying certificate and keys as query parameters?

--
MK

http://github.com/michaelklishin
http://twitter.com/michaelklishin

_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: shovel with authentization

Tomas Tulka
> Have you tried specifying certificate and keys as query parameters?

I need only PLAIN auth and by the def. it should work with user:pass@host
And I don't see a parameter for auth in URI query parameters.

Auth works in general, because I am able to connect and send a message, the problem must be in the shovel

Thx
Reply | Threaded
Open this post in threaded view
|

Re: shovel with authentization

mc717990
If the shovel is running you have a connection to the remote system (I usually look at the web management interface to verify this, plus log files).  At that point I'd guess a routing issue is to blame.  If you can successfully publish a message to Server1 and then that message disappears and never shows up on Server2.  You can try adding:
{publish_fields, [{exchange, <<"">>},{routing_key, <<"MyQ">>}]}
to your shovel configuration.  Of course make sure the routing/binding works on both sides.  E.g. if you publish a message to Server2 that's identical to the publish to Server1, does it show up in a Queue?  

Jason



On Tue, Jan 28, 2014 at 9:29 AM, Tomas Tulka <[hidden email]> wrote:
> Have you tried specifying certificate and keys as query parameters?

I need only PLAIN auth and by the def. it should work with user:pass@host
And I don't see a parameter for auth in URI query parameters.

Auth works in general, because I am able to connect and send a message, the
problem must be in the shovel

Thx



--
View this message in context: http://rabbitmq.1065348.n5.nabble.com/shovel-with-authentication-tp32914p32916.html
Sent from the RabbitMQ mailing list archive at Nabble.com.
_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss



--
Jason McIntosh
https://github.com/jasonmcintosh/
573-424-7612

_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: shovel with authentization

Tomas Tulka
> At that point I'd guess a routing issue is to blame

but if it is true, a message shouldn't have arrived to Server2 even in case with no auth, but it does...

maybe it needs some additional setup on Server2...???
Reply | Threaded
Open this post in threaded view
|

Re: shovel with authentization

Tim Watson-6
In the URI's you're asking the shovel to connect on (i.e., "<a href="amqps://guest:guest@Server1">amqps://guest:guest@Server1" and "<a href="amqps://guest:guest@Server2">amqps://guest:guest@Server2"), do the "Server1" and "Server2" portions refer to both the correct IP address *and* the correct port on which those RabbitMQ instances are listening for SSL connections? Your java code connects to 5671, which is typically the non-ssl port. If the shovel is trying to establish SSL connections (which is what the "amqps://" prefix means) on the wrong port, it will fail. I'd expect to see something in the logs though.

Cheers,
Tim

On 29 Jan 2014, at 06:52, Tomas Tulka wrote:

At that point I'd guess a routing issue is to blame

but if it is true, a message shouldn't have arrived to Server2 even in case
with no auth, but it does...

maybe it needs some additional setup on Server2...???



--
View this message in context: http://rabbitmq.1065348.n5.nabble.com/shovel-with-authentication-tp32914p32930.html
Sent from the RabbitMQ mailing list archive at Nabble.com.
_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss


_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: shovel with authentization

Tomas Tulka
Hi Tim,
SSL works fine (a message is sent and received). The problem occurs when I add "guest:guest" to broker url and make java clients to connect with the credentials (a message is sent but not received).
Reply | Threaded
Open this post in threaded view
|

Re: shovel with authentization

Tim Watson-6
In reply to this post by Tim Watson-6
On 29 Jan 2014, at 11:01, Tim Watson wrote:

> Your java code connects to 5671, which is typically the non-ssl port.

Er, *cough*, no it isn't - that would be 5672. Derp.

_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: shovel with authentization

Tomas Tulka
> Er, *cough*, no it isn't - that would be 5672. Derp.

with port 5672 a client is unable to connect
Reply | Threaded
Open this post in threaded view
|

Re: shovel with authentization

michaelklishin
2014-01-30 Tomas Tulka <[hidden email]>
with port 5672 a client is unable to connect

This means 

 * you have a non-standard RabbitMQ configuration
 * a firewall gets in the way
 * you are trying to use SSL over non-SSL port

the first two cases can be verified by connecting with telnet. The latter will result in error

_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: shovel with authentization

Tomas Tulka
> the first two cases can be verified by connecting with telnet. The latter will result in error
messages in RabbitMQ log.

it is verified, no error in any log.

everything works fine without using credentials in the java client > getting off setUsername and setPassword from java code, it starts to work (with SSL) -> no problem of SSL which in fact working fine, problem must be with authentization
Reply | Threaded
Open this post in threaded view
|

Re: shovel with authentization

Tomas Tulka
In reply to this post by Tim Watson-6
Regarding the port 5672: I used the same configuration as in the documentation:
http://www.rabbitmq.com/ssl.html

and it works fine. When the port in the conf is changed to 5672 I got error in the startup and the broker will never start:

{"init terminating in do_boot",{rabbit,failure_during_boot,{could_not_start,rabbit,{bad_return,{{rabbit,start,[normal,[]]},{'EXIT',{rabbit,failure_during_boot,{case_clause,{error,{already_started,<0.227.0>}}}}}}}}}}

This is confusing
Reply | Threaded
Open this post in threaded view
|

Re: shovel with authentization

michaelklishin
In reply to this post by Tomas Tulka

2014-01-30 Tomas Tulka <[hidden email]>
everything works fine without using credentials in the java client > getting
off setUsername and setPassword from java code, it starts to work (with SSL)
-> no problem of SSL which in fact working fine, problem must be with
authentization

If the problem is with authentication (credentials) or incorrect access rights, there must be errors in the log
about refused access.


_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: shovel with authentization

michaelklishin
In reply to this post by Tomas Tulka

2014-01-30 Tomas Tulka <[hidden email]>
{"init terminating in
do_boot",{rabbit,failure_during_boot,{could_not_start,rabbit,{bad_return,{{rabbit,start,[normal,[]]},{'EXIT',{rabbit,failure_during_boot,{case_clause,{error,{already_started,<0.227.0>}}}}}}}}}}

This is confusing

You cannot use the same port for both SSL and non-SSL. A non-SSL TCP listener is already running on 5672.

If you want SSL, use 5671.
--
MK

http://github.com/michaelklishin
http://twitter.com/michaelklishin

_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: shovel with authentization

Tomas Tulka
In reply to this post by michaelklishin
> If the problem is with authentication (credentials) or incorrect access rights

No error in logs. The thing is that I can create a connection with credentials (I tried a wrong pass and I was refused by an error -> ok) and I can send a message to the broker by such a connection, but I will never receive it. When no credentials -> send and receive as well successfully.
Reply | Threaded
Open this post in threaded view
|

Re: shovel with authentization

Tomas Tulka
In reply to this post by michaelklishin
> If you want SSL, use 5671.

alright, so I am using SSL and 5671 and it works fine. Only the auth problem (see my previous reply)
Reply | Threaded
Open this post in threaded view
|

Re: shovel with authentization

michaelklishin
In reply to this post by Tomas Tulka
2014-01-30 Tomas Tulka <[hidden email]>
When no credentials -> send and receive as well successfully.

There is no such thing as "no credentials" in RabbitMQ. If you specify no
credentials, then guest/guest is used.

I'm confused at this point, is it a client to node A connection that presumably
has silent authentication failures or node A to node B with Shovel over amqps?
--
MK

http://github.com/michaelklishin
http://twitter.com/michaelklishin

_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: shovel with authentication

Tomas Tulka
In reply to this post by Tomas Tulka
I have solved this strange issue!
Consider this declaration of a source broker:
        {declarations, [
                {'exchange.declare', [
                    {exchange, <<"MyQ-exchange">>},
                    {type, <<"direct">>},
                    auto_delete
                    ]},
                {'queue.declare', [
                    {queue, <<"MyQ">>},
                    {arguments,[{<<"x-expires">>, long, 15000}]}
                ]},
                {'queue.bind', [
                    {exchange, <<"MyQ-exchange">>},
                    {queue,    <<"MyQ">>}
                ]}

        ]}]},

Removing those two lines it started to work! So strange...
Reply | Threaded
Open this post in threaded view
|

Re: shovel with authentication

michaelklishin

2014-01-30 Tomas Tulka <[hidden email]>:
        {declarations, [
                {'exchange.declare', [
                    {exchange, <<"MyQ-exchange">>},
                    {type, <<"direct">>},
                    *auto_delete*
                    ]},
                {'queue.declare', [
                    {queue, <<"MyQ">>},
                    *{arguments,[{<<"x-expires">>, long, 15000}]}*
                ]},
                {'queue.bind', [
                    {exchange, <<"MyQ-exchange">>},
                    {queue,    <<"MyQ">>}
                ]}

        ]}]},

Removing those two lines it started to work! So strange...

This suggests that the queue already existed but with different arguments.

If Shovel doesn't handle it, it should at least complain in the log.
--
MK

http://github.com/michaelklishin
http://twitter.com/michaelklishin

_______________________________________________
rabbitmq-discuss mailing list
[hidden email]
https://lists.rabbitmq.com/cgi-bin/mailman/listinfo/rabbitmq-discuss
Reply | Threaded
Open this post in threaded view
|

Re: shovel with authentication

Tomas Tulka
> This suggests that the queue already existed but with different arguments.

Sure, for this test I changes the name of a queue to avoid this error...

Maybe the authorization and SSL deal badly with auto-deleting queues... donno
12