Re: [pim] Group-Specific Queries dropped by kernel

<mal.hubert@bt.com> Tue, 16 April 2019 08:07 UTC

Return-Path: <mal.hubert@bt.com>
X-Original-To: pim@ietfa.amsl.com
Delivered-To: pim@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 2482212034A for <pim@ietfa.amsl.com>; Tue, 16 Apr 2019 01:07:44 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.9
X-Spam-Level:
X-Spam-Status: No, score=-1.9 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id gi5OyJzO9RJ8 for <pim@ietfa.amsl.com>; Tue, 16 Apr 2019 01:07:40 -0700 (PDT)
Received: from smtpb1.bt.com (smtpb1.bt.com [213.121.32.99]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 0FFA312034D for <pim@ietf.org>; Tue, 16 Apr 2019 01:07:39 -0700 (PDT)
Received: from rew09926dag07c.domain1.systemhost.net (10.55.202.42) by BWP09926073.bt.com (10.50.151.96) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.1.1531.3; Tue, 16 Apr 2019 09:07:05 +0100
Received: from rew09926dag07b.domain1.systemhost.net (10.55.202.38) by rew09926dag07c.domain1.systemhost.net (10.55.202.42) with Microsoft SMTP Server (TLS) id 15.0.1395.4; Tue, 16 Apr 2019 09:07:36 +0100
Received: from rew09926dag07b.domain1.systemhost.net ([fe80::3597:bc94:cdb8:9d49]) by rew09926dag07b.domain1.systemhost.net ([fe80::3597:bc94:cdb8:9d49%12]) with mapi id 15.00.1395.000; Tue, 16 Apr 2019 09:07:37 +0100
From: <mal.hubert@bt.com>
To: <linus.luessing@c0d3.blue>
CC: <pim@ietf.org>
Thread-Topic: [pim] Group-Specific Queries dropped by kernel
Thread-Index: AdTxLB/StBdCEYp1RDa29yj4Bqm5LAArfG0AAGIIUFAAGJLmAAAZe4rA
Date: Tue, 16 Apr 2019 08:07:36 +0000
Message-ID: <3a24781640a04759936c57714f311729@rew09926dag07b.domain1.systemhost.net>
References: <40be04cccedb47b78e62771e4aeaabda@rew09926dag07b.domain1.systemhost.net> <20190413102000.GA2085@otheros> <65170f1824bb44cab5dde88bc16f7fc5@rew09926dag07b.domain1.systemhost.net> <20190415205036.GE2085@otheros>
In-Reply-To: <20190415205036.GE2085@otheros>
Accept-Language: en-GB, en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ms-exchange-transport-fromentityheader: Hosted
x-originating-ip: [10.55.202.242]
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
MIME-Version: 1.0
Archived-At: <https://mailarchive.ietf.org/arch/msg/pim/z8p4_CEwCxy_m8HBZ8TvTyKSrx8>
Subject: Re: [pim] Group-Specific Queries dropped by kernel
X-BeenThere: pim@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Protocol Independent Multicast <pim.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/pim>, <mailto:pim-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/pim/>
List-Post: <mailto:pim@ietf.org>
List-Help: <mailto:pim-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/pim>, <mailto:pim-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 16 Apr 2019 08:07:49 -0000

Linus,

Many thanks for this. We will do some more investigations. Maybe rewriting the source on ingress will work around it for now.

Thanks

Mal

-----Original Message-----
From: Linus Lüssing [mailto:linus.luessing@c0d3.blue] 
Sent: 15 April 2019 21:51
To: Hubert,MT,Mal,TLV2 R
Cc: pim@ietf.org
Subject: Re: [pim] Group-Specific Queries dropped by kernel

Hi Mal,

Thanks for the capture. That looks good indeed, tcpdump or
wireshark do not complain about the format of the query either.

I tested this with a veth pair and isolated network namespace on
Linux and used tcpreplay to replay your IGMP query:

Setup:
$ ip netns add testnet
$ ip link add veth0 type veth peer name veth1
$ ip link set netns testnet dev veth1
$ ip link set up dev veth0
$ ip netns exec testnet ip link set up dev veth1
$ ip netns exec testnet ip a a 192.168.47.2/24 dev veth1

socat, to have a multicast listener for 234.81.130:
$ ip netns exec testnet socat UDP4-RECVFROM:6666,ip-add-membership=234.81.130.133:192.168.47.2 -

tcpreplay:
A) $ tcpreplay-edit --srcipmap=0.0.0.0/0:192.168.47.1 --fixcsum -i veth0 ./mal-h-query.pcap
B) $ tcpreplay-edit --srcipmap=0.0.0.0/0:0.0.0.0 --fixcsum -i veth0 ./mal-h-query.pcap


On my laptop, running a 4.19.16 kernel provided by Debian variant
A), which rewrites the source address to 192.168.47.1, works, I
get a report back. While variant B), which keeps the 0.0.0.0
source address, does not and reproduces the issue you described.


Then I built a 5.1-rc5 kernel with some extra debug output - and
there both variants A) and B) worked...

---
$ ip netns exec testnet tcpdump -e -v -l -n -i veth1 igmp
tcpdump: listening on veth1, link-type EN10MB (Ethernet), capture size 262144 bytes
22:47:02.395743 00:16:fa:db:99:65 > 01:00:5e:51:82:85, ethertype IPv4 (0x0800), length 60: (tos 0xc0, ttl 1, id 0, offset 0, flags [none], proto IGMP (2), length 36, options (RA))
    0.0.0.0 > 234.81.130.133: igmp query v3 [max resp time 0.1s] [gaddr 234.81.130.133]
22:47:02.480801 9a:81:3e:46:f6:ef > 01:00:5e:00:00:16, ethertype IPv4 (0x0800), length 54: (tos 0xc0, ttl 1, id 0, offset 0, flags [DF], proto IGMP (2), length 40, options (RA))
    192.168.47.2 > 224.0.0.22: igmp v3 report, 1 group record(s) [gaddr 234.81.130.133 is_ex, 0 source(s)]
22:47:09.740288 00:16:fa:db:99:65 > 01:00:5e:51:82:85, ethertype IPv4 (0x0800), length 60: (tos 0xc0, ttl 1, id 0, offset 0, flags [none], proto IGMP (2), length 36, options (RA))
    192.168.47.1 > 234.81.130.133: igmp query v3 [max resp time 0.1s] [gaddr 234.81.130.133]
22:47:09.809405 9a:81:3e:46:f6:ef > 01:00:5e:00:00:16, ethertype IPv4 (0x0800), length 54: (tos 0xc0, ttl 1, id 0, offset 0, flags [DF], proto IGMP (2), length 40, options (RA))
    192.168.47.2 > 224.0.0.22: igmp v3 report, 1 group record(s) [gaddr 234.81.130.133 is_ex, 0 source(s)]
^C
4 packets captured
4 packets received by filter
0 packets dropped by kernel
---

Could you maybe check whether a 5.1-rc5 kernel makes a difference for
you, too? This is a bit strange because 4.19.16 and 5.1-rc5 are not
that far apart. I did not expect that.

Regards, Linus


On Mon, Apr 15, 2019 at 08:12:28AM +0000, mal.hubert@bt.com wrote:
> Linus,
> 
> Thanks for looking at this. Capture attached.
> 
> Mal
> 
> -----Original Message-----
> From: Linus Lüssing [mailto:linus.luessing@c0d3.blue] 
> Sent: 13 April 2019 11:20
> To: Hubert,MT,Mal,TLV2 R
> Cc: pim@ietf.org
> Subject: Re: [pim] Group-Specific Queries dropped by kernel
> 
> Hi Hubert,
> 
> On Fri, Apr 12, 2019 at 12:34:52PM +0000, mal.hubert@bt.com wrote:
> > Hey
> > 
> >  
> > 
> > Hopefully someone on here can answer this basic query for me. It would appear
> > that the Linux kernel by default will allow IGMPv3 "General Query" if it has a
> > source IP of 0.0.0.0 but will drop any "Group-Specific Queries" if the source
> > IP is 0..0.0.0. This doesn't appear to be specific to my implementation and
> > seems to be the implementation in the Linux Kernel.
> > 
> >  
> > 
> > Some questions about this, if someone would be so kind as to help.
> > 
> >  
> > 
> >  
> > 
> > 1)    Is this by design in the rfc3367 standard and would someone point out to
> > me where it is and enlighten me as to why this is the case.
> 
> I couldn't find anything like this in RFC3367. I searched for
> 0.0.0.0 and "zero" in this document, but these only seem to be
> mentioned for the IGMPv3 report source address and the group
> address for IGMP queries.
> 
> There is a "4.2.13. IP Source Addresses for Reports" but no
> equivalent in 4.1 for IGMP queries.
> 
> > 2)    Is this by design in the Kernel and would enlighten me as to why this is
> > the case.
> 
> Hm, for the Linux bridge IGMP snooping code at least I think we do not
> have such a restriction (or I keep missing it while rereading the code). Both
> an IGMPv3 "General Query" and "Group-Specific Queries" with a
> 0.0.0.0 source addres should be accepted here.
> 
> In the Linux IPv4 stack code I wasn't able to find such a restriction either.
> igmp_rcv() or igmp_heard_query() do not seem to have such a
> check? But I'm not that familiar with the IP stack code - do you
> know where this check should be?
> 
> 
> I do see a comment "Stop IGMP from 0.0.0.0 being accepted." (which
> seems wrong) in the changelog here:
> 
> https://elixir.bootlin.com/linux/v5.0/source/net/ipv4/igmp.c#L47
> 
> But as it came with the big git import in 2005 I'm unable to track
> with "git blame" etc. what Alan had actually changed in the code.
> 
> 
> Could you share a tcpdump capture of the IGMP queries your
> application is sending? Maybe "by coincidence" the IGMP or IP
> checksums are messed up only for the Group-Specific IGMP queries
> in your application or something like that?
> 
> Does Wireshark remark anything suspicious for your IGMP queries?
> 
> Regards, Linus