19 Zář
2022
19 Zář
'22
13:24
Hello everyone,
at AS50242 we experience problem with resolving connectivity.samsung.com.cn
We run two servers, each with 4 instances. Both servers have working
dual-stack (v4/v6).
knot-dnsutils/unknown,now 3.1.1-cznic.1 amd64 [installed]
knot-resolver-module-http/unknown,now 5.5.0-cznic.1 all [installed,automatic]
knot-resolver-release/unknown,now 1.9-1 all [installed]
knot-resolver/unknown,now 5.5.0-cznic.1 amd64 [installed]
Dnsviz shows problem reaching few IPv6 servers of .cn TLD via UDP. I
can not understand, why both of our servers response with SERVFAIL.
Any ideas how to troubleshoot more?
Thank you,
Blažej
19 Zář
19 Zář
13:29
Hi Blažej,
can you trace the query using this feature?
https://knot-resolver.readthedocs.io/en/stable/modules-http-trace.html#usin…
This will provide extremely detailed information on why Knot resolver
responds with SERVFAIL.
Best regards
Robert
On Mon, Sep 19, 2022 at 1:25 PM Blažej Krajňák <blazej.krajnak(a)gmail.com>
wrote:
Hello everyone,
at AS50242 we experience problem with resolving
connectivity.samsung.com.cn
We run two servers, each with 4 instances. Both servers have working
dual-stack (v4/v6).
knot-dnsutils/unknown,now 3.1.1-cznic.1 amd64 [installed]
knot-resolver-module-http/unknown,now 5.5.0-cznic.1 all
[installed,automatic]
knot-resolver-release/unknown,now 1.9-1 all [installed]
knot-resolver/unknown,now 5.5.0-cznic.1 amd64 [installed]
Dnsviz shows problem reaching few IPv6 servers of .cn TLD via UDP. I
can not understand, why both of our servers response with SERVFAIL.
Any ideas how to troubleshoot more?
Thank you,
Blažej
--
--
Robert Šefr
CTO
robert.sefr(a)whalebone.io | +420608737930 | www.whalebone.io
Whalebone, s.r.o.
Jezuitská 14/13, 602 00 Brno
Czech Republic
13:39
Hello Robert,
thanks for hint. The trace:
curl http://localhost:8453/trace/connectivity.samsung.com.cn
[iterat][4123608.00] 'connectivity.samsung.com.cn.' type 'A' new uid
was
assigned .01, parent uid .00
[cache ][4123608.01] => no NSEC* cached for zone: samsung.com.cn.
[cache ][4123608.01] => skipping zone: samsung.com.cn., NSEC, hash 0;new
TTL -123456789, ret -2
[cache ][4123608.01] => skipping zone: samsung.com.cn., NSEC, hash 0;new
TTL -123456789, ret -2
[zoncut][4123608.01] found cut: samsung.com.cn. (rank 010 return codes:
DS 1, DNSKEY 1)
[resolv][4123608.01] => NS is provably without DS, going insecure
[select][4123608.01] => id: '61652' choosing:
'dns1.esamsungchina.com.cn.'(a)'27.106.204.42#00053' with timeout 10000 ms
zone cut: 'samsung.com.cn.'
[resolv][4123608.01] => id: '61652' querying:
'dns1.esamsungchina.com.cn.'(a)'27.106.204.42#00053' zone cut:
'samsung.com.cn.'
qname: 'coNNECTIVitY.SAMSunG.com.CN.' qtype: 'A' proto: 'tcp'
[worker][4123608.01] => connecting to: '27.106.204.42#00053'
[worker][4123608.01] => connection to '27.106.204.42#00053' failed
(internal timeout)
[select][4123608.01] => id: '61652' noting selection error:
'dns1.esamsungchina.com.cn.'(a)'27.106.204.42#00053' zone cut:
'samsung.com.cn.'
error: 4 TCP_CONNECT_TIMEOUT
[iterat][4123608.01] 'connectivity.samsung.com.cn.' type 'A' new uid
was
assigned .02, parent uid .00
[select][4123608.02] => id: '65410' choosing to resolve AAAA: '
dns1.esamsungchina.com.cn.' zone cut: 'samsung.com.cn.'
[plan ][4123608.02] plan 'dns1.esamsungchina.com.cn.' type 'AAAA' uid
[4123608.03]
[iterat][4123608.03] 'dns1.esamsungchina.com.cn.' type 'AAAA' new uid
was assigned .04, parent uid .02
[cache ][4123608.04] => no NSEC* cached for zone: esamsungchina.com.cn.
[cache ][4123608.04] => skipping zone: esamsungchina.com.cn., NSEC,
hash 0;new TTL -123456789, ret -2
[cache ][4123608.04] => skipping zone: esamsungchina.com.cn., NSEC,
hash 0;new TTL -123456789, ret -2
[zoncut][4123608.04] found cut: esamsungchina.com.cn. (rank 010 return
codes: DS 1, DNSKEY 1)
[resolv][4123608.04] => NS is provably without DS, going insecure
[select][4123608.04] => id: '29392' choosing:
'dns.esamsungchina.com.cn.'(a)'27.106.204.41#00053' with timeout 10000 ms
zone cut: 'esamsungchina.com.cn.'
[resolv][4123608.04] => id: '29392' querying:
'dns.esamsungchina.com.cn.'(a)'27.106.204.41#00053' zone cut: '
esamsungchina.com.cn.' qname: 'DNS1.ESAMsUNgchiNa.coM.cN.' qtype:
'AAAA'
proto: 'tcp'
[worker][4123608.04] => connecting to: '27.106.204.41#00053'
[worker][4123608.04] => connection to '27.106.204.41#00053' failed
(internal timeout)
[select][4123608.04] => id: '29392' noting selection error:
'dns.esamsungchina.com.cn.'(a)'27.106.204.41#00053' zone cut: '
esamsungchina.com.cn.' error: 4 TCP_CONNECT_TIMEOUT
[iterat][4123608.04] 'dns1.esamsungchina.com.cn.' type 'AAAA' new uid
was assigned .05, parent uid .02
[cache ][4123608.05] => no NSEC* cached for zone: esamsungchina.com.cn.
[cache ][4123608.05] => skipping zone: esamsungchina.com.cn., NSEC,
hash 0;new TTL -123456789, ret -2
[cache ][4123608.05] => skipping zone: esamsungchina.com.cn., NSEC,
hash 0;new TTL -123456789, ret -2
[select][4123608.05] => id: '16382' choosing:
'dns1.esamsungchina.com.cn.'(a)'27.106.204.42#00053' with timeout 10000 ms
zone cut: 'esamsungchina.com.cn.'
[resolv][4123608.05] => id: '16382' querying:
'dns1.esamsungchina.com.cn.'(a)'27.106.204.42#00053' zone cut: '
esamsungchina.com.cn.' qname: 'DNs1.esamSUNgcHiNA.COM.cn.' qtype:
'AAAA'
proto: 'udp'
[select][4123608.05] => id: '16382' noting selection error:
'dns1.esamsungchina.com.cn.'(a)'27.106.204.42#00053' zone cut: '
esamsungchina.com.cn.' error: 1 QUERY_TIMEOUT
[worker][4123608.00] internal timeout for resolving the request has expired
[resolv][4123608.00] request failed, answering with empty SERVFAIL
[resolv][4123608.05] finished in state: 8, queries: 0, mempool: 114752 B
po 19. 9. 2022 o 13:29 Robert Šefr <robert.sefr(a)whalebone.io> napísal(a):
Hi Blažej,
can you trace the query using this feature?
https://knot-resolver.readthedocs.io/en/stable/modules-http-trace.html#usin…
This will provide extremely detailed information on why Knot resolver
responds with SERVFAIL.
Best regards
Robert
On Mon, Sep 19, 2022 at 1:25 PM Blažej Krajňák <blazej.krajnak(a)gmail.com>
wrote:
Hello everyone,
at AS50242 we experience problem with resolving
connectivity.samsung.com.cn
We run two servers, each with 4 instances. Both servers have working
dual-stack (v4/v6).
knot-dnsutils/unknown,now 3.1.1-cznic.1 amd64 [installed]
knot-resolver-module-http/unknown,now 5.5.0-cznic.1 all
[installed,automatic]
knot-resolver-release/unknown,now 1.9-1 all [installed]
knot-resolver/unknown,now 5.5.0-cznic.1 amd64 [installed]
Dnsviz shows problem reaching few IPv6 servers of .cn TLD via UDP. I
can not understand, why both of our servers response with SERVFAIL.
Any ideas how to troubleshoot more?
Thank you,
Blažej
--
--
Robert Šefr
CTO
robert.sefr(a)whalebone.io | +420608737930 | www.whalebone.io
Whalebone, s.r.o.
Jezuitská 14/13, 602 00 Brno
Czech Republic
--
13:55
So, the log shows that it won't connect even on TCP level. TCP is tried
first at this point due to cache containing information about these IPs
timing out (over UDP).
However, when I try from my network these IPs do reply, both over UDP
and TCP. I'd probably look into traceroute, etc.
--Vladimir
14:17
What I see, the problematic IPs 27.106.204.41 and 27.106.204.42 look
to be "open resolvers" with high timeout rate. I think they are
misconfigured and misused what causes the overal problems.
dig google.com @27.106.204.41
; <<>> DiG 9.16.15-Debian <<>> google.com @27.106.204.41
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 35456
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;google.com. IN A
;; ANSWER SECTION:
google.com. 60 IN A 93.46.8.90
;; Query time: 188 msec
;; SERVER: 27.106.204.41#53(27.106.204.41)
;; WHEN: Mon Sep 19 14:11:38 CEST 2022
;; MSG SIZE rcvd: 54
po 19. 9. 2022 o 13:55 Vladimír Čunát <vladimir.cunat(a)nic.cz> napísal(a):
So, the log shows that it won't connect even on TCP level. TCP is tried first at
this point due to cache containing information about these IPs timing out (over UDP).
However, when I try from my network these IPs do reply, both over UDP and TCP. I'd
probably look into traceroute, etc.
--Vladimir
17:09
Thanks to everyone.
It seems that our netblock is blocked and we do not receive responses
from 27.106.204.41 and 27.106.204.42.
Knot Resolver is acting correctly.
Blažej
po 19. 9. 2022 o 14:17 Blažej Krajňák <blazej.krajnak(a)gmail.com> napísal(a):
What I see, the problematic IPs 27.106.204.41 and 27.106.204.42 look
to be "open resolvers" with high timeout rate. I think they are
misconfigured and misused what causes the overal problems.
dig google.com @27.106.204.41
; <<>> DiG 9.16.15-Debian <<>> google.com @27.106.204.41
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 35456
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;google.com. IN A
;; ANSWER SECTION:
google.com. 60 IN A 93.46.8.90
;; Query time: 188 msec
;; SERVER: 27.106.204.41#53(27.106.204.41)
;; WHEN: Mon Sep 19 14:11:38 CEST 2022
;; MSG SIZE rcvd: 54
po 19. 9. 2022 o 13:55 Vladimír Čunát <vladimir.cunat(a)nic.cz> napísal(a):
>
> So, the log shows that it won't connect even on TCP level. TCP is tried first
at this point due to cache containing information about these IPs timing out (over UDP).
>
> However, when I try from my network these IPs do reply, both over UDP and TCP.
I'd probably look into traceroute, etc.
>
> --Vladimir
20 Zář
20 Zář
8:31
Hello Blažej,
just a short comment on this - such awkward blocks actually happen
frequently and they are challenging to troubleshoot. They usually affect
only particular network ranges, therefore tools such as dnviz may even
mislead you to a wrong direction.
The trace is extremely useful and provides solid proof of what is
happening. The usual learning is that Knot resolver returns the answer for
a solid reason, though it may not be obvious at a first glance.
Best regards
Robert
On Mon, Sep 19, 2022 at 5:10 PM Blažej Krajňák <blazej.krajnak(a)gmail.com>
wrote:
Thanks to everyone.
It seems that our netblock is blocked and we do not receive responses
from 27.106.204.41 and 27.106.204.42.
Knot Resolver is acting correctly.
Blažej
po 19. 9. 2022 o 14:17 Blažej Krajňák <blazej.krajnak(a)gmail.com>
napísal(a):
--
Robert Šefr
CTO
robert.sefr(a)whalebone.io | +420608737930 | www.whalebone.io
Whalebone, s.r.o.
Jezuitská 14/13, 602 00 Brno
Czech Republic
What I see, the problematic IPs 27.106.204.41 and 27.106.204.42 look
to be "open resolvers" with high timeout rate. I think they are
misconfigured and misused what causes the overal problems.
dig google.com @27.106.204.41
; <<>> DiG 9.16.15-Debian <<>> google.com @27.106.204.41
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 35456
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;google.com. IN A
;; ANSWER SECTION:
google.com. 60 IN A 93.46.8.90
;; Query time: 188 msec
;; SERVER: 27.106.204.41#53(27.106.204.41)
;; WHEN: Mon Sep 19 14:11:38 CEST 2022
;; MSG SIZE rcvd: 54
po 19. 9. 2022 o 13:55 Vladimír Čunát <vladimir.cunat(a)nic.cz>
napísal(a):
>
> So, the log shows that it won't connect even on TCP level. TCP is
tried
first at this point due to cache containing information about these
IPs timing out (over UDP).
>
> However, when I try from my network these IPs do reply, both over UDP
and
TCP. I'd probably look into traceroute, etc.
>
> --Vladimir
--
9:13
Yeah. Thanks Robert for hints.
Just to note for others, who will read the thread, I used this
workaround until the problem will be solved.
policy.add(policy.suffix(
policy.STUB('8.8.8.8', '1.1.1.1'),
{todname('samsung.com.cn')}))
ut 20. 9. 2022 o 8:32 Robert Šefr <robert.sefr(a)whalebone.io> napísal(a):
Hello Blažej,
just a short comment on this - such awkward blocks actually happen frequently and they
are challenging to troubleshoot. They usually affect only particular network ranges,
therefore tools such as dnviz may even mislead you to a wrong direction.
The trace is extremely useful and provides solid proof of what is happening. The usual
learning is that Knot resolver returns the answer for a solid reason, though it may not be
obvious at a first glance.
Best regards
Robert
On Mon, Sep 19, 2022 at 5:10 PM Blažej Krajňák <blazej.krajnak(a)gmail.com> wrote:
Thanks to everyone.
It seems that our netblock is blocked and we do not receive responses
from 27.106.204.41 and 27.106.204.42.
Knot Resolver is acting correctly.
Blažej
po 19. 9. 2022 o 14:17 Blažej Krajňák <blazej.krajnak(a)gmail.com> napísal(a):
--
Robert Šefr
CTO
robert.sefr(a)whalebone.io | +420608737930 | www.whalebone.io
Whalebone, s.r.o.
Jezuitská 14/13, 602 00 Brno
Czech Republic
--
What I see, the problematic IPs 27.106.204.41 and 27.106.204.42 look
to be "open resolvers" with high timeout rate. I think they are
misconfigured and misused what causes the overal problems.
dig google.com @27.106.204.41
; <<>> DiG 9.16.15-Debian <<>> google.com @27.106.204.41
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 35456
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;google.com. IN A
;; ANSWER SECTION:
google.com. 60 IN A 93.46.8.90
;; Query time: 188 msec
;; SERVER: 27.106.204.41#53(27.106.204.41)
;; WHEN: Mon Sep 19 14:11:38 CEST 2022
;; MSG SIZE rcvd: 54
po 19. 9. 2022 o 13:55 Vladimír Čunát <vladimir.cunat(a)nic.cz> napísal(a):
>
> So, the log shows that it won't connect even on TCP level. TCP is tried first
at this point due to cache containing information about these IPs timing out (over UDP).
>
> However, when I try from my network these IPs do reply, both over UDP and TCP.
I'd probably look into traceroute, etc.
>
> --Vladimir
--
582
days inactive
583
days old
knot-resolver-users@lists.nic.cz
7 comments
3 participants
participants (3)
-
Blažej Krajňák -
Robert Šefr -
Vladimír Čunát