Jan Schaumann

> making this is a valid URL: https://!$%:)(*&^@www.netmeister.org/blog/urls.html

Uh, no. "%:)" is not <"%" HEXDIG HEXDIG> nor is % allowed outside of that. (Although your browser will likely accept it)

> This includes spaces, and the following two URLs lead to the same file located in a directory that's named " ":
> https://www.netmeister.org/blog/urls/ /f
> https://www.netmeister.org/blog/urls/%20/f
> Your client may automatically percent-encode the space, but e.g., curl(1) lets you send the raw space:

Uh, no. Just because one of your clients is wrong and some servers allow it doesn't mean it's allowed by the spec.

In fact, the HTTP/1.1 RFC defers to RFC2396 for the meaning of <abs_path>: <path_segments> which begin with a /.

What is <path_segments>? A bunch of slash-delimited <segment>s.

What is <segment>? A bunch of <pchar> and maybe a semicolon.

What is <pchar>? <unreserved>, <escaped>, or some special characters (not including space).

What is <unreserved>? Letters, digits, and some special characters (not including space).

What is <escaped>? <"%" hex hex>.

Most HTTP clients and servers are pretty forgiving about what they accept, because other people do broken stuff, like sending them literal spaces. But that doesn't mean it's "allowed", that doesn't mean every server allows it, and that doesn't mean it's a good idea.

> That is, if your web server supports (and has enabled) user directories, and you submit a request for "~username": [it does stuff]

Uh, no.…

Well, essentially, all of the traditional text-only clients do (such as Mutt, which is what I use), I think Thunderbird does as well, beyond that, no clue, but there probably are others.

Traditionally, if you chose to reply to an email, the mail client would put the mail you were responding to into your editor, with > marks in front of every line. The only socially acceptable way[1] to write your answer was to insert your responses in between those quoted lines, without > marks, and to delete everything of the original mail that was irrelevant to establish the context for the reply.

That's how easy it is.

Usually, a good mail client would also color quoted lines so that it's easy to see and read only the response lines in a received email, so you'd only have to read the quoted text if you weren't sure about the context.

Clueless developers and users around the turn of the millenium then somehow didn't understand the purpose of providing you with a quoted version of the email you replied to, and started putting the reply above the quoted mail, thus ending up sending a copy of the mail they just received back to the person they received it from ... and nobody seemed to notice how idiotic an idea that actually is, and so it became sortof the new norm in large parts of the internet to attach large blobs of completely useless and often close to unreadable text to every mail, up to the point where some people even invented justifications for the behaviour that mostly tend to…

Well... There's a more serious first start browser comparison by netmeister.org [0] which shows that 4 downloads are made.

I downloaded and extracted the files. They look like helpers or partials for Brave internal extensions.

All of them include manifest files with their names:

- 1_0_14: "Brave HTTPS Everywhere Updater extension". Contains a 1MB ZIPped database of https domains.

- 1_0_21: "Brave NTP sponsored images component". Contains three photos (to display in their new tab probably).

- 1_0_22: "Brave Local Data Files Updater extension". Seems to contain whitelists and blacklists for extensions, autoplay, referers, trackers, etc.

- 1_0_498: "Brave Ad Block Updater extension". Contains a 2.4 MB filter list for their adblocker implementation.

Nothing seems to be harmful at all. This mechanism is used by almost all Chrome/Chromium based browsers to update their internal extensions and components.

But, if the poster cares about backdoors... Well, every major browser out there has features that could be used to backdoor their users like Firefox Telemetry Experiments (which download xpi files) and Chrome Components. They also can change properties at will unless its disabled (via flags, about:config, recompiling, etc).

Note: I'm a Vivaldi and Chromium user. I only use Brave with iOS which is kind of a different beast (since everything has to be implemented on top of iOS provided WebKit) since it somehow blocks ads better than stock Safari with AdGuard filters. For…

That's an excellent question! When I was writing that I was pretty deep in DNS code and standards I think I just understood that as a rule of DNS. Or at the least, I believed I understood. Looking back a decade later I can't find any standard which supports my statement. However lots of other secondary sources seem to agree with me.

"In these situations, the client needs to re-transmit over TCP for which the size limit is 64000 bytes" [0]

"The first response via UDP is, no surprise, truncated, so we retry via TCP. But now the DNS result delivered via TCP is also truncated! That is, the DNS server has determined that the result will not fit into the maximum response size. Why is that?

"Our payload is 4096 * 16 = 65536 bytes RDATA, which should fit into the DNS packet, which uses a two byte RDLENGTH field. But we also need to again account for the overhead noted above: 12 bytes DNS header, 36 bytes for the query, 11 bytes additional records, and 16 bytes for each A record, yielding (4096 * 16) + 12 + 36 + 11 = 65595 bytes in total. And the maximum size of an IP packet is 16 bit (via the IPv4 total length / IPv6 payload length). And that is now our limiting factor: 65536 bytes for DNS overhead + payload."

My best guess is that DNS just makes the assumption that everything needs to fit in a single IP packet and it doesn't care if it is UDP or TCP.

It is worth investigating if you have spare time.

[0]:…

I just tested this with dnsmasq[1] and can confirm it works.

It's good to know that there is a method but it looks a bit strange imo (see this article[2]):

>> Now this seems a bit weird: DoH to a public resolver seemed intended to defeat an untrusted local resolver, but an untrusted local resolver can disable the use of DoH to the public resolver in this fashion. In fact, if you have content filtering in place on your DNS resolver, Firefox will also use that as a metric to decide whether or not to enable DoH to the default public resolver.

>> In other words: if the browser detects that one of the things that DoH would help protect against (censorship / content filtering) is in place, then it will not enable DoH. ?? Well, for you as the network provider, that's good news -- you remain in charge, as so often, it boils down to having to trust the things you have to trust -- if you (the end user) don't trust your network provider, you can't use their services to bootstrap a trusted environment.

[1] just add the line for the canary to /etc/dnsmasq.conf:

address=/use-application-dns.net/

[2] https://www.netmeister.org/blog/doh-dot-dnssec.html

> Although there is: https://www.netmeister.org/blog/urls.html

I think the section named "Pathname" is wrong. It describes the path of an URL as if every server was Apache serving static files with its default configuration. It should describe how the path is converted into a HTTP request.

For instance, the article states that "all of these go to the same place : https://example.org https://example.org/ https://example.org// https://example.org//////////////////". That's wrong. A web client send a distinct HTTP request for each case, e.g starting with `GET // HTTP/1.1`. So the server will receive distinct paths. The assertion of "going to the same place" makes no sense in the general case.