Long /etc/hosts entries lead to unbearably slow resolution

Originator:mhaecker
Number:rdar://24237290 Date Originated:19-jan-2016
Status:open Resolved:no
Product:OS X Product Version:El Capitan
Classification:Hang Reproducible:Always
 
Summary:
Entries with long lines in /etc/hosts lead to very slow host resolution.

For example an entry like this:

123.123.123.123 long.list of.host.names at.least.20.com ... example.com

is really slow when asking for example for 
curl --head http://example.com/somefile.png

Steps to Reproduce:
1. Add an entry to /etc/hosts like this: 123.123.123.123 long.list of.host.names at.least.20.com ... example.com
2. Request a file from one of the hosts there, in my example it is curl --head http://example.com/somefile.png
3. watch the slowness

Expected Results:
It should be _FAST_. No Network communication is required to resolve this.

Actual Results:
It is _SLOW_ as molasses. Interestingly this doesn't seem to happen for all urls that I ask for, and I haven't yet found the pattern.

Version:
10.11.2 (15C50)

Notes:
A workaround of sorts is to split long lines like this into something like this:

123.123.123.123 foo bar baz
123.123.123.123 foo.bar bar.bar baz.bar
123.123.123.123 foo.baz bar.bz baz.baz

which is much, much faster, if you ensure that every line has at most 10 entries.

BUT: why the heck do I have to deal with this?

Also I'm not the first person to find this problem. See: http://stackoverflow.com/questions/10064581/how-can-i-eliminate-slow-resolving-loading-of-localhost-virtualhost-a-2-3-secon

Configuration:
OS X El Capitan on a mac book pro from 2012

Comments

my reply

Dear engineers. I am not sure I understand you correctly? Why on earth would an /etc/hosts entry only have 10 allowed aliases? The man page doesn't mention anything like this, BSD and Linux systems don't behave like this, we build etc. hosts entries automatically for server tests with servers that have many many webpages that we want to point to testing boxes before we release changes into production.

Sure I can build that into the automation to generate these /etc/hosts entries so it has a special case for OS X - but why on earth? I cannot imagine the reason for this drastic slowdown to be something that takes longer than to reply to this bug report twice to solve.

If you seriously consider max 10 entries to be allowed in /etc/hosts per line, this needs to a) be documented and b) raise an error if it is violated so the user has a chance to actually find this problem. Just slowing down exponentially when there are more entries doesn't help find the problem and leads to hours wasted debugging this problem for every poor soul stumbling about this.

So why not invest the 1-2 hours to actually fix this and be done with it?

So, yes I do have a workaround, but man is it embarrassing to to tell my linux coworkers what the problem is and what I have to do to fix it.

Is the issue resolved for me? Not till you either update the documentation and add a visible error condition once this happens or fix this to not be slow as molasse as soon as you have more entries.

Be user friendly and do something so users don't waste time chasing the reason for these slowdowns.

By mhaecker at Feb. 8, 2016, 10:52 a.m. (reply...)

Feedback from apple:

Engineering has provided the following feedback regarding this issue:

This is expected behavior since the max no. of aliases in /etc/hosts file allowed is 10. Having more than 10 aliases in /etc/hosts file does not seem to be practical.

Does the below not work for your use case? Change your /etc/hosts file to have multiple entries like this below for it to work 1.2.3.4 bar1 bar2 bar3 bar4 bar5 bar6 bar7 bar8 bar9 bar10 1.2.3.4 bar11 bar12 bar13 bar14 bar15 bar16 bar17 bar18 bar19

Please let us know whether the issue is resolved for you by updating your bug report.

By mhaecker at Feb. 8, 2016, 10:52 a.m. (reply...)

Please note: Reports posted here will not necessarily be seen by Apple. All problems should be submitted at bugreport.apple.com before they are posted here. Please only post information for Radars that you have filed yourself, and please do not include Apple confidential information in your posts. Thank you!