Re: Re[2]: Some proxy needs

["Nicolas Mailhot" <[e-mail]>]

Le Mar 10 avril 2012 11:23, Per Buer a écrit :
> Nicholas, list,
>
> On Tue, Apr 10, 2012 at 8:56 AM, Nicolas Mailhot
> <[e-mail]> wrote:
>>
>>
>> > So, if the proxy farm fails to hash incoming requests on source IP or
>> > target URL then this might happen.
>>
>> That breaks load balancing as soon as your network is big enough, with
>> different parts that get active at different points of the day.
>
> Sorry if I seem to miss the point, but why would it break? Are you worried
> that one point in the farm would get too hot?

I'm worried first that the activity distribution changes over time (so a
static table does not work), and second that any network-wide state (such as
an hashtable) is a contention point and potential spof. Making an individual
gateway smarter is simple (scaling the farms is just the matter of adding new
boxes). Making balancing smart and robust when some gateways or even some
physical processing sites can go down at any time is hard.

That's why direct browser-to-proxy communication will always be preferred and
more reliable than depending on a magical holistic hashtable balancer.
Round-robin is dumb but robust and stateless.

>> And anyway even if your solution was possible, you still get unhappy users
>> that serial refresh because they're not seeing initial progress in their web
>> clients
>
> If the loadbalancer was balancing on target URLs the request would end up at
> the same proxy. The proxy should be smart enough to coalesce the request
> into the ongoing one and this one could feed the user a couple of bytes so
> at the client understands that there is some progress on the download as
> some proxies have done for years.

Can't do when inspecting for malware. Malware inspection is not streamable,
you need to decompress recursively all the container formats (isos, zips,
jars, etc) to inspect the content before feeding the first byte to clients.
Just decompressing the container often requires the full file to have been
received proxy-side before starting the processing. The gateway could tell the
browser it's working on its request, but not give it part of the file.

-- 
Nicolas Mailhot