Re: [Tools-implementation] Revisiting whether we should continue using Docker as we currently do.

Russ Housley <housley@vigilsec.com> Wed, 09 September 2020 20:02 UTC

Return-Path: <housley@vigilsec.com>
X-Original-To: tools-implementation@ietfa.amsl.com
Delivered-To: tools-implementation@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id F0CF13A0D54 for <tools-implementation@ietfa.amsl.com>; Wed, 9 Sep 2020 13:02:22 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.896
X-Spam-Level:
X-Spam-Status: No, score=-1.896 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, HTML_MESSAGE=0.001, SPF_HELO_NONE=0.001, SPF_NONE=0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id OCiA77N5AmYx for <tools-implementation@ietfa.amsl.com>; Wed, 9 Sep 2020 13:02:20 -0700 (PDT)
Received: from mail.smeinc.net (mail.smeinc.net [209.135.209.11]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id C43383A0B41 for <tools-implementation@ietf.org>; Wed, 9 Sep 2020 13:02:20 -0700 (PDT)
Received: from localhost (localhost [127.0.0.1]) by mail.smeinc.net (Postfix) with ESMTP id 6D7C6300B70 for <tools-implementation@ietf.org>; Wed, 9 Sep 2020 16:02:18 -0400 (EDT)
X-Virus-Scanned: amavisd-new at mail.smeinc.net
Received: from mail.smeinc.net ([127.0.0.1]) by localhost (mail.smeinc.net [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id AO-IjC7ZtYpb for <tools-implementation@ietf.org>; Wed, 9 Sep 2020 16:02:16 -0400 (EDT)
Received: from a860b60074bd.fios-router.home (pool-141-156-161-153.washdc.fios.verizon.net [141.156.161.153]) by mail.smeinc.net (Postfix) with ESMTPSA id 557A9300435; Wed, 9 Sep 2020 16:02:16 -0400 (EDT)
From: Russ Housley <housley@vigilsec.com>
Message-Id: <D4D723C2-93C5-45F7-9B05-92763D144DF7@vigilsec.com>
Content-Type: multipart/alternative; boundary="Apple-Mail=_EFDD6B93-6631-4491-9656-824ECF59CC91"
Mime-Version: 1.0 (Mac OS X Mail 12.4 \(3445.104.15\))
Date: Wed, 09 Sep 2020 16:02:17 -0400
In-Reply-To: <3ae28788-898a-de72-22b6-b0f036d1b23a@nostrum.com>
Cc: "tools-implementation@ietf.org" <tools-implementation@ietf.org>
To: Robert Sparks <rjsparks@nostrum.com>
References: <3ae28788-898a-de72-22b6-b0f036d1b23a@nostrum.com>
X-Mailer: Apple Mail (2.3445.104.15)
Archived-At: <https://mailarchive.ietf.org/arch/msg/tools-implementation/DfwRNGtIcYVrbr91IAPgagXpfgw>
Subject: Re: [Tools-implementation] Revisiting whether we should continue using Docker as we currently do.
X-BeenThere: tools-implementation@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Tools Implementation <tools-implementation.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/tools-implementation>, <mailto:tools-implementation-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/tools-implementation/>
List-Post: <mailto:tools-implementation@ietf.org>
List-Help: <mailto:tools-implementation-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/tools-implementation>, <mailto:tools-implementation-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 09 Sep 2020 20:02:23 -0000

One of the reasons that we went to containers was to isolate some python2 from python3.  Will we just use virtual environment for that?

Russ

> On Sep 9, 2020, at 12:21 PM, Robert Sparks <rjsparks@nostrum.com> wrote:
> 
> A few weeks ago, shortly after the yc.o team's work managed to crash the host they were working on through changing settings in a container, I proposed that we unroll the things we currently have in containers on production. At the time, Glen suggested we not do that. I'd like to ask the question again - I think we may have more to consider.
> 
> 1. I remain uneasy about docker's implementation on OpenSuse. The container crash above, the issues we've run into with containers locking (and sometimes causing processes talking to them like apache) to hang are suspicious. That we've not been able to pin down what's really going on suggests to me the issue is in a place we can't really look, inside docker's interstitial networking or filesystem abstraction code perhaps.
> 
> 2. Many of the containers we have (and in particular the one for the website) really need to be designed differently if they are going to remain deployed as containers. The amount of file-system mapping they do is not what the docker architects expect as a normal use-case. Mapping sockets in the way we do is also likely not something they focus on testing.
> 
> 3. Docker is making Glen uncomfortable and the benefit for him (operationally) of the containerization is not proportional to the extra problems it is bringing.
> 
> So I again suggest that we unroll for the production deploys, at least for now. I think we can unroll everything at this point, but there might still be a hitch in unrolling the trac instances. Henrik - could you remind me what our thinking was with respect to those?
> 
> I do plan to keep up the pressure to have containerized versions of these services - this isn't a call to abandon Docker - but I suggest we need to change how we're currently using it.
> 
> RjS
> 
> 
> -- 
> Tools-implementation mailing list
> Tools-implementation@ietf.org
> https://www.ietf.org/mailman/listinfo/tools-implementation