[Nemops-interest] Re: NEMOPS Workshop Report

Michael Richardson <mcr+ietf@sandelman.ca> Thu, 20 February 2025 19:37 UTC

Return-Path: <mcr+ietf@sandelman.ca>
X-Original-To: nemops-interest@ietfa.amsl.com
Delivered-To: nemops-interest@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id C4DEFC16940B; Thu, 20 Feb 2025 11:37:00 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.108
X-Spam-Level:
X-Spam-Status: No, score=-2.108 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01, URIBL_DBL_BLOCKED_OPENDNS=0.001, URIBL_ZEN_BLOCKED_OPENDNS=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=sandelman.ca
Received: from mail.ietf.org ([50.223.129.194]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id b112wW8MjCJG; Thu, 20 Feb 2025 11:36:57 -0800 (PST)
Received: from tuna.sandelman.ca (tuna.sandelman.ca [IPv6:2607:f0b0:f:3:216:3eff:fe7c:d1f3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-256) server-digest SHA256) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 1047FC1654FE; Thu, 20 Feb 2025 11:36:55 -0800 (PST)
Received: from localhost (localhost [127.0.0.1]) by tuna.sandelman.ca (Postfix) with ESMTP id 8082E1800D; Thu, 20 Feb 2025 14:36:54 -0500 (EST)
Received: from tuna.sandelman.ca ([127.0.0.1]) by localhost (localhost [127.0.0.1]) (amavis, port 10024) with LMTP id jckgSNeYeoGb; Thu, 20 Feb 2025 14:36:52 -0500 (EST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=sandelman.ca; s=mail; t=1740080212; bh=41v2uw3bMul0uMdWHq65xxUzTQC6pksGnXoOC9Mehv8=; h=From:To:Subject:In-Reply-To:References:Date:From; b=ROHSZiFN+vTljivC/9553b6xClyf89+AN5k/C9tWybj5wQ2A1BzH1ullzimgsQMah PPUhWwMQFz1bb88u6OjmzqhGfiK7NM2k8jT2Uehj85vx/T0nCxWIR680+Ptf/HCGx7 ndoT2Gbv0l67R78R18Fw3erWXrLXvrjDv3Cxewk5qchwHp7uAhn6KDf9VpUGZwXIou v/Rhf00qKqdObRuK03EGipmfn3aK8glTG3/5EOkJ+zFWWSfRR2Kp4DAANLhJ9rmjTb luPvbOggwt99jH2q1Zw0i6vPgLLuUy/XK3yjpRn1CpQvvmN76coOsr0ufULgYGpu1s Py2C24Mq+ygvw==
Received: from sandelman.ca (obiwan.sandelman.ca [209.87.249.21]) by tuna.sandelman.ca (Postfix) with ESMTP id 4E7591800C; Thu, 20 Feb 2025 14:36:52 -0500 (EST)
Received: from obiwan.sandelman.ca (localhost [IPv6:::1]) by sandelman.ca (Postfix) with ESMTP id 4AACC5DA; Thu, 20 Feb 2025 14:36:52 -0500 (EST)
From: Michael Richardson <mcr+ietf@sandelman.ca>
To: Dhruv Dhody <dd@dhruvdhody.com>, nemops-interest@iab.org, nemops-workshop-attendees@iab.org, architecture-discuss@iab.org
In-Reply-To: <CAP7zK5YV5s2-0jutfN0mg3CBUNbZgA4KJbkFdqFXvbEqphsj-Q@mail.gmail.com>
References: <CAP7zK5YV5s2-0jutfN0mg3CBUNbZgA4KJbkFdqFXvbEqphsj-Q@mail.gmail.com>
X-Mailer: MH-E 8.6+git; nmh 1.8+dev; GNU Emacs 28.2
X-Face: $\n1pF)h^`}$H>Hk{L"x@)JS7<%Az}5RyS@k9X%29-lHB$Ti.V>2bi.~ehC0;<'$9xN5Ub# z!G,p`nR&p7Fz@^UXIn156S8.~^@MJ*mMsD7=QFeq%AL4m<nPbLgmtKK-5dC@#:k
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="=-=-="; micalg="pgp-sha512"; protocol="application/pgp-signature"
Date: Thu, 20 Feb 2025 14:36:52 -0500
Message-ID: <30029.1740080212@obiwan.sandelman.ca>
Message-ID-Hash: M4WRJK6YGYAUEQTQ7E3LFWNE5UXH2SYG
X-Message-ID-Hash: M4WRJK6YGYAUEQTQ7E3LFWNE5UXH2SYG
X-MailFrom: mcr+ietf@sandelman.ca
X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header
X-Mailman-Version: 3.3.9rc6
Precedence: list
Subject: [Nemops-interest] Re: NEMOPS Workshop Report
List-Id: Next Era of Network Management Operations <nemops-interest.iab.org>
Archived-At: <https://mailarchive.ietf.org/arch/msg/nemops-interest/IUL9-UKbq24aZyev4Y_hENAq0Xo>
List-Archive: <https://mailarchive.ietf.org/arch/browse/nemops-interest>
List-Help: <mailto:nemops-interest-request@iab.org?subject=help>
List-Owner: <mailto:nemops-interest-owner@iab.org>
List-Post: <mailto:nemops-interest@iab.org>
List-Subscribe: <mailto:nemops-interest-join@iab.org>
List-Unsubscribe: <mailto:nemops-interest-leave@iab.org>

Thank you for the write up.

Dhruv Dhody <dd@dhruvdhody.com> wrote:
    > Wes and I have posted the initial version of the NEMOPS Workshop Report -

    > https://datatracker.ietf.org/doc/draft-iab-nemops-workshop-report/
    > https://www.ietf.org/archive/id/draft-iab-nemops-workshop-report-01.html

    > We’d appreciate your feedback, ideally on the nemops-interest@iab.org list
    > or directly to the authors. You can also submit a PR on GitHub:

}   Many operators
}   prioritize operational conferences over standards development
}   organizations (SDOs), such as RIPE, NANOG, APRICOT, LACNIC, AutoConn,
}   etc.
}
}   To address this, the IAB workshop's Program Committee (PC) planned
}   outreach initiatives to foster discussions and gather interest by
}   engaging with operators at these venues and conducting information/
}   requirement-gathering sessions.  Participants were encouraged to

Did we succeed in getting more operators to the workshop?
I observed the PC outreach at RIPE89, but I'm not sure it resulted in the
groundswell that was desired.

Yesterday, I walked a new (very young) network admin through adding a new
[not new to us, just been sitting in a box since 2019] switch CREDIL.org's
network monitoring tool... That is, we connected it to SNMP...
The level of idiosyncracy is great, caused in large part by
heterogeneous equipment which is definitely not new.
We (had to) fixed quite a number of things along the way, so it took about 5 hours.
This was the new person's first experience with Cisco-style router CLIs, and
she was really impressed with how useful completion was.  We used a Web
interface for a bit to add a static IPv6 for the control access, but the Web
interface *did the wrong thing*.  We never spoke of YANG.
We use tftp to backup configurations, which we then manually check into git.
("git app -p" is very good way to review what you actually did, and also to
learn what went wrong with the web interface attempt)

We'd love to use scp, but the security posture of it is almost worse.
We also never get ssh access to our switches to work consistently, so serial
consoles are mandatory as backup, and guess what: RESTCONF does not work
across that link.   RFC8994 would be nice.
(Yes, I have running code)

I don't think section 3.1.3 really articulates this well enough:
"Additionally, the lack of
 simple tools for smaller networks operating under tight timelines and
 budgets was emphasized."

this is just insufficient emphasis here.
I could think to replace all of section 3.1.x with:
  "Additionally, the lack of
   simple tools other than Net-SNMP for smaller networks operating under
   tight timelines, a mix of older and newer systems and very small budgets
   was emphasized."

Organizations slightly larger than CREDIL typically "solve" these problems
by buying all same-vendor equipment every ~10 years or so.  It comes with a
thick manual which one person reads, just before being lured away by higher
salary to be an Application Engineer for said vendor.  If they were rich at
the time, or the discount was large, they bought the management platform with
a pretty interface, but in practice no ability to do anything really useful.

section 3.2 seems to all be about problems with managing systems at larger
scale.  My complaint all along is that RESTCONF(YANG) does not scale.  Scale doesn't
mean "big", it means, something can work from smallest to largest scale. RESTCONF
doesn't work in the small.  One never starts a three switch network with
RESTCONF, and then use it as you grow.  One starts with a three hundred
switch network.

Lots of talk about open source, but not really any mention of how such a
thing could be funded.  That's really the problem.


}   1.  In network deployments, operations are typically at the bottom of
}       the ladder.  It's the most squeezed for time and resources.
}       Network engineers are not typically seasoned developers.

It's way way worse than that.  Most government and NGO network engineers are
prevented by policy (from their own organization!) from even having developer
tools on their desktop.  Getting access to equipment to do testing on (like
developing new tools) is essentially impossible in many places.

}   4.  It was suggested that other domains (e.g., K8N/automation) are
}       years ahead of the current network engineering stack.

k8n has useless networking.  It's "years ahead", in that it does a few things
autonomically, but does them wrong.  Managers are told the wrong way is "better"

}   5.  There was a point about navigating non-device-specific models
}       being difficult.  If understood correctly, the Network Engineer
}       knows the CLI command but has trouble grepping for it in YANG
}       modules defined by SDOs.

I can discover most anything in the CLI by hitting TAB/? enough.
One switches and routers which I've never touched or read the manual for.

--
]               Never tell me the odds!                 | ipv6 mesh networks [
]   Michael Richardson, Sandelman Software Works        |    IoT architect   [
]     mcr@sandelman.ca  http://www.sandelman.ca/        |   ruby on rails    [





--
Michael Richardson <mcr+IETF@sandelman.ca>   . o O ( IPv6 IøT consulting )
           Sandelman Software Works Inc, Ottawa and Worldwide