Re: [quicwg/base-drafts] Python linting script (#1032)

Martin Thomson <notifications@github.com> Mon, 01 January 2018 23:17 UTC

Return-Path: <bounces+848413-a050-quic-issues=ietf.org@sgmail.github.com>
X-Original-To: quic-issues@ietfa.amsl.com
Delivered-To: quic-issues@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id ED91412426E for <quic-issues@ietfa.amsl.com>; Mon, 1 Jan 2018 15:17:54 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.03
X-Spam-Level:
X-Spam-Status: No, score=-2.03 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_MSPIKE_H4=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001, T_RP_MATCHES_RCVD=-0.01] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=github.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id HyMGCGZnkavI for <quic-issues@ietfa.amsl.com>; Mon, 1 Jan 2018 15:17:53 -0800 (PST)
Received: from o8.sgmail.github.com (o8.sgmail.github.com [167.89.101.199]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id E6DB4124D37 for <quic-issues@ietf.org>; Mon, 1 Jan 2018 15:17:52 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=github.com; h=from:reply-to:to:cc:in-reply-to:references:subject:mime-version:content-type:content-transfer-encoding:list-id:list-archive:list-post:list-unsubscribe; s=s20150108; bh=Rc2MSbU/VxRpEfagMjm+t25v6C4=; b=wR3STMuyvSmHaf3G YI/KmfOvcPTYEsk18no4VEQtpRifwH+TUkJ0s1wGMbx+5i3p2KaCsPPbOtUVwx6f KgSBFLTv9P7Di4hpXmZjgq00YKHzwOqAkLeiJxHa1hGuMPl3FASAqYqLsjGr+v5I c49bqeXHd5uv3G/a/3JHl/YIOZE=
Received: by filter0416p1mdw1.sendgrid.net with SMTP id filter0416p1mdw1-18501-5A4AC19F-3 2018-01-01 23:17:51.168527193 +0000 UTC
Received: from github-smtp2a-ext-cp1-prd.iad.github.net (github-smtp2a-ext-cp1-prd.iad.github.net [192.30.253.16]) by ismtpd0004p1iad1.sendgrid.net (SG) with ESMTP id erkxZhezTASkj_xMDC-BDA for <quic-issues@ietf.org>; Mon, 01 Jan 2018 23:17:51.152 +0000 (UTC)
Date: Mon, 01 Jan 2018 23:17:51 +0000
From: Martin Thomson <notifications@github.com>
Reply-To: quicwg/base-drafts <reply+0166e4ab6b55b798bc32441ef06196d96c7fc6606510fb1292cf000000011662839e92a169ce10f35235@reply.github.com>
To: quicwg/base-drafts <base-drafts@noreply.github.com>
Cc: Subscribed <subscribed@noreply.github.com>
Message-ID: <quicwg/base-drafts/pull/1032/review/86071151@github.com>
In-Reply-To: <quicwg/base-drafts/pull/1032@github.com>
References: <quicwg/base-drafts/pull/1032@github.com>
Subject: Re: [quicwg/base-drafts] Python linting script (#1032)
Mime-Version: 1.0
Content-Type: multipart/alternative; boundary="--==_mimepart_5a4ac19f5842_3c93fa16355ef3024665cb"; charset="UTF-8"
Content-Transfer-Encoding: 7bit
Precedence: list
X-GitHub-Sender: martinthomson
X-GitHub-Recipient: quic-issues
X-GitHub-Reason: subscribed
X-Auto-Response-Suppress: All
X-GitHub-Recipient-Address: quic-issues@ietf.org
X-SG-EID: l64QuQ2uJCcEyUykJbxN122A6QRmEpucztpreh3Pak2Mz8GEdD3W4yn/OQQn0HjanfOX2IG9WZVCxo 0uoX1/Q/BkLV5dd6HQZ13gnAg9A1SzfEuukHa8JSzOPBOkSt76BsoK/F05FGa9Pq7wfJRw0BFrjz4p 8531nMOA3jWSbm6co/IQ4s5enl5LZRW8NR2CvEgakhbfHoYKTEf2W5/bSa5zEQUrXXArtGEO+by1QJ g=
Archived-At: <https://mailarchive.ietf.org/arch/msg/quic-issues/DgJcSAFn2JRnLTjv-o39dvLZIC4>
X-BeenThere: quic-issues@ietf.org
X-Mailman-Version: 2.1.22
List-Id: Notification list for GitHub issues related to the QUIC WG <quic-issues.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/quic-issues>, <mailto:quic-issues-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/quic-issues/>
List-Post: <mailto:quic-issues@ietf.org>
List-Help: <mailto:quic-issues-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/quic-issues>, <mailto:quic-issues-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 01 Jan 2018 23:17:55 -0000

martinthomson commented on this pull request.

How fast is it?  It looks generally good though.

> @@ -0,0 +1,64 @@
+#!/usr/bin/env python3
+
+import sys,getopt,re
+
+def main(argv):
+    inputfile = ''
+    maxLineLength = 80
+    maxFigureLineLength = 65
+    foundError = False
+
+    try:
+        opts,args = getopt.getopt(argv,"i:lf")

Doesn't this need to be `"i:l:f:"`?  I see that you haven't tested these so...

> @@ -0,0 +1,64 @@
+#!/usr/bin/env python3

can you run autopep8 over this?

> +        elif opt == "-f":
+            maxFigureLineLength = arg
+
+    insideFigure = False
+    beforeAbstract = True
+    with open(inputfile,'U') as draft:
+        linecounter = 1
+        lines = draft.readlines()
+        for line in lines:
+            line = line.rstrip('\r\n')
+            linenumber = linecounter
+            linecounter += 1
+
+            ## Skip everything before abstract
+            if beforeAbstract:
+                matchObj = re.match('^--- abstract',line)

you can compile regexes that you run multiple times so that they run more quickly:

```py
abstract = re.compile('^--- abstract')
# ...
matchObj = abstract.match(line)
```



> +            ## Skip everything before abstract
+            if beforeAbstract:
+                matchObj = re.match('^--- abstract',line)
+                if matchObj:
+                    beforeAbstract = False
+                continue
+
+            ## Skip tables
+            matchObj = re.match('^\s*\|',line)
+            if matchObj:
+                continue
+
+            ## Toggle figure state
+            matchObj = re.match('^[~`]{3,}',line)
+            if matchObj:
+                insideFigure = False if insideFigure else True

```py
insideFigure = not insideFigure
```

> +        linecounter = 1
+        lines = draft.readlines()
+        for line in lines:
+            line = line.rstrip('\r\n')
+            linenumber = linecounter
+            linecounter += 1
+
+            ## Skip everything before abstract
+            if beforeAbstract:
+                matchObj = re.match('^--- abstract',line)
+                if matchObj:
+                    beforeAbstract = False
+                continue
+
+            ## Skip tables
+            matchObj = re.match('^\s*\|',line)

Use `'^\s*(?:\||{:)'` here so that you skip captions as well.

> +            if matchObj:
+                insideFigure = False if insideFigure else True
+                continue
+
+            ## Check length
+            length = len(line)
+            limit = maxFigureLineLength if insideFigure else maxLineLength
+            if length > limit:
+                foundError = True
+                print("{0}: Line is {1} characters; limit is {2}".format(linenumber,length,limit))
+                print(line)
+
+    sys.exit( 1 if foundError else 0)
+
+if __name__ == "__main__":
+   main(sys.argv[1:])

No EOL.

> @@ -16,13 +16,5 @@ latest:: lint
 .PHONY: lint
 lint::
 	@err=0; for f in draft-*.md ; do \
-	  if cat "$$f" | (l=0; while read -r a; do l=$$(($$l + 1)); echo -E "$$l:$$a"; done) | \
-	     sed -e '1,/--- abstract/d;/^[0-9]*: *|/d' | tr -d '\r' | grep '^[0-9]*:.\{81\}'; then \
-	    echo "$$f contains a line with >80 characters"; err=1; \
-	  fi; \
-	  if cat "$$f" | (l=0; while read -r a; do l=$$(($$l + 1)); echo -E "$$l:$$a"; done) | \
-	     sed -e '/^[0-9]*:~~~/,/^[0-9]*:~~~/p;/^[0-9]*:```/,/^[0-9]*:```/p;d' | \
-	     tr -d '\r' | grep '^[0-9]*:.\{66\}'; then \
-	    echo "$$f contains a figure with >65 characters"; err=1; \
-	  fi; \
+	  ./.lint.py -i $$f || err=1; \

Rather than iterate here, could you instead iterate in the script?  It would be more natural to drop the `-i` thing and iterate over the remaining arguments that getopt doesn't consume.

> @@ -0,0 +1,64 @@
+#!/usr/bin/env python3
+
+import sys,getopt,re
+
+def main(argv):
+    inputfile = ''
+    maxLineLength = 80
+    maxFigureLineLength = 65
+    foundError = False
+
+    try:
+        opts,args = getopt.getopt(argv,"i:lf")
+    except getopt.GetoptError:
+        print('.lint.py -i <input_file> [-l <line-length>] [-f <figure-line-length>]')

sys.stderr.write('...\n') for errors

-- 
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/quicwg/base-drafts/pull/1032#pullrequestreview-86071151