1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
|
Return-Path: <ick-discuss-bounces@ick.liw.fi>
X-Original-To: distix@pieni.net
Delivered-To: distix@pieni.net
Received: from yaffle.pepperfish.net (yaffle.pepperfish.net [88.99.213.221])
by pieni.net (Postfix) with ESMTPS id 71D0043D43
for <distix@pieni.net>; Sat, 9 Feb 2019 05:11:34 +0000 (UTC)
Received: from platypus.pepperfish.net (unknown [10.112.101.20])
by yaffle.pepperfish.net (Postfix) with ESMTP id 421C941123
for <distix@pieni.net>; Sat, 9 Feb 2019 05:11:34 +0000 (GMT)
Received: from ip6-localhost.nat ([::1] helo=platypus.pepperfish.net)
by platypus.pepperfish.net with esmtp (Exim 4.80 #2 (Debian))
id 1gsKvO-0005Pb-6f; Sat, 09 Feb 2019 05:11:34 +0000
Received: from [10.112.101.104] (helo=mx4.pepperfish.net)
by platypus.pepperfish.net with esmtps (Exim 4.80 #2 (Debian))
id 1gsKvN-0005PI-HG
for <ick-discuss@ick.liw.fi>; Sat, 09 Feb 2019 05:11:33 +0000
Received: from mail.steve.org.uk ([176.9.183.102] helo=ssh.steve.org.uk)
by mx4.pepperfish.net with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.89) (envelope-from <steve@steve.org.uk>) id 1gsKvJ-0004YT-Sc
for ick-discuss@ick.liw.fi; Sat, 09 Feb 2019 05:11:32 +0000
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
d=steve.org.uk; s=20150726; h=References:In-Reply-To:Message-ID:Date:Subject:
From:To:Sender:Reply-To:Cc:MIME-Version:Content-Type:
Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date:
Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id:
List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive;
bh=m1i47aSPFG4FuSMWk/eB35KNatqT8n+iV0+cGfJlMSg=; b=l3nIUq+xMBrtsP4nvSGhI68gU
hl8j82c0MfDrAeIkkDnKrKV/iBAbjFNyW33mtxBbzGkS+JQudcZ9s9EhOKhBfCVJ0QwZ6/VFKdMQA
TfmWg9xTu+01P6jwZjVI6CA5ahu7F1AvO8qQ/meDqBC9Q2goJocsJjrGXV9bAXwCAdl40=;
Received: from steve by ssh.steve.org.uk with local (Exim 4.89)
(envelope-from <steve@steve.org.uk>) id 1gsKvB-000148-OB
for ick-discuss@ick.liw.fi; Sat, 09 Feb 2019 05:11:21 +0000
To: ick-discuss@ick.liw.fi
From: Steve Kemp <steve@steve.org.uk>
Date: Sat, 09 Feb 2019 05:07:55 +0000
Message-ID: <1549688875.1308.1@ssh.steve.org.uk>
In-Reply-To: <5e2b278847b76dd0311d5050b3455b12b4dd3077.camel@liw.fi>
References: <5e2b278847b76dd0311d5050b3455b12b4dd3077.camel@liw.fi>
X-added-header: steve.org.uk
X-Pepperfish-Transaction: d941-f1af-588d-7a08
X-Spam-Score: -0.2
X-Spam-Score-int: -1
X-Spam-Bar: /
X-Scanned-By: pepperfish.net, Sat, 09 Feb 2019 05:11:32 +0000
X-Spam-Report: Content analysis details: (-0.2 points)
pts rule name description
---- ---------------------- --------------------------------------------------
-0.0 BAYES_20 BODY: Bayes spam probability is 5 to 20%
[score: 0.1998]
-0.1 DKIM_VALID_EF Message has a valid DKIM or DK signature from
envelope-from domain
-0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from
author's domain
-0.1 DKIM_VALID Message has at least one valid DKIM or DK signature
0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily
valid
X-ACL-Warn: message may be spam
X-Scan-Signature: bdc2be5dc634686df142fe367da3f314
Subject: Re: Plan for using Muck for the Ick controller
X-BeenThere: ick-discuss@ick.liw.fi
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: discussions about the ick CI system <ick-discuss-ick.liw.fi>
List-Unsubscribe: <https://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/ick-discuss-ick.liw.fi>,
<mailto:ick-discuss-request@ick.liw.fi?subject=unsubscribe>
List-Archive: <http://listmaster.pepperfish.net/pipermail/ick-discuss-ick.liw.fi>
List-Post: <mailto:ick-discuss@ick.liw.fi>
List-Help: <mailto:ick-discuss-request@ick.liw.fi?subject=help>
List-Subscribe: <https://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/ick-discuss-ick.liw.fi>,
<mailto:ick-discuss-request@ick.liw.fi?subject=subscribe>
Sender: ick-discuss-bounces@ick.liw.fi
Errors-To: ick-discuss-bounces@ick.liw.fi
> * The controller will reconstruct the whole build log when it's
> requested for a complete log, by fetching all log objects that refer
> to the specified build, and catenating the log output snippets in
> order of the sequence number.
Assume you're building a kernel, which might spit out 400,000 lines
of build-log. Assume each new update, due to buffering, or sizing,
is 10 lines.
To get the whole build-log you're going to need to:
* Get the first piece.
* Get the next child.
* Get the next child.
* Get the next child, 39,997 times more.
That seems like it will scale terrible. The only potential win I can
see here is imagining that you might want to view the output via a
brower and you'll probably only care about the LAST 100 lines, not the
FIRST. (i.e. The bit which will typically contain "built blah", or
"build failed".)
I'd be tempted to write the log locally, then upload as one massive
object when complete. Sure it might be big, but it's more efficient
to stream a 1Mb HTTP transfer than to keep repeating HTTP-calls to
get the next bit.
(For a web-case you can use AJAX to stream on-scroll. But for email
output you'll want the whole thing to send..)
Steve
--
https://www.steve.org.uk/
_______________________________________________
ick-discuss mailing list
ick-discuss@ick.liw.fi
https://listmaster.pepperfish.net/cgi-bin/mailman/listinfo/ick-discuss-ick.liw.fi
|
